Berkeley Notes
HomeResources
Search
⌃K
Links
Comment on page

Introduction to Probability

Definition 1

A probability space is a triple
(Ω,F,P)(\Omega, \mathcal{F}, P)
 where
Ω\Omega
 is a set of objects called the sample space,
F\mathcal{F}
 is a family of subsets of
Ω\Omega
 called events, and the probability measure
P:F→[0,1]P:\mathcal{F}\rightarrow [0,1]
.
One key assumption we make is that
F\mathcal{F}
 is a
σ\sigma
-algebra containing
Ω\Omega
, meaning that countably many complements, unions, and intersections of events in
F\mathcal{F}
 are also events in
F\mathcal{F}
. The probability measure
PP
 must obey Kolmogorov’s Axioms.
  1. 1.
    ​
    ∀A∈F, P(A)≥0\forall A \in \mathcal{F},\ P(A) \geq 0
    ​
  2. 2.
    ​
    P(Ω)=1P(\Omega) = 1
    ​
  3. 3.
    If
    A1,A2,⋯∈FA_1, A_2, \cdots\in \mathcal{F}
     and
    ∀i≠j, Ai⋂Aj=∅\forall i\ne j,\ A_i\bigcap A_j=\emptyset
    , then
    P(⋃i≥1Ai)=∑i≥1P(Ai)P\left(\bigcup_{i\geq 1}A_i\right) = \sum_{i\geq1}P(A_i)
    ​
We choose
Ω\Omega
 and
F\mathcal{F}
 to model problems in a way that makes our calculations easy.

Theorem 1

​
P(Ac)=1−P(A)P(A^c) = 1 - P(A)
​

Theorem 2 (Inclusion-Exclusion Principle)

​
P(⋃i=1nAi)=∑k=1n(−1)k+1(∑1≤i1<⋯<ik≤nP(Ai1∩⋯∩Aik))P\left( \bigcup_{i=1}^{n}A_i \right) = \sum_{k=1}^{n}(-1)^{k+1}\left( \sum_{1\leq i_1<\cdots<i_k\leq n} P(A_{i_1}\cap \cdots \cap A_{i_k}) \right)
​

Theorem 3 (Law of Total Probability)

If
A1,A2,⋯A_1, A_2, \cdots
 partition
Ω\Omega
 (i.e
AiA_i
 are disjoint and
∪Ai=Ω\cup A_i = \Omega
), then for event
BB
,
​
P(B)=∑iP(B∩Ai)P(B) = \sum_iP(B\cap A_i)
​

Conditional Probability

Definition 2

If
BB
 is an event with
P(B)>0P(B)>0
, then the conditional probability of
AA
 given
BB
 is
​
P(A∣B)=P(A∩B)P(B)P(A|B) = \frac{P(A\cap B)}{P(B)}
​
Intuitively, conditional probabilty is the probability of event
AA
 given that event
BB
 has occurred. In terms of probability spaces, it is as if we have taken
(Ω,F,P)(\Omega, \mathcal{F}, P)
 and now have a probabilty measure
P(⋅∣C)P(\cdot|C)
 belonging to the space
(Ω,F,P(⋅∣C))(\Omega, \mathcal{F}, P(\cdot|C))
.

Theorem 4 (Bayes Theorem)

​
P(A∣B)=P(B∣A)P(A)P(B)P(A|B) = \frac{P(B|A)P(A)}{P(B)}
​

Independence

Definition 3

Events
AA
 and
BB
 are independent if
P(A∩B)=P(A)P(B)P(A\cap B) = P(A)P(B)
​
If
P(B)>0P(B)>0
, then
A,BA, B
 are independent if and only if
P(A∣B)=P(A)P(A|B) = P(A)
. In other words, knowing
BB
 occurred gave no extra information about
AA
.

Definition 4

If
A,B,CA,B,C
 with
P(C)>0P(C)>0
 satisfy
P(A∩B∣C)=P(A∣C)P(B∣C)P(A\cap B|C) = P(A|C)P(B|C)
, then
AA
 and
BB
 are conditionally independent given
CC
.
Conditional independence is a special case of independence where
AA
 and
BB
 are not necessarily independent in the original probability space which has the measure
PP
, but are independent in the new probability space conditioned on
CC
 with the measure
P(⋅∣C)P(\cdot|C)
.
Previous
EECS126
Next
Random Variables and their Distributions
Last modified 2yr ago