Introduction to Probability

Definition 1

A probability space is a triple

$(\Omega, \mathcal{F}, P)$

where $\Omega$

is a set of objects called the sample space, $\mathcal{F}$

is a family of subsets of $\Omega$

called events, and the probability measure $P:\mathcal{F}\rightarrow [0,1]$

.One key assumption we make is that **Kolmogorov’s Axioms**.

$\mathcal{F}$

is a $\sigma$

-algebra containing $\Omega$

, meaning that countably many complements, unions, and intersections of events in $\mathcal{F}$

are also events in $\mathcal{F}$

. The probability measure $P$

must obey - 1.$\forall A \in \mathcal{F},\ P(A) \geq 0$
- 2.$P(\Omega) = 1$
- 3.If$A_1, A_2, \cdots\in \mathcal{F}$and$\forall i\ne j,\ A_i\bigcap A_j=\emptyset$, then$P\left(\bigcup_{i\geq 1}A_i\right) = \sum_{i\geq1}P(A_i)$

We choose

$\Omega$

and $\mathcal{F}$

to model problems in a way that makes our calculations easy.Theorem 1

$P(A^c) = 1 - P(A)$

Theorem 2 (Inclusion-Exclusion Principle)

$P\left( \bigcup_{i=1}^{n}A_i \right) = \sum_{k=1}^{n}(-1)^{k+1}\left( \sum_{1\leq i_1<\cdots<i_k\leq n} P(A_{i_1}\cap \cdots \cap A_{i_k}) \right)$

Theorem 3 (Law of Total Probability)

If

$A_1, A_2, \cdots$

partition $\Omega$

(i.e $A_i$

are disjoint and $\cup A_i = \Omega$

), then for event $B$

,

$P(B) = \sum_iP(B\cap A_i)$

Conditional Probability

Definition 2

If

$B$

is an event with $P(B)>0$

, then the conditional probability of $A$

given $B$

is

$P(A|B) = \frac{P(A\cap B)}{P(B)}$

Intuitively, conditional probabilty is the probability of event

$A$

given that event $B$

has occurred. In terms of probability spaces, it is as if we have taken $(\Omega, \mathcal{F}, P)$

and now have a probabilty measure $P(\cdot|C)$

belonging to the space $(\Omega, \mathcal{F}, P(\cdot|C))$

.Theorem 4 (Bayes Theorem)

$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

Independence

Definition 3

Events

$A$

and $B$

are independent if $P(A\cap B) = P(A)P(B)$

If

$P(B)>0$

, then $A, B$

are independent if and only if $P(A|B) = P(A)$

. In other words, knowing $B$

occurred gave no extra information about $A$

.Definition 4

If

$A,B,C$

with $P(C)>0$

satisfy $P(A\cap B|C) = P(A|C)P(B|C)$

, then $A$

and $B$

are conditionally independent given $C$

.Conditional independence is a special case of independence where

$A$

and $B$

are not necessarily independent in the original probability space which has the measure $P$

, but are independent in the new probability space conditioned on $C$

with the measure $P(\cdot|C)$

.Last modified 8mo ago

Copy link