# Introduction to Probability

{% hint style="info" %}
### Definition 1

A probability space is a triple $$(\Omega, \mathcal{F}, P)$$ where $$\Omega$$ is a set of objects called the sample space, $$\mathcal{F}$$ is a family of subsets of $$\Omega$$ called events, and the probability measure $$P:\mathcal{F}\rightarrow [0,1]$$.
{% endhint %}

One key assumption we make is that $$\mathcal{F}$$ is a $$\sigma$$-algebra containing $$\Omega$$, meaning that countably many complements, unions, and intersections of events in $$\mathcal{F}$$ are also events in $$\mathcal{F}$$. The probability measure $$P$$ must obey **Kolmogorov’s Axioms**.

1. $$\forall A \in \mathcal{F},\ P(A) \geq 0$$
2. $$P(\Omega) = 1$$
3. If $$A_1, A_2, \cdots\in \mathcal{F}$$ and $$\forall i\ne j,\ A_i\bigcap A_j=\emptyset$$, then $$P\left(\bigcup_{i\geq 1}A_i\right) = \sum_{i\geq1}P(A_i)$$

We choose $$\Omega$$ and $$\mathcal{F}$$ to model problems in a way that makes our calculations easy.

{% hint style="info" %}
### Theorem 1

$$P(A^c) = 1 - P(A)$$
{% endhint %}

{% hint style="info" %}
### Theorem 2 (Inclusion-Exclusion Principle) <a href="#theorem-2" id="theorem-2"></a>

$$P\left( \bigcup_{i=1}^{n}A_i \right) = \sum_{k=1}^{n}(-1)^{k+1}\left( \sum_{1\leq i_1<\cdots<i_k\leq n} P(A_{i_1}\cap \cdots \cap A_{i_k}) \right)$$
{% endhint %}

{% hint style="info" %}
### Theorem 3 (Law of Total Probability) <a href="#theorem-3" id="theorem-3"></a>

If $$A_1, A_2, \cdots$$ partition $$\Omega$$ (i.e $$A_i$$ are disjoint and $$\cup A_i = \Omega$$), then for event $$B$$,

$$P(B) = \sum_iP(B\cap A_i)$$
{% endhint %}

## Conditional Probability

{% hint style="info" %}
### Definition 2

If $$B$$ is an event with $$P(B)>0$$, then the conditional probability of $$A$$ given $$B$$ is

$$P(A|B) = \frac{P(A\cap B)}{P(B)}$$
{% endhint %}

Intuitively, conditional probabilty is the probability of event $$A$$ given that event $$B$$ has occurred. In terms of probability spaces, it is as if we have taken $$(\Omega, \mathcal{F}, P)$$ and now have a probabilty measure $$P(\cdot|C)$$ belonging to the space $$(\Omega, \mathcal{F}, P(\cdot|C))$$.

{% hint style="info" %}
### Theorem 4 (Bayes Theorem) <a href="#theorem-4" id="theorem-4"></a>

$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
{% endhint %}

## Independence

{% hint style="info" %}
### Definition 3

Events $$A$$ and $$B$$ are independent if $$P(A\cap B) = P(A)P(B)$$
{% endhint %}

If $$P(B)>0$$, then $$A, B$$ are independent if and only if $$P(A|B) = P(A)$$. In other words, knowing $$B$$ occurred gave no extra information about $$A$$.

{% hint style="info" %}
### Definition 4

If $$A,B,C$$ with $$P(C)>0$$ satisfy $$P(A\cap B|C) = P(A|C)P(B|C)$$, then $$A$$ and $$B$$ are conditionally independent given $$C$$.
{% endhint %}

Conditional independence is a special case of independence where $$A$$ and $$B$$ are not necessarily independent in the original probability space which has the measure $$P$$, but are independent in the new probability space conditioned on $$C$$ with the measure $$P(\cdot|C)$$.
