> For the complete documentation index, see [llms.txt](https://aparande.gitbook.io/berkeley-notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://aparande.gitbook.io/berkeley-notes/eecs126-0/eecs126-1.md).

# Introduction to Probability

{% hint style="info" %}

### Definition 1

A probability space is a triple $$(\Omega, \mathcal{F}, P)$$ where $$\Omega$$ is a set of objects called the sample space, $$\mathcal{F}$$ is a family of subsets of $$\Omega$$ called events, and the probability measure $$P:\mathcal{F}\rightarrow \[0,1]$$.
{% endhint %}

One key assumption we make is that $$\mathcal{F}$$ is a $$\sigma$$-algebra containing $$\Omega$$, meaning that countably many complements, unions, and intersections of events in $$\mathcal{F}$$ are also events in $$\mathcal{F}$$. The probability measure $$P$$ must obey **Kolmogorov’s Axioms**.

1. $$\forall A \in \mathcal{F},\ P(A) \geq 0$$
2. $$P(\Omega) = 1$$
3. If $$A\_1, A\_2, \cdots\in \mathcal{F}$$ and $$\forall i\ne j,\ A\_i\bigcap A\_j=\emptyset$$, then $$P\left(\bigcup\_{i\geq 1}A\_i\right) = \sum\_{i\geq1}P(A\_i)$$

We choose $$\Omega$$ and $$\mathcal{F}$$ to model problems in a way that makes our calculations easy.

{% hint style="info" %}

### Theorem 1

$$P(A^c) = 1 - P(A)$$
{% endhint %}

{% hint style="info" %}

### Theorem 2 (Inclusion-Exclusion Principle) <a href="#theorem-2" id="theorem-2"></a>

$$P\left( \bigcup\_{i=1}^{n}A\_i \right) = \sum\_{k=1}^{n}(-1)^{k+1}\left( \sum\_{1\leq i\_1<\cdots\<i\_k\leq n} P(A\_{i\_1}\cap \cdots \cap A\_{i\_k}) \right)$$
{% endhint %}

{% hint style="info" %}

### Theorem 3 (Law of Total Probability) <a href="#theorem-3" id="theorem-3"></a>

If $$A\_1, A\_2, \cdots$$ partition $$\Omega$$ (i.e $$A\_i$$ are disjoint and $$\cup A\_i = \Omega$$), then for event $$B$$,

$$P(B) = \sum\_iP(B\cap A\_i)$$
{% endhint %}

## Conditional Probability

{% hint style="info" %}

### Definition 2

If $$B$$ is an event with $$P(B)>0$$, then the conditional probability of $$A$$ given $$B$$ is

$$P(A|B) = \frac{P(A\cap B)}{P(B)}$$
{% endhint %}

Intuitively, conditional probabilty is the probability of event $$A$$ given that event $$B$$ has occurred. In terms of probability spaces, it is as if we have taken $$(\Omega, \mathcal{F}, P)$$ and now have a probabilty measure $$P(\cdot|C)$$ belonging to the space $$(\Omega, \mathcal{F}, P(\cdot|C))$$.

{% hint style="info" %}

### Theorem 4 (Bayes Theorem) <a href="#theorem-4" id="theorem-4"></a>

$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
{% endhint %}

## Independence

{% hint style="info" %}

### Definition 3

Events $$A$$ and $$B$$ are independent if $$P(A\cap B) = P(A)P(B)$$
{% endhint %}

If $$P(B)>0$$, then $$A, B$$ are independent if and only if $$P(A|B) = P(A)$$. In other words, knowing $$B$$ occurred gave no extra information about $$A$$.

{% hint style="info" %}

### Definition 4

If $$A,B,C$$ with $$P(C)>0$$ satisfy $$P(A\cap B|C) = P(A|C)P(B|C)$$, then $$A$$ and $$B$$ are conditionally independent given $$C$$.
{% endhint %}

Conditional independence is a special case of independence where $$A$$ and $$B$$ are not necessarily independent in the original probability space which has the measure $$P$$, but are independent in the new probability space conditioned on $$C$$ with the measure $$P(\cdot|C)$$.