> For the complete documentation index, see [llms.txt](https://aparande.gitbook.io/berkeley-notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://aparande.gitbook.io/berkeley-notes/ee222-0/ee222-1.md).

# Real Analysis

{% hint style="info" %}

#### Definition 1

The extended real line is the set

$${-\infty} \cup \mathbb{R} \cup {\infty}.$$
{% endhint %}

{% hint style="info" %}

#### Definition 2

The supremum of a set $$S \subset \mathbb{R}$$ is a value $$a \in \mathbb{R}\_e$$ such that $$\forall s\in S,\ s \leq a$$ and if $$b \in \mathbb{R}\_e$$ such that $$\forall s\in S,\ s \leq b$$, then $$a \leq b$$.
{% endhint %}

Supremum is essentially the “least upper bound” in a set. It always exists, and is called $$\sup S$$. The opposite of supremum is the infinimum.

{% hint style="info" %}

#### Definition 3

The infinimum of a set $$S \subset \mathbb{R}$$ is a value $$a \in \mathbb{R}\_e$$ such that $$\forall s\in S,\ s \geq a$$ and if $$b \in \mathbb{R}\_e$$ such that $$\forall s\in S,\ s \geq b$$, then $$a \geq b$$.
{% endhint %}

The infinimum is the “greatest upper bound”. Like the supremum, it always exists, and it is denoted $$\inf S$$. Supremum and Infinimum can be applied to scalar function $$f: S\to \mathbb{R}$$ by letting

$$\sup\_{x\in S} f(x) = \sup {f(x) | x\in S }.$$

## Norms

{% hint style="info" %}

#### Definition 4

Let $$V$$ be a vector space of $$\mathbb{R}$$, then $$|\cdot|: V \to \mathbb{R}$$ is a norm if $$\forall \boldsymbol{x},\boldsymbol{y}\in V, \alpha \in \mathbb{R}$$,

$$|\boldsymbol{x}| \geq 0, \qquad \boldsymbol{x} = 0 \Leftrightarrow |\boldsymbol{x}| = 0, \qquad |\alpha \boldsymbol{x}| = |\alpha||\boldsymbol{x}|, \qquad |\boldsymbol{x} + \boldsymbol{y}| \leq |\boldsymbol{x}| + |\boldsymbol{y}|.$$
{% endhint %}

{% hint style="info" %}

#### Definition 5

A normed space $$(V, |\cdot|)$$ is a vector space which is equipped with a norm $$|\cdot|: V \to \mathbb{R}$$.
{% endhint %}

If we have an operator $$A$$ which takes vectors from normed space $$(X, |\cdot|\_X)$$ and outputs vectors in normed space $$(Y, |\cdot|\_Y)$$, then we can define another norm on the vector space of operators from $$X\to Y$$.

{% hint style="info" %}

#### Definition 6

Let $$A:X\to Y$$ be an operator between normed spaces $$(X, |\cdot|\_X)$$ and $$(Y, |\cdot|\_Y)$$, then the induced norm of $$A$$ is

$$|A|*i = \sup*{|\boldsymbol{x}|\_X \neq 0} \frac{|A\boldsymbol{x}|\_Y}{|\boldsymbol{x}|\_X}$$
{% endhint %}

The induced norm can be thought of as the maximum gain of the operator.

{% hint style="info" %}

#### Definition 7

Two norms $$|\cdot|$$ and $$|||\cdot|||$$ on a vector space $$V$$ are said to be equivalent if $$\exists k\_1, k\_2 > 0$$ such that

$$\forall \boldsymbol{x}\in V,\ k\_1|\boldsymbol{x}| \leq |||\boldsymbol{x}||| \leq k\_2|\boldsymbol{x}|$$
{% endhint %}

If $$V$$ is a finite dimensional vector space if and only if all norms of $$V$$ are equivalent.

## Sets

{% hint style="info" %}

#### Definition 8

Let $$(V, |\cdot|)$$ be a normed space, $$a\in \mathbb{R}$$, $$a > 0$$, $$\boldsymbol{x}\_0\in V$$, then the open ball of radius $$a$$ centered around $$x\_0$$ is given by

$$B\_a(\boldsymbol{x}\_0) = { \boldsymbol{x} \in V \ | \ |\boldsymbol{x} - \boldsymbol{x}\_0| < a }$$
{% endhint %}

{% hint style="info" %}

#### Definition 9

A set $$S\subset V$$ is open if $$\forall \boldsymbol{s}*0\in S,\ \exists \epsilon > 0$$ such that $$B*\epsilon(\boldsymbol{s}\_0) \subset S$$.
{% endhint %}

Open sets have a boundary which is not included in the set. By convention, we say that the empty set is open.

The opposite of an open set is a closed set.

{% hint style="info" %}

#### Definition 10

A set $$S$$ is closed if $$\sim S$$is open.
{% endhint %}

Closed sets have a boundary which is included in the set.

## Convergence

{% hint style="info" %}

#### Definition 11

A sequence of points $$\boldsymbol{x}\_k$$ in normed space $$(V, |\cdot|)$$ converges to a point $$\bar{\boldsymbol{x}}$$ if

$$\forall \epsilon > 0,\ \exists N < \infty,\ \text{ such that } \forall k \geq N, |\boldsymbol{x}\_k - \bar{\boldsymbol{x}}| < \epsilon$$
{% endhint %}

Convergence means that we can always find a finite time such that after that time, all points in the sequence stay within a specified norm ball.

{% hint style="info" %}

#### Definition 12

A sequence $$\boldsymbol{x}\_k$$ is cauchy if

$$\forall \epsilon > 0,\ \exists N < \infty \text{ such that } \forall n,m \geq N, |\boldsymbol{x}\_m - \boldsymbol{x}\_n| < \epsilon$$
{% endhint %}

A Cauchy sequence has a looser type of convergence than a convergent sequence since it only requires all elements to in the sequence to be part of the same norm ball after some time instead of requiring the sequence to get closer and closer to a single point.

{% hint style="info" %}

#### Theorem 1

If $$\boldsymbol{x}\_n$$ is a convergent sequence, then $$\boldsymbol{x}\_n$$is a also a Cauchy sequence.
{% endhint %}

{% hint style="info" %}

#### Definition 13

A normed space $$(V, |\cdot|)$$ is complete if every Cauchy sequence converges to a point in $$V$$.
{% endhint %}

Because a complete space requires that Cauchy sequences converge, all cauchy sequences are convergent in a complete space. Two important complete spaces are

1. Every finite dimensional vector space
2. $$(C\[a,b], |\cdot|\_\infty)$$, the set of continuously differentiable functions on the closed interval $$\[a,b]$$ equipped with the infinity norm.

A complete normed space is also called a **Banach Space**.

## Contractions

{% hint style="info" %}

#### Definition 14

A point $$\boldsymbol{x}^*$$ is a fixed point of a function $$P:X\to X$$ if $$P(\boldsymbol{x}^*)=\boldsymbol{x}^\*$$.
{% endhint %}

{% hint style="info" %}

#### Definition 15

A function $$P:X\to X$$ is a contraction if $$\exists c\in\mathbb{R}, 0 \leq c < 1$$ such that

$$\forall \boldsymbol{x},\boldsymbol{y}\in X,\ |P(\boldsymbol{x}) - P(\boldsymbol{y})| \leq c |\boldsymbol{x}-\boldsymbol{y}|$$
{% endhint %}

Informally, a contraction is a function which makes distances smaller. Suppose we look at a sequence defined by iterates of a function

$$\boldsymbol{x}\_{k+1} = P(\boldsymbol{x}\_k)$$

where $$P$$ is a function $$P:X\to X$$. When does this sequence converge, and to what point will it converge?

{% hint style="info" %}

#### Theorem 2 (Contraction Mapping Theorem) <a href="#theorem-2" id="theorem-2"></a>

If $$P:X\to X$$ is a contraction on the Banach space $$(X, |\cdot|)$$, then there is a unique $$\boldsymbol{x}^*\in X$$ such that $$P(\boldsymbol{x}^*) = \boldsymbol{x}^*$$ and $$\forall \boldsymbol{x}*0\in X$$, the sequence $$\boldsymbol{x}*{n+1} = P(\boldsymbol{x}\_n)$$ converges to $$\boldsymbol{x}^*$$.
{% endhint %}

The contraction mapping theorem proves that contractions have a unique fixed points, and that repeatedly applying the contraction will converge to the fixed point.

## Continuity

{% hint style="info" %}

#### Definition 16

A function $$h:V\to W$$ on normed spaces $$(V, |\cdot|\_V)$$ and $$(W, |\cdot|\_W)$$ is continuous at a point $$\boldsymbol{x}\_0$$ if $$\forall \epsilon > 0, \exists \delta > 0$$ such that

$$|\boldsymbol{x}-\boldsymbol{x}\_0|\_V < \delta \implies |h(\boldsymbol{x}) - h(\boldsymbol{x\_0})|\_W < \epsilon$$

\label{thm:continuity}
{% endhint %}

Continuity essentially means that given an $$\epsilon-$$ball in $$W$$, we can find a $$\delta-$$ball in $$V$$ which is mapped to the ball in $$W$$. If a function is continuous at all points $$\boldsymbol{x}\_0$$, then we say the function is continuous.

We can make the definition of continuity more restrictive by restraining the rate of growth of the function.

{% hint style="info" %}

#### Definition 17

A function $$h:V\to W$$ on normed spaces $$(V, |\cdot|\_V)$$ and $$(W, |\cdot|\_W)$$ is Lipschitz continuous at $$\boldsymbol{x}\_0\in V$$ if $$\exists r > 0$$ and $$L < \infty$$ such that

$$\forall \boldsymbol{x}, \boldsymbol{y}\in B\_r(\boldsymbol{x}\_0),\ |h(\boldsymbol{x}) - h(\boldsymbol{y})|\_W \leq L |\boldsymbol{x} - \boldsymbol{y}|\_V$$
{% endhint %}

A good interpretation of Lipschitz Continuity is that given two points in a ball around $$\boldsymbol{x}\_0$$, the slope of the line connecting those two points is less than $$L$$. It means that the function is growing slower than linear for some region around $$\boldsymbol{x}\_0$$. Lipschitz continuity implies continuity. If a function is lipschitz continuous with respect to one norm, it is also lipschitz continuous with respect to all equivalent norms.

When the function $$h$$ is a function on $$\mathbb{R}^n$$ and is also differentiable, then Lipschitz continuity is easy to determine.

{% hint style="info" %}

#### Theorem 3

For a differentiable function $$h:\mathbb{R}^n\to\mathbb{R}^n$$,

$$\exists r>0, L < \infty, \boldsymbol{x}\_0\in\mathbb{R}^n,\ \forall \boldsymbol{x}\in B\_r(\boldsymbol{x}\_0), \left\lvert\left\lvert\frac{\partial h}{\partial \boldsymbol{x}}\right\rvert\right\rvert\_2 \leq L$$

implies Lipschitz Continuity at $$\boldsymbol{x}\_0$$.
{% endhint %}

This captures the idea of growing slower than linear in high dimensional space.

{% hint style="info" %}

#### Definition 18

A function $$h:\mathbb{R}\to V$$ is piecewise continuous if $$\forall k\in \mathbb{Z}$$, $$h:\[-k, k] \to V$$ is continuous except at a possibly finite number of points, and at the points of discontinuity $$t\_i$$, $$\lim\_{s\to0^+} h(t\_i+s)$$ and $$\lim\_{s\to0^-}h(t\_i+s)$$exist and are finite.
{% endhint %}