Berkeley Notes
  • Introduction
  • EE120
    • Introduction to Signals and Systems
    • The Fourier Series
    • The Fourier Transform
    • Generalized transforms
    • Linear Time-Invariant Systems
    • Feedback Control
    • Sampling
    • Appendix
  • EE123
    • The DFT
    • Spectral Analysis
    • Sampling
    • Filtering
  • EECS126
    • Introduction to Probability
    • Random Variables and their Distributions
    • Concentration
    • Information Theory
    • Random Processes
    • Random Graphs
    • Statistical Inference
    • Estimation
  • EECS127
    • Linear Algebra
    • Fundamentals of Optimization
    • Linear Algebraic Optimization
    • Convex Optimization
    • Duality
  • EE128
    • Introduction to Control
    • Modeling Systems
    • System Performance
    • Design Tools
    • Cascade Compensation
    • State-Space Control
    • Digital Control Systems
    • Cayley-Hamilton
  • EECS225A
    • Hilbert Space Theory
    • Linear Estimation
    • Discrete Time Random Processes
    • Filtering
  • EE222
    • Real Analysis
    • Differential Geometry
    • Nonlinear System Dynamics
    • Stability of Nonlinear Systems
    • Nonlinear Feedback Control
Powered by GitBook
On this page
  • Norms
  • Sets
  • Convergence
  • Contractions
  • Continuity

Was this helpful?

  1. EE222

Real Analysis

PreviousEE222NextDifferential Geometry

Last updated 2 years ago

Was this helpful?

Definition 1

The extended real line is the set

{−∞}∪R∪{∞}.\{-\infty\} \cup \mathbb{R} \cup \{\infty\}.{−∞}∪R∪{∞}.

Definition 2

The supremum of a set S⊂RS \subset \mathbb{R}S⊂R is a value a∈Rea \in \mathbb{R}_ea∈Re​ such that ∀s∈S, s≤a\forall s\in S,\ s \leq a∀s∈S, s≤a and if b∈Reb \in \mathbb{R}_eb∈Re​ such that ∀s∈S, s≤b\forall s\in S,\ s \leq b∀s∈S, s≤b, then a≤ba \leq ba≤b.

Supremum is essentially the “least upper bound” in a set. It always exists, and is called sup⁡S\sup SsupS. The opposite of supremum is the infinimum.

Definition 3

The infinimum of a set S⊂RS \subset \mathbb{R}S⊂R is a value a∈Rea \in \mathbb{R}_ea∈Re​ such that ∀s∈S, s≥a\forall s\in S,\ s \geq a∀s∈S, s≥a and if b∈Reb \in \mathbb{R}_eb∈Re​ such that ∀s∈S, s≥b\forall s\in S,\ s \geq b∀s∈S, s≥b, then a≥ba \geq ba≥b.

The infinimum is the “greatest upper bound”. Like the supremum, it always exists, and it is denoted inf⁡S\inf SinfS. Supremum and Infinimum can be applied to scalar function f:S→Rf: S\to \mathbb{R}f:S→R by letting

sup⁡x∈Sf(x)=sup⁡{f(x)∣x∈S}.\sup_{x\in S} f(x) = \sup \{f(x) | x\in S \}.supx∈S​f(x)=sup{f(x)∣x∈S}.

Norms

Definition 4

Let VVV be a vector space of R\mathbb{R}R, then ∥⋅∥:V→R\|\cdot\|: V \to \mathbb{R}∥⋅∥:V→R is a norm if ∀x,y∈V,α∈R\forall \boldsymbol{x},\boldsymbol{y}\in V, \alpha \in \mathbb{R}∀x,y∈V,α∈R,

∥x∥≥0,x=0⇔∥x∥=0,∥αx∥=∣α∣∥x∥,∥x+y∥≤∥x∥+∥y∥.\|\boldsymbol{x}\| \geq 0, \qquad \boldsymbol{x} = 0 \Leftrightarrow \|\boldsymbol{x}\| = 0, \qquad \|\alpha \boldsymbol{x}\| = |\alpha|\|\boldsymbol{x}\|, \qquad \|\boldsymbol{x} + \boldsymbol{y}\| \leq \|\boldsymbol{x}\| + \|\boldsymbol{y}\|.∥x∥≥0,x=0⇔∥x∥=0,∥αx∥=∣α∣∥x∥,∥x+y∥≤∥x∥+∥y∥.

Definition 5

A normed space (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) is a vector space which is equipped with a norm ∥⋅∥:V→R\|\cdot\|: V \to \mathbb{R}∥⋅∥:V→R.

Definition 6

The induced norm can be thought of as the maximum gain of the operator.

Definition 7

Sets

Definition 8

Definition 9

Open sets have a boundary which is not included in the set. By convention, we say that the empty set is open.

The opposite of an open set is a closed set.

Definition 10

Closed sets have a boundary which is included in the set.

Convergence

Definition 11

Convergence means that we can always find a finite time such that after that time, all points in the sequence stay within a specified norm ball.

Definition 12

A Cauchy sequence has a looser type of convergence than a convergent sequence since it only requires all elements to in the sequence to be part of the same norm ball after some time instead of requiring the sequence to get closer and closer to a single point.

Theorem 1

Definition 13

Because a complete space requires that Cauchy sequences converge, all cauchy sequences are convergent in a complete space. Two important complete spaces are

  1. Every finite dimensional vector space

A complete normed space is also called a Banach Space.

Contractions

Definition 14

Definition 15

Informally, a contraction is a function which makes distances smaller. Suppose we look at a sequence defined by iterates of a function

Theorem 2 (Contraction Mapping Theorem)

The contraction mapping theorem proves that contractions have a unique fixed points, and that repeatedly applying the contraction will converge to the fixed point.

Continuity

Definition 16

\label{thm:continuity}

We can make the definition of continuity more restrictive by restraining the rate of growth of the function.

Definition 17

Theorem 3

This captures the idea of growing slower than linear in high dimensional space.

Definition 18

If we have an operator AAA which takes vectors from normed space (X,∥⋅∥X)(X, \|\cdot\|_X)(X,∥⋅∥X​) and outputs vectors in normed space (Y,∥⋅∥Y)(Y, \|\cdot\|_Y)(Y,∥⋅∥Y​), then we can define another norm on the vector space of operators from X→YX\to YX→Y.

Let A:X→YA:X\to YA:X→Y be an operator between normed spaces (X,∥⋅∥X)(X, \|\cdot\|_X)(X,∥⋅∥X​) and (Y,∥⋅∥Y)(Y, \|\cdot\|_Y)(Y,∥⋅∥Y​), then the induced norm of AAA is

∥A∥i=sup⁡∥x∥X≠0∥Ax∥Y∥x∥X\|A\|_i = \sup_{\|\boldsymbol{x}\|_X \neq 0} \frac{\|A\boldsymbol{x}\|_Y}{\|\boldsymbol{x}\|_X}∥A∥i​=sup∥x∥X​=0​∥x∥X​∥Ax∥Y​​

Two norms ∥⋅∥\|\cdot\|∥⋅∥ and ∣∣∣⋅∣∣∣|||\cdot|||∣∣∣⋅∣∣∣ on a vector space VVV are said to be equivalent if ∃k1,k2>0\exists k_1, k_2 > 0∃k1​,k2​>0 such that

∀x∈V, k1∥x∥≤∣∣∣x∣∣∣≤k2∥x∥\forall \boldsymbol{x}\in V,\ k_1\|\boldsymbol{x}\| \leq |||\boldsymbol{x}||| \leq k_2\|\boldsymbol{x}\|∀x∈V, k1​∥x∥≤∣∣∣x∣∣∣≤k2​∥x∥

If VVV is a finite dimensional vector space if and only if all norms of VVV are equivalent.

Let (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) be a normed space, a∈Ra\in \mathbb{R}a∈R, a>0a > 0a>0, x0∈V\boldsymbol{x}_0\in Vx0​∈V, then the open ball of radius aaa centered around x0x_0x0​ is given by

Ba(x0)={x∈V ∣ ∥x−x0∥<a}B_a(\boldsymbol{x}_0) = \{ \boldsymbol{x} \in V \ | \ \|\boldsymbol{x} - \boldsymbol{x}_0\| < a \}Ba​(x0​)={x∈V ∣ ∥x−x0​∥<a}

A set S⊂VS\subset VS⊂V is open if ∀s0∈S, ∃ϵ>0\forall \boldsymbol{s}_0\in S,\ \exists \epsilon > 0∀s0​∈S, ∃ϵ>0 such that Bϵ(s0)⊂SB_\epsilon(\boldsymbol{s}_0) \subset SBϵ​(s0​)⊂S.

A set SSS is closed if ∼S\sim S∼Sis open.

A sequence of points xk\boldsymbol{x}_kxk​ in normed space (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) converges to a point xˉ\bar{\boldsymbol{x}}xˉ if

∀ϵ>0, ∃N<∞,  such that ∀k≥N,∥xk−xˉ∥<ϵ\forall \epsilon > 0,\ \exists N < \infty,\ \text{ such that } \forall k \geq N, \|\boldsymbol{x}_k - \bar{\boldsymbol{x}}\| < \epsilon∀ϵ>0, ∃N<∞,  such that ∀k≥N,∥xk​−xˉ∥<ϵ

A sequence xk\boldsymbol{x}_kxk​ is cauchy if

∀ϵ>0, ∃N<∞ such that ∀n,m≥N,∥xm−xn∥<ϵ\forall \epsilon > 0,\ \exists N < \infty \text{ such that } \forall n,m \geq N, \|\boldsymbol{x}_m - \boldsymbol{x}_n\| < \epsilon∀ϵ>0, ∃N<∞ such that ∀n,m≥N,∥xm​−xn​∥<ϵ

If xn\boldsymbol{x}_nxn​ is a convergent sequence, then xn\boldsymbol{x}_nxn​is a also a Cauchy sequence.

A normed space (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) is complete if every Cauchy sequence converges to a point in VVV.

(C[a,b],∥⋅∥∞)(C[a,b], \|\cdot\|_\infty)(C[a,b],∥⋅∥∞​), the set of continuously differentiable functions on the closed interval [a,b][a,b][a,b] equipped with the infinity norm.

A point x∗\boldsymbol{x}^*x∗ is a fixed point of a function P:X→XP:X\to XP:X→X if P(x∗)=x∗P(\boldsymbol{x}^*)=\boldsymbol{x}^*P(x∗)=x∗.

A function P:X→XP:X\to XP:X→X is a contraction if ∃c∈R,0≤c<1\exists c\in\mathbb{R}, 0 \leq c < 1∃c∈R,0≤c<1 such that

∀x,y∈X, ∥P(x)−P(y)∥≤c∥x−y∥\forall \boldsymbol{x},\boldsymbol{y}\in X,\ \|P(\boldsymbol{x}) - P(\boldsymbol{y})\| \leq c \|\boldsymbol{x}-\boldsymbol{y}\|∀x,y∈X, ∥P(x)−P(y)∥≤c∥x−y∥

xk+1=P(xk)\boldsymbol{x}_{k+1} = P(\boldsymbol{x}_k)xk+1​=P(xk​)

where PPP is a function P:X→XP:X\to XP:X→X. When does this sequence converge, and to what point will it converge?

If P:X→XP:X\to XP:X→X is a contraction on the Banach space (X,∥⋅∥)(X, \|\cdot\|)(X,∥⋅∥), then there is a unique x∗∈X\boldsymbol{x}^*\in Xx∗∈X such that P(x∗)=x∗P(\boldsymbol{x}^*) = \boldsymbol{x}^*P(x∗)=x∗ and ∀x0∈X\forall \boldsymbol{x}_0\in X∀x0​∈X, the sequence xn+1=P(xn)\boldsymbol{x}_{n+1} = P(\boldsymbol{x}_n)xn+1​=P(xn​) converges to x∗\boldsymbol{x}^*x∗.

A function h:V→Wh:V\to Wh:V→W on normed spaces (V,∥⋅∥V)(V, \|\cdot\|_V)(V,∥⋅∥V​) and (W,∥⋅∥W)(W, \|\cdot\|_W)(W,∥⋅∥W​) is continuous at a point x0\boldsymbol{x}_0x0​ if ∀ϵ>0,∃δ>0\forall \epsilon > 0, \exists \delta > 0∀ϵ>0,∃δ>0 such that

∥x−x0∥V<δ  ⟹  ∥h(x)−h(x0)∥W<ϵ\|\boldsymbol{x}-\boldsymbol{x}_0\|_V < \delta \implies \|h(\boldsymbol{x}) - h(\boldsymbol{x_0})\|_W < \epsilon∥x−x0​∥V​<δ⟹∥h(x)−h(x0​)∥W​<ϵ

Continuity essentially means that given an ϵ−\epsilon-ϵ−ball in WWW, we can find a δ−\delta-δ−ball in VVV which is mapped to the ball in WWW. If a function is continuous at all points x0\boldsymbol{x}_0x0​, then we say the function is continuous.

A function h:V→Wh:V\to Wh:V→W on normed spaces (V,∥⋅∥V)(V, \|\cdot\|_V)(V,∥⋅∥V​) and (W,∥⋅∥W)(W, \|\cdot\|_W)(W,∥⋅∥W​) is Lipschitz continuous at x0∈V\boldsymbol{x}_0\in Vx0​∈V if ∃r>0\exists r > 0∃r>0 and L<∞L < \inftyL<∞ such that

∀x,y∈Br(x0), ∥h(x)−h(y)∥W≤L∥x−y∥V\forall \boldsymbol{x}, \boldsymbol{y}\in B_r(\boldsymbol{x}_0),\ \|h(\boldsymbol{x}) - h(\boldsymbol{y})\|_W \leq L \|\boldsymbol{x} - \boldsymbol{y}\|_V∀x,y∈Br​(x0​), ∥h(x)−h(y)∥W​≤L∥x−y∥V​

A good interpretation of Lipschitz Continuity is that given two points in a ball around x0\boldsymbol{x}_0x0​, the slope of the line connecting those two points is less than LLL. It means that the function is growing slower than linear for some region around x0\boldsymbol{x}_0x0​. Lipschitz continuity implies continuity. If a function is lipschitz continuous with respect to one norm, it is also lipschitz continuous with respect to all equivalent norms.

When the function hhh is a function on Rn\mathbb{R}^nRn and is also differentiable, then Lipschitz continuity is easy to determine.

For a differentiable function h:Rn→Rnh:\mathbb{R}^n\to\mathbb{R}^nh:Rn→Rn,

∃r>0,L<∞,x0∈Rn, ∀x∈Br(x0),∣∣∂h∂x∣∣2≤L\exists r>0, L < \infty, \boldsymbol{x}_0\in\mathbb{R}^n,\ \forall \boldsymbol{x}\in B_r(\boldsymbol{x}_0), \left\lvert\left\lvert\frac{\partial h}{\partial \boldsymbol{x}}\right\rvert\right\rvert_2 \leq L∃r>0,L<∞,x0​∈Rn, ∀x∈Br​(x0​),​​∂x∂h​​​2​≤L

implies Lipschitz Continuity at x0\boldsymbol{x}_0x0​.

A function h:R→Vh:\mathbb{R}\to Vh:R→V is piecewise continuous if ∀k∈Z\forall k\in \mathbb{Z}∀k∈Z, h:[−k,k]→Vh:[-k, k] \to Vh:[−k,k]→V is continuous except at a possibly finite number of points, and at the points of discontinuity tit_iti​, lim⁡s→0+h(ti+s)\lim_{s\to0^+} h(t_i+s)lims→0+​h(ti​+s) and lim⁡s→0−h(ti+s)\lim_{s\to0^-}h(t_i+s)lims→0−​h(ti​+s)exist and are finite.