Berkeley Notes
  • Introduction
  • EE120
    • Introduction to Signals and Systems
    • The Fourier Series
    • The Fourier Transform
    • Generalized transforms
    • Linear Time-Invariant Systems
    • Feedback Control
    • Sampling
    • Appendix
  • EE123
    • The DFT
    • Spectral Analysis
    • Sampling
    • Filtering
  • EECS126
    • Introduction to Probability
    • Random Variables and their Distributions
    • Concentration
    • Information Theory
    • Random Processes
    • Random Graphs
    • Statistical Inference
    • Estimation
  • EECS127
    • Linear Algebra
    • Fundamentals of Optimization
    • Linear Algebraic Optimization
    • Convex Optimization
    • Duality
  • EE128
    • Introduction to Control
    • Modeling Systems
    • System Performance
    • Design Tools
    • Cascade Compensation
    • State-Space Control
    • Digital Control Systems
    • Cayley-Hamilton
  • EECS225A
    • Hilbert Space Theory
    • Linear Estimation
    • Discrete Time Random Processes
    • Filtering
  • EE222
    • Real Analysis
    • Differential Geometry
    • Nonlinear System Dynamics
    • Stability of Nonlinear Systems
    • Nonlinear Feedback Control
Powered by GitBook
On this page
  • Norms
  • Sets
  • Convergence
  • Contractions
  • Continuity

Was this helpful?

  1. EE222

Real Analysis

Definition 1

The extended real line is the set

{−∞}∪R∪{∞}.\{-\infty\} \cup \mathbb{R} \cup \{\infty\}.{−∞}∪R∪{∞}.

Definition 2

The supremum of a set S⊂RS \subset \mathbb{R}S⊂R is a value a∈Rea \in \mathbb{R}_ea∈Re​ such that ∀s∈S, s≤a\forall s\in S,\ s \leq a∀s∈S, s≤a and if b∈Reb \in \mathbb{R}_eb∈Re​ such that ∀s∈S, s≤b\forall s\in S,\ s \leq b∀s∈S, s≤b, then a≤ba \leq ba≤b.

Supremum is essentially the “least upper bound” in a set. It always exists, and is called sup⁡S\sup SsupS. The opposite of supremum is the infinimum.

Definition 3

The infinimum of a set S⊂RS \subset \mathbb{R}S⊂R is a value a∈Rea \in \mathbb{R}_ea∈Re​ such that ∀s∈S, s≥a\forall s\in S,\ s \geq a∀s∈S, s≥a and if b∈Reb \in \mathbb{R}_eb∈Re​ such that ∀s∈S, s≥b\forall s\in S,\ s \geq b∀s∈S, s≥b, then a≥ba \geq ba≥b.

The infinimum is the “greatest upper bound”. Like the supremum, it always exists, and it is denoted inf⁡S\inf SinfS. Supremum and Infinimum can be applied to scalar function f:S→Rf: S\to \mathbb{R}f:S→R by letting

sup⁡x∈Sf(x)=sup⁡{f(x)∣x∈S}.\sup_{x\in S} f(x) = \sup \{f(x) | x\in S \}.supx∈S​f(x)=sup{f(x)∣x∈S}.

Norms

Definition 4

Let VVV be a vector space of R\mathbb{R}R, then ∥⋅∥:V→R\|\cdot\|: V \to \mathbb{R}∥⋅∥:V→R is a norm if ∀x,y∈V,α∈R\forall \boldsymbol{x},\boldsymbol{y}\in V, \alpha \in \mathbb{R}∀x,y∈V,α∈R,

∥x∥≥0,x=0⇔∥x∥=0,∥αx∥=∣α∣∥x∥,∥x+y∥≤∥x∥+∥y∥.\|\boldsymbol{x}\| \geq 0, \qquad \boldsymbol{x} = 0 \Leftrightarrow \|\boldsymbol{x}\| = 0, \qquad \|\alpha \boldsymbol{x}\| = |\alpha|\|\boldsymbol{x}\|, \qquad \|\boldsymbol{x} + \boldsymbol{y}\| \leq \|\boldsymbol{x}\| + \|\boldsymbol{y}\|.∥x∥≥0,x=0⇔∥x∥=0,∥αx∥=∣α∣∥x∥,∥x+y∥≤∥x∥+∥y∥.

Definition 5

A normed space (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) is a vector space which is equipped with a norm ∥⋅∥:V→R\|\cdot\|: V \to \mathbb{R}∥⋅∥:V→R.

If we have an operator AAA which takes vectors from normed space (X,∥⋅∥X)(X, \|\cdot\|_X)(X,∥⋅∥X​) and outputs vectors in normed space (Y,∥⋅∥Y)(Y, \|\cdot\|_Y)(Y,∥⋅∥Y​), then we can define another norm on the vector space of operators from X→YX\to YX→Y.

Definition 6

Let A:X→YA:X\to YA:X→Y be an operator between normed spaces (X,∥⋅∥X)(X, \|\cdot\|_X)(X,∥⋅∥X​) and (Y,∥⋅∥Y)(Y, \|\cdot\|_Y)(Y,∥⋅∥Y​), then the induced norm of AAA is

∥A∥i=sup⁡∥x∥X≠0∥Ax∥Y∥x∥X\|A\|_i = \sup_{\|\boldsymbol{x}\|_X \neq 0} \frac{\|A\boldsymbol{x}\|_Y}{\|\boldsymbol{x}\|_X}∥A∥i​=sup∥x∥X​=0​∥x∥X​∥Ax∥Y​​

The induced norm can be thought of as the maximum gain of the operator.

Definition 7

Two norms ∥⋅∥\|\cdot\|∥⋅∥ and ∣∣∣⋅∣∣∣|||\cdot|||∣∣∣⋅∣∣∣ on a vector space VVV are said to be equivalent if ∃k1,k2>0\exists k_1, k_2 > 0∃k1​,k2​>0 such that

∀x∈V, k1∥x∥≤∣∣∣x∣∣∣≤k2∥x∥\forall \boldsymbol{x}\in V,\ k_1\|\boldsymbol{x}\| \leq |||\boldsymbol{x}||| \leq k_2\|\boldsymbol{x}\|∀x∈V, k1​∥x∥≤∣∣∣x∣∣∣≤k2​∥x∥

If VVV is a finite dimensional vector space if and only if all norms of VVV are equivalent.

Sets

Definition 8

Let (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) be a normed space, a∈Ra\in \mathbb{R}a∈R, a>0a > 0a>0, x0∈V\boldsymbol{x}_0\in Vx0​∈V, then the open ball of radius aaa centered around x0x_0x0​ is given by

Ba(x0)={x∈V ∣ ∥x−x0∥<a}B_a(\boldsymbol{x}_0) = \{ \boldsymbol{x} \in V \ | \ \|\boldsymbol{x} - \boldsymbol{x}_0\| < a \}Ba​(x0​)={x∈V ∣ ∥x−x0​∥<a}

Definition 9

A set S⊂VS\subset VS⊂V is open if ∀s0∈S, ∃ϵ>0\forall \boldsymbol{s}_0\in S,\ \exists \epsilon > 0∀s0​∈S, ∃ϵ>0 such that Bϵ(s0)⊂SB_\epsilon(\boldsymbol{s}_0) \subset SBϵ​(s0​)⊂S.

Open sets have a boundary which is not included in the set. By convention, we say that the empty set is open.

The opposite of an open set is a closed set.

Definition 10

A set SSS is closed if ∼S\sim S∼Sis open.

Closed sets have a boundary which is included in the set.

Convergence

Definition 11

A sequence of points xk\boldsymbol{x}_kxk​ in normed space (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) converges to a point xˉ\bar{\boldsymbol{x}}xˉ if

∀ϵ>0, ∃N<∞,  such that ∀k≥N,∥xk−xˉ∥<ϵ\forall \epsilon > 0,\ \exists N < \infty,\ \text{ such that } \forall k \geq N, \|\boldsymbol{x}_k - \bar{\boldsymbol{x}}\| < \epsilon∀ϵ>0, ∃N<∞,  such that ∀k≥N,∥xk​−xˉ∥<ϵ

Convergence means that we can always find a finite time such that after that time, all points in the sequence stay within a specified norm ball.

Definition 12

A sequence xk\boldsymbol{x}_kxk​ is cauchy if

∀ϵ>0, ∃N<∞ such that ∀n,m≥N,∥xm−xn∥<ϵ\forall \epsilon > 0,\ \exists N < \infty \text{ such that } \forall n,m \geq N, \|\boldsymbol{x}_m - \boldsymbol{x}_n\| < \epsilon∀ϵ>0, ∃N<∞ such that ∀n,m≥N,∥xm​−xn​∥<ϵ

A Cauchy sequence has a looser type of convergence than a convergent sequence since it only requires all elements to in the sequence to be part of the same norm ball after some time instead of requiring the sequence to get closer and closer to a single point.

Theorem 1

If xn\boldsymbol{x}_nxn​ is a convergent sequence, then xn\boldsymbol{x}_nxn​is a also a Cauchy sequence.

Definition 13

A normed space (V,∥⋅∥)(V, \|\cdot\|)(V,∥⋅∥) is complete if every Cauchy sequence converges to a point in VVV.

Because a complete space requires that Cauchy sequences converge, all cauchy sequences are convergent in a complete space. Two important complete spaces are

  1. Every finite dimensional vector space

  2. (C[a,b],∥⋅∥∞)(C[a,b], \|\cdot\|_\infty)(C[a,b],∥⋅∥∞​), the set of continuously differentiable functions on the closed interval [a,b][a,b][a,b] equipped with the infinity norm.

A complete normed space is also called a Banach Space.

Contractions

Definition 14

A point x∗\boldsymbol{x}^*x∗ is a fixed point of a function P:X→XP:X\to XP:X→X if P(x∗)=x∗P(\boldsymbol{x}^*)=\boldsymbol{x}^*P(x∗)=x∗.

Definition 15

A function P:X→XP:X\to XP:X→X is a contraction if ∃c∈R,0≤c<1\exists c\in\mathbb{R}, 0 \leq c < 1∃c∈R,0≤c<1 such that

∀x,y∈X, ∥P(x)−P(y)∥≤c∥x−y∥\forall \boldsymbol{x},\boldsymbol{y}\in X,\ \|P(\boldsymbol{x}) - P(\boldsymbol{y})\| \leq c \|\boldsymbol{x}-\boldsymbol{y}\|∀x,y∈X, ∥P(x)−P(y)∥≤c∥x−y∥

Informally, a contraction is a function which makes distances smaller. Suppose we look at a sequence defined by iterates of a function

xk+1=P(xk)\boldsymbol{x}_{k+1} = P(\boldsymbol{x}_k)xk+1​=P(xk​)

where PPP is a function P:X→XP:X\to XP:X→X. When does this sequence converge, and to what point will it converge?

Theorem 2 (Contraction Mapping Theorem)

If P:X→XP:X\to XP:X→X is a contraction on the Banach space (X,∥⋅∥)(X, \|\cdot\|)(X,∥⋅∥), then there is a unique x∗∈X\boldsymbol{x}^*\in Xx∗∈X such that P(x∗)=x∗P(\boldsymbol{x}^*) = \boldsymbol{x}^*P(x∗)=x∗ and ∀x0∈X\forall \boldsymbol{x}_0\in X∀x0​∈X, the sequence xn+1=P(xn)\boldsymbol{x}_{n+1} = P(\boldsymbol{x}_n)xn+1​=P(xn​) converges to x∗\boldsymbol{x}^*x∗.

The contraction mapping theorem proves that contractions have a unique fixed points, and that repeatedly applying the contraction will converge to the fixed point.

Continuity

Definition 16

A function h:V→Wh:V\to Wh:V→W on normed spaces (V,∥⋅∥V)(V, \|\cdot\|_V)(V,∥⋅∥V​) and (W,∥⋅∥W)(W, \|\cdot\|_W)(W,∥⋅∥W​) is continuous at a point x0\boldsymbol{x}_0x0​ if ∀ϵ>0,∃δ>0\forall \epsilon > 0, \exists \delta > 0∀ϵ>0,∃δ>0 such that

∥x−x0∥V<δ  ⟹  ∥h(x)−h(x0)∥W<ϵ\|\boldsymbol{x}-\boldsymbol{x}_0\|_V < \delta \implies \|h(\boldsymbol{x}) - h(\boldsymbol{x_0})\|_W < \epsilon∥x−x0​∥V​<δ⟹∥h(x)−h(x0​)∥W​<ϵ

\label{thm:continuity}

Continuity essentially means that given an ϵ−\epsilon-ϵ−ball in WWW, we can find a δ−\delta-δ−ball in VVV which is mapped to the ball in WWW. If a function is continuous at all points x0\boldsymbol{x}_0x0​, then we say the function is continuous.

We can make the definition of continuity more restrictive by restraining the rate of growth of the function.

Definition 17

A function h:V→Wh:V\to Wh:V→W on normed spaces (V,∥⋅∥V)(V, \|\cdot\|_V)(V,∥⋅∥V​) and (W,∥⋅∥W)(W, \|\cdot\|_W)(W,∥⋅∥W​) is Lipschitz continuous at x0∈V\boldsymbol{x}_0\in Vx0​∈V if ∃r>0\exists r > 0∃r>0 and L<∞L < \inftyL<∞ such that

∀x,y∈Br(x0), ∥h(x)−h(y)∥W≤L∥x−y∥V\forall \boldsymbol{x}, \boldsymbol{y}\in B_r(\boldsymbol{x}_0),\ \|h(\boldsymbol{x}) - h(\boldsymbol{y})\|_W \leq L \|\boldsymbol{x} - \boldsymbol{y}\|_V∀x,y∈Br​(x0​), ∥h(x)−h(y)∥W​≤L∥x−y∥V​

A good interpretation of Lipschitz Continuity is that given two points in a ball around x0\boldsymbol{x}_0x0​, the slope of the line connecting those two points is less than LLL. It means that the function is growing slower than linear for some region around x0\boldsymbol{x}_0x0​. Lipschitz continuity implies continuity. If a function is lipschitz continuous with respect to one norm, it is also lipschitz continuous with respect to all equivalent norms.

When the function hhh is a function on Rn\mathbb{R}^nRn and is also differentiable, then Lipschitz continuity is easy to determine.

Theorem 3

For a differentiable function h:Rn→Rnh:\mathbb{R}^n\to\mathbb{R}^nh:Rn→Rn,

∃r>0,L<∞,x0∈Rn, ∀x∈Br(x0),∣∣∂h∂x∣∣2≤L\exists r>0, L < \infty, \boldsymbol{x}_0\in\mathbb{R}^n,\ \forall \boldsymbol{x}\in B_r(\boldsymbol{x}_0), \left\lvert\left\lvert\frac{\partial h}{\partial \boldsymbol{x}}\right\rvert\right\rvert_2 \leq L∃r>0,L<∞,x0​∈Rn, ∀x∈Br​(x0​),​​∂x∂h​​​2​≤L

implies Lipschitz Continuity at x0\boldsymbol{x}_0x0​.

This captures the idea of growing slower than linear in high dimensional space.

Definition 18

A function h:R→Vh:\mathbb{R}\to Vh:R→V is piecewise continuous if ∀k∈Z\forall k\in \mathbb{Z}∀k∈Z, h:[−k,k]→Vh:[-k, k] \to Vh:[−k,k]→V is continuous except at a possibly finite number of points, and at the points of discontinuity tit_iti​, lim⁡s→0+h(ti+s)\lim_{s\to0^+} h(t_i+s)lims→0+​h(ti​+s) and lim⁡s→0−h(ti+s)\lim_{s\to0^-}h(t_i+s)lims→0−​h(ti​+s)exist and are finite.

PreviousEE222NextDifferential Geometry

Last updated 2 years ago

Was this helpful?