Linear Algebraic Optimization

Many optimization problems can be solved using the machinery of Linear Algebra. These problems do not have inequality constraints or non-euclidean norms in the objective function.

Projection

The idea behind projection is to find the closest point in a set closest (with respect to particular norm) to a given point.

Definition 28

where the norm is the one induced by the inner product.

Theorem 9

Matrix Pseudo-inverses

Definition 29

There are several special cases of pseudoinverses.

Explained Variance

This is because the singular values give us a notion of how much of the Frobenius Norm (Total Variance) each dyad explains.

PCA

Removing Constraints

we can write an equivalent unconstrained problem

Last updated