Summary of Matrix Decompositions

Summary of Matrix Decompositions#

Here is a comparison of the key matrix decompositions related to eigenvalues and singular values.

Decomposition

Requirements for Existence

Components & Properties

Interpretation & Applications

Relation to Others

Schur Decomposition
\(A = Q T Q^H\)

Exists for any square complex matrix \(A\).

Q: Unitary (\(Q^H Q = I\)).
T: Upper triangular.
Diagonal of T contains the eigenvalues of A.

Interpretation: Any linear transformation is unitarily similar to a triangular one.
Applications: Numerically stable computation of all eigenvalues.

This is the general form from which eigendecomposition is a special case.

Real Schur Decomposition
\(A = Q S Q^T\)

Exists for any square real matrix \(A\).

Q: Real orthogonal (\(Q^T Q = I\)).
S: Real quasi-triangular (block upper triangular).
1x1 diagonal blocks are real eigenvalues; 2x2 blocks correspond to complex conjugate eigenvalues.

Interpretation: Represents any real transformation in a basis that makes it “as triangular as possible” using only real numbers.
Applications: Stable eigenvalue computation in real arithmetic.

The real-valued equivalent of the complex Schur decomposition.

Eigendecomposition
\(A = X \Lambda X^{-1}\)

Exists only for diagonalizable matrices (those with \(n\) linearly independent eigenvectors).

X: Invertible matrix whose columns are the eigenvectors.
\(\Lambda\): Diagonal matrix of eigenvalues.
\(X\) is not guaranteed to be unitary.

Interpretation: The transformation is a pure scaling along the eigenvector axes. Associated with “Time” and the evolution of dynamical systems (\(A^k\)).
Applications: Solving linear differential equations, analyzing stability, computing matrix powers.

A special case of the Schur decomposition where the triangular matrix \(T\) is diagonal.

Unitary Diagonalization
\(A = Q \Lambda Q^H\)

Exists only for normal matrices (\(A A^H = A^H A\)).

Q: Unitary matrix whose columns are an orthonormal basis of eigenvectors.
\(\Lambda\): Diagonal matrix of eigenvalues.

Interpretation: The transformation is a pure scaling along a set of orthogonal axes.
Applications: Quantum mechanics, signal processing.

The ideal form of both the Schur and eigendecomposition. Occurs when the matrix is normal.

Singular Value Decomposition (SVD)
\(A = U \Sigma V^T\)

Exists for any \(m \times n\) matrix \(A\), real or complex.

U: Unitary/Orthogonal matrix (\(m \times m\)) of left singular vectors.
V: Unitary/Orthogonal matrix (\(n \times n\)) of right singular vectors.
\(\Sigma\): Rectangular diagonal matrix of non-negative singular values.

Interpretation: Decomposes any transformation into rotation, scaling, and another rotation. Associated with “Space” and the geometry of the transformation.
Applications: PCA, low-rank approximation, computing pseudoinverses, determining rank and condition number.

Generalizes eigendecomposition to any matrix. The singular values of A are the square roots of the eigenvalues of \(A^T A\) and \(A A^T\).