Orthogonal Matrices#
Orthogonal matrices describe linear transformations of real Euclidean space that preserve lengths and angles. Rotations and reflections are examples. These transformations fix the origin; translations are not represented by orthogonal matrices.
Definition and Core Properties#
A real square matrix \(Q\in\mathbb{R}^{n\times n}\) is orthogonal if
Writing its columns as \(q_1,\ldots,q_n\), we have \((Q^TQ)_{ij}=q_i^Tq_j\). Thus the definition says exactly that the columns are orthonormal: each column has length one, and distinct columns are perpendicular.
The columns are therefore linearly independent, so \(Q\) is invertible. Multiplying \(Q^TQ=I_n\) on the right by \(Q^{-1}\) gives
The second identity says that the rows are orthonormal as well. Conversely, \(QQ^T=I_n\) implies the same properties, so for square matrices either identity is sufficient.
Rectangular Matrices with Orthonormal Columns#
A matrix \(Q\in\mathbb{R}^{m\times n}\) with \(m>n\) can also satisfy \(Q^TQ=I_n\). We will call this a matrix with orthonormal columns, reserving orthogonal matrix for the square case. Such a matrix preserves lengths when mapping \(\mathbb{R}^n\) into \(\mathbb{R}^m\), but it has no two-sided inverse and \(QQ^T\neq I_m\).
For example,
In general, \(QQ^T\) is the orthogonal projection onto the column space of \(Q\). It leaves vectors in that space unchanged and sends vectors perpendicular to it to zero. We will discuss this in the next section on projections.
Unitary Matrices#
The complex counterpart is a unitary matrix: a square matrix \(Q\in\mathbb{C}^{n\times n}\) satisfying
where \(Q^H\) denotes the conjugate transpose. The conjugation is essential for preserving complex inner products and lengths.
Key Results and Mathematical Formulas#
Length, Distance, and Angle Preservation#
For real vectors \(x,y\) and an orthogonal matrix \(Q\),
Thus inner products are preserved. Taking \(y=x\) gives
so \(\|Qx\|_2=\|x\|_2\). Applying this to \(x-y\) also gives
Because both inner products and lengths are unchanged, the angle between any two nonzero vectors is unchanged. A distance-preserving map is called an isometry. These arguments also apply to rectangular matrices with orthonormal columns. For unitary matrices, replace each transpose by a conjugate transpose.
In particular, the operator 2-norm of an orthogonal or unitary matrix is \(1\):
Products and Inverses#
The product of two orthogonal matrices of the same size is orthogonal:
The inverse \(Q^{-1}=Q^T\) is also orthogonal, since \(QQ^T=I_n\). Together with the identity matrix, these properties make the orthogonal matrices a group, denoted \(O(n)\). The corresponding statements hold for unitary matrices using conjugate transposes.
Determinant and Orientation#
For a real orthogonal matrix,
Hence \(\det(Q)=\pm1\). Both cases preserve volume, because \(|\det(Q)|=1\).
If \(\det(Q)=1\), the transformation preserves orientation. These matrices form the special orthogonal group, denoted \(SO(n)\). In two dimensions they are rotations about the origin; in three dimensions they are rotations about an axis through the origin.
If \(\det(Q)=-1\), the transformation reverses orientation. It can be a reflection, but need not be a single reflection across a hyperplane. For example, \(-I_3\) reverses all three coordinate directions and has determinant \(-1\); a reflection across a plane leaves that plane fixed.
For a complex unitary matrix, the corresponding conclusion is
Thus \(|\det(Q)|=1\), although the determinant need not be real.
Why Orthogonal Matrices Are Useful in Numerical Linear Algebra#
Orthogonal transformations preserve the size of an error already present in a vector. If \(x\) is replaced by \(x+e\), then, in exact arithmetic,
Computing the transformation can introduce additional rounding errors, so this identity alone does not guarantee the accuracy of an algorithm. We will study those questions in the next chapter, Solving Linear Systems.
Orthogonal matrices also make changes of coordinates convenient. Applying the inverse means applying \(Q^T\), without computing a general matrix inverse. For a square matrix \(A\), changing to an orthonormal basis gives \(Q^TAQ\), which is similar to \(A\). Later chapters use orthogonal transformations in QR factorization, the singular value decomposition, and eigenvalue algorithms.
Reflections Across Hyperplanes#
Let \(w\in\mathbb{R}^n\) be nonzero. The reflection across the hyperplane perpendicular to \(w\) is
To see its action, write \(x=x_{\perp}+\alpha w\), where \(w^Tx_{\perp}=0\). Then
Thus \(H\) leaves the hyperplane fixed and reverses the perpendicular component. Applying it twice returns the original vector, so \(H^2=I_n\). The formula also gives \(H^T=H\), and therefore
Hence \(H\) is orthogonal and \(H^{-1}=H\). In an orthonormal basis consisting of \(w/\|w\|_2\) and vectors perpendicular to \(w\), its matrix is \(\operatorname{diag}(-1,1,\ldots,1)\), so \(\det(H)=-1\).
These are Householder reflections. Their computational use is developed later in the section on Householder transformations.
The Cartan–Dieudonné Theorem#
The Cartan–Dieudonné theorem shows that hyperplane reflections suffice to construct every real orthogonal transformation.
Theorem 2 (Cartan–Dieudonné theorem)
Every orthogonal transformation of \(\mathbb{R}^n\) is a composition of at most \(n\) reflections across hyperplanes through the origin. Equivalently,
where each \(H_i\) is a hyperplane reflection. For \(k=0\), the product is the identity matrix.
Proof by Induction#
Proof. For \(n=1\), an orthogonal matrix is either \([1]\) or \([-1]\). These require zero or one reflection, respectively.
Assume the theorem holds in dimension \(n-1\), and let \(Q\) be orthogonal on \(\mathbb{R}^n\). Write \(e_1=(1,0,\ldots,0)^T\) and \(v=Qe_1\). Length preservation gives \(\|v\|_2=1\).
Case 1: \(v=e_1\). Then \(Q\) fixes \(e_1\). The subspace
is preserved by \(Q\), because for \(x\in S\),
The restriction of \(Q\) to \(S\) is therefore an orthogonal transformation of an \((n-1)\)-dimensional Euclidean space. By induction, it is a product of at most \(n-1\) reflections on \(S\).
Each such reflection has a normal vector \(w\in S\). Using the same formula \(I_n-2ww^T/(w^Tw)\) extends it to a reflection on \(\mathbb{R}^n\) that fixes \(e_1\). The product of these extended reflections agrees with \(Q\) on both \(S\) and \(e_1\), so it equals \(Q\) on all of \(\mathbb{R}^n\).
Case 2: \(v\neq e_1\). Set \(w=v-e_1\) and define
We verify that \(Hv=e_1\). Since \(v^Tv=1\), writing \(v_1=e_1^Tv\) gives
Consequently,
The matrix \(HQ\) is orthogonal and fixes \(e_1\). By Case 1, it is a product of at most \(n-1\) reflections. Since \(H^{-1}=H\), multiplying by \(H\) expresses \(Q\) as a product of at most \(n\) reflections.
Because each reflection has determinant \(-1\), a product of \(k\) reflections has determinant \((-1)^k\). Thus orientation-preserving orthogonal transformations require an even number of reflections, and orientation-reversing ones require an odd number.