Vector norms

• Key tool for proofs and formal derivations
• Essential to describe and understand the properties of objects in LA
• Used to convert a vector to a single scalar number, its “size.”
• The most common norm is the 2-norm:

$$\Vert x{\Vert }_2=(\sum _{i=1}^n(x_i{)}^2{)}^{1/2}=\sqrt{x^{T}x}$$

Dot product and the 2-norm are connected through:

$$x^{T}y=\Vert x{\Vert }_2\Vert y{\Vert }_2\cos \theta$$

So the dot product can be used to measure the angle between two vectors.

If two vectors are orthogonal, their dot product is equal to 0.

A unit vector is a vector with norm 1: $\Vert x{\Vert }_2=1$.

In data science and linear algebra, it’s common to use different norms. They differ by the weight they assign to the components of a vector.

• 1-norm: all scales contribute equally

$$\Vert x{\Vert }_1=\sum _{i=1}^n|x_i|$$

• 2-norm: emphasizes the larger entries
• Largest entry:

$$\Vert x{\Vert }_{\infty }=\underset{1\le i\le n}{\max }|x_i|$$

• Intermediate; as $p\to \infty$, the norm becomes closer to the max-norm

$$\Vert x{\Vert }_p=(\sum _{i=1}^n|x_i{|}^p{)}^{1/p}$$

Computational cost: $O(n)$.

Balls in different norms Vectors and matrices