L 1 L_1 L1 Norm (Taxicab Norm or Manhattan norm) | L 2 L_2 L2 Norm (Euclidean norm) | Max Norm |
---|---|---|
The L 1 L_1 L1 norm is calculated as the sum of the absolute vector values. | The L 2 L_2 L2 norm is calculated as the square root of the sum of the squared vector values. | The max norm is calculated as returning the maximum value of the vector, hence the name. |
∥ v ∥ 1 = ∣ a 1 ∣ + ∣ a 2 ∣ + ∣ a 3 ∣ \begin{Vmatrix} \boldsymbol{v} \end{Vmatrix}_1 = \begin{vmatrix} a_1 \end{vmatrix} + \begin{vmatrix} a_2 \end{vmatrix} + \begin{vmatrix} a_3 \end{vmatrix} ∥∥v∥∥1=∣∣a1∣∣+∣∣a2∣∣+∣∣a3∣∣ | ∥ v ∥ 2 = ∑ i = 1 3 a i 2 = a 1 2 + a 2 2 + a 3 2 \begin{Vmatrix} \boldsymbol{v} \end{Vmatrix}_2 = \sqrt{\sum\limits_{i=1}^3{a_i^2}} = \sqrt{a_1^2 + a_2^2 + a_3^2} ∥∥v∥∥2=i=1∑3ai2=a12+a22+a32 | ∥ v ∥ ∞ = max ( a 1 , a 2 , a 3 ) \begin{Vmatrix} \boldsymbol{v} \end{Vmatrix}_{\infty} = \max(a_1, a_2, a_3) ∥∥v∥∥∞=max(a1,a2,a3) |
In effect, the L 1 L_1 L1 norm is a calculation of the Manhattan distance from the origin of the vector space. | The L2 norm is also known as the Euclidean norm as it is calculated as the Euclidean distance from the origin. | |
The L 1 L_1 L1 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small, and in turn, the model less complex. | Like the L 1 L_1 L1 norm, the L 2 L_2 L2 norm is often used when fitting machine learning algorithms as a regularization method, e.g. a method to keep the coefficients of the model small and, in turn, the model less complex. | Max norm is also used as a regularization in machine learning, such as on neural network weights, called max norm regularization. |
By far, the L 2 L_2 L2 norm is more commonly used than other vector norms in machine learning. |
L 1 L_1 L1 norm
- The
L
1
L_1
L1 norm of a vector can be calculated in
NumPy
using thenorm()
function with a parameter to specify the norm order, in this case1
.
# l1 norm of a vector
from numpy import array
from numpy.linalg import norm
a = array([1, 2, 3])
print(a) # [1 2 3]
l1 = norm(a, 1)
print(l1) # 6
L 2 L_2 L2 Norm
- The
L
2
L_2
L2 norm of a vector can be calculated in
NumPy
using thenorm()
function with default parameters.
# l2 norm of a vector
from numpy import array
from numpy.linalg import norm
a = array([1, 2, 3])
print(a) # [1 2 3]
l2 = norm(a)
print(l2) # 3.7416573867739413
Max Norm
The max norm of a vector can be calculated in NumPy
using the norm()
function with the order parameter set to inf
.
# max norm of a vector
from numpy import inf
from numpy import array
from numpy.linalg import norm
a = array([1, 2, 3])
print(a) # [1 2 3]
maxnorm = norm(a, inf)
print(maxnorm) # 3.0