Matrices and Matrix Arithmetic

Matrices are a foundational element of linear algebra. Matrices are used throughout the field of machine learning in the description of algorithms and processes such as the input data variable (X) when training an algorithm. In this tutorial, you will discover matrices in linear algebra and how to manipulate them in Python. After completing this tutorial, you will know:

• What a matrix is and how to define one in Python with NumPy
• How to perform element-wise operations such as addition, subtraction, and the Hadamard product.
• How to multiply matrices together and the intuition behind the operation.

1.1 Tutorial Overview

This tutorial is divided into 6 parts ;they are:

1. What is a Matrix
2. Defining a Matrix
3. Matrix Arithmetic
4. Matrix-Matrix Multiplication
5. Matrix-Vector Multiplication
6. Matrix-Scalar Multiplication

1.2 What is Matrix

A  matrix is a two-dimensional array of scalars with one or more columns and one or more rows.

A matrix is a two-dimensional array (a table) of numbers.

The notation for a matrix is often an uppercase letter, such as A, and entries are referred to by their two-dimensional subscript of row (i) and column (j), such as ai,j . For example, we can define a 3-row, 2-column matrix:

                                        A = ((a1,1, a1,2),(a2,1, a2,2),(a3,1, a3,2))

It is more common to see matrices defined using a horizontal notation.

 

        A likely first place you may encounter a matrix in machine learning is in model training data comprised of many rows and columns and often represented using the capital letter X. The geometric analogy used to help understand vectors and some of their operations does not hold with matrices. Further, a vector itself may be considered a matrix with one column and multiple rows. Often the dimensions of the matrix are denoted as m and n or m × n for the number of rows and the number of columns respectively. Now that we know what a matrix is, let’s look at defining one in Python.

1.3 Defining  a Matrix

We can represent a matrix in Python using a two-dimensional NumPy array. A NumPy array

can be constructed given a list of lists. For example ,below is a 2 row, 3 column matrix

# create matrix
from numpy import array
A = array([[1, 2, 3],[4, 5, 6]])
print(A)

Running the example prints the created matrix showing the expected structure.

 1.4 Matrix Arithmetic

In this section will demonstrate simple matrix-matrix arithmetic, where all operations are performed element-wise between two matrices of equal size to result in a new matrix with the same size.

1.4.1 Matrix Addition

# Example of matrix addition
# matrix addition
from numpy import array
# define first matrix
A = array([
    [1, 2, 3],
    [4, 5, 6]
])
print(A)
# define second matrix
B = array([[1, 2, 3],
          [4, 5, 6]])
print(B)
# add matrices
C = A + B
print(C)

The example first defines two 2 × 3 matrices and then adds them together. Running the example first prints the two parent matrices and then the result of adding them together.

 

1.4.2 Matrix Subtraction

Similarly, one matrix can be subtracted from another matrix with the same dimensions.

                                                C = A - B

The scalar elements in the resulting matrix are calculated as the subtraction of the elements in each of the matrices.

We can implement this in python using the minus operator directly on the Two NumPy  arrays.

# Example of matrix substraction
# matrix substraction
from numpy import array
# define first matrix
A = array([
    [1, 2, 3],
    [4, 5, 6]])
print(A)
# define second matrix 
B = array([
    [0.5, 0.5, 0.5],
    [0.5, 0.5, 0.5]
])
print(B)

# substract matrices
C = A - B
print(C)

The example first defines two 2 ×3 matrices and then subtracts one from the other. Running the example first prints the two parent matrices and then subtracts the first matrix from the second.

1.4.3  Matrix Multiplication(Hadamard Product)

We can implement this in Python using the star operator directly on the two NumPy arrays.

# matrix Hadamard product
from numpy import array
# define first matrix
A = array([
    [1, 2, 3],
    [4, 5, 6]
])
print(A)
# define second matrix
B = array([
    [1, 2, 3],
    [4, 5, 6]
])
print(B)

# multiply matrices
C = A * B
print(C)

The example first defines two 2 ×3 matrices and then multiplies them together. Running the example first prints the two parent matrices and then the result of multiplying them together with a Hadamard Product.

 

1.4.4 Matrix Division

We can implement this in Python using the division operator directly on the two NumPy arrays.

# Example of matrix division
# matrix division
from numpy import array
# define first matrix
A = array([
    [1, 2, 3],
    [4, 5, 6]
])
print(A)

# define second matrix
B = array([
    [1, 2, 3],
    [4, 5, 6]
])
print(B)

# divide matrix
C = A / B

print(C)

1.5 Matrix-Matrix Multiplication

 

 

# Example of matrix-matrix dot product
# matrix dot product
from numpy import array
# define first matrix
A = array([
    [1,2],
    [3,4],
    [5,6]
])
print(A)
# define second matrix
B = array([
    [1, 2],
    [3, 4]
])
print(B)

# multiply matrices
C = A.dot(B)
print(C)

# multiply matrices with @ operator
D = A @ B
print(D)

 

1.6 Matrix-Vector Multiplication

 The matrix-vector multiplication can be implemented in NumPy using the dot() function

 

# Exmaple of matrix-vector dot product
# matrix-vector multiplication
from numpy import array
# define matrix 
A = array([
    [1, 2],
    [3, 4],
    [5, 6]
])
print(A)
# define vector
B = array([0.5, 0.6])
print(B)

# multiply 
C = A.dot(B)
print(C)

The example first defines a 3 × 2 matrix and a 2 element vector and then multiplies them together. Running the example first prints the parent matrix and vector and then the result of multiplying them together.

 1.7 Matrix-Scalar Multiplication

A matrix can be multiplied by a scalar. This can be represented using the dot notation between the matrix and the scalar. C = A · b

 

# Example of matrix-scalar dot product
# matrix-scalar multiplication
from numpy import array
# define matrix
A = array([
    [1, 2],
    [3, 4],
    [5, 6]
])
print(A)
# define scalar
b = 0.5
print(b)
# multiply
C = A * b
print(C)

1.8 Extensions

This section lists some ideas for extending the tutorial that you may wish to explore.

• Create one example using each operation using your own small array data.
• Implement each matrix arithmetic operation manually for matrices defined as lists of lists.
• Search machine learning papers and find 1 example of each operation being used.

1.9 Summary

In this tutorial, you discovered matrices in linear algebra and how to manipulate them in Python. Specifically, you learned:

• What a matrix is and how to define one in Python with NumPy.
• How to perform element-wise operations such as addition, subtraction, and the Hadamard product
• How to multiply matrices together and the intuition behind the operation.