About Basics of Linear Algebra for Machine Learning

Linear Algebra Is Important in Machine Learning

 There is no doubt that linear algebra is important in machine learning. Linear algebra is the mathematics of data. It’s all vectors and matrices of numbers. Modern statistics is described using the notation of linear algebra and modern statistical methods harness the tools of linear algebra. Modern machine learning methods are described the same way, using the notations and tools drawn directly from linear algebra. Even some classical methods used in the field, such as linear regression via linear least squares and singular-value decomposition, are linear algebra methods, and other methods, such as principal component analysis, were born from the marriage of linear algebra and statistics. To read and understand machine learning, you must be able to read and understand linear algebra.

Practitioners Study Linear Algebra Too Early

If you ask how to get started in machine learning, you will very likely be told to start with linear algebra. We know that knowledge of linear algebra is critically important, but it does not have to be the place to start. Learning linear algebra first, then calculus, probability, statistics, and eventually machine learning theory is a long and slow bottom-up path. A better fit for developers is to start with systematic procedures that get results, and work back to the deeper understanding of theory, using working results as a context. I call this the top-down or results-first approach to machine learning, and linear algebra is not the first step, but perhaps the second or third.

Practitioners Study Too Much Linear Algebra

When practitioners do circle back to study linear algebra, they learn far more of the field than is required for or relevant to machine learning. Linear algebra is a large field of study that has tendrils into engineering, physics and quantum physics. There are also theorems and derivations for nearly everything, most of which will not help you get better skill from or a deeper understanding of your machine learning model. Only a specific subset of linear algebra is required, though you can always go deeper once you have the basics.

Practitioners Study Linear Algebra Wrong

Linear algebra textbooks will teach you linear algebra in the classical university bottom-up approach. This is too slow (and painful) for your needs as a machine learning practitioner. Like learning machine learning itself, take the top-down approach. Rather than starting with theorems and abstract concepts, you can learn the basics of linear algebra in a concrete way with data structures and worked examples of operations on those data structures. It’s so much faster. Once you know how operations work, you can circle back and learn how they were derived.

A Better Way

This book was born out of my frustrations at seeing practitioner after practitioner diving into linear algebra textbooks and online courses designed for undergraduate students and giving up. The bottom-up approach is hard, especially if you already have a full time job. Linear algebra is not only important to machine learning, but it is also a lot of fun, or can be if it is approached in the right way. I put together this book to help you see the field the way I see it: as just another set of tools we can harness on our journey toward machine learning mastery.