Linear Algebra for Machine Learning with Scala

In this tutorial, we will cover some basics of linear algebra. We will use ND4S – Scala bindings for ND4J, scientific computing library for the JVM.

Linear Algebra building blocks

We begin by discussing the building blocks of linear algebra: scalars, vectors, matrices and tensors:

• Scalar – is a single number
• Vector – is an ordered array of single numbers
• Matrix –  is a two dimensional array of numbers, arranged in rows and columns
• Tensor – is a multidimensional array

Why do we need Linear Algebra for Machine Learning?

Machine learning is all about data and data can be represented as a vector, matrix or tensor. To be effective machine learning often requires large amounts of data,  computations on large matrices can be performed very efficiently using highly optimized libraries for matrix operations like ND4j.

ND4J

The core data structure in Nd4j is the NDArray, which is a multi-dimensional array of numbers: a vector, matrix or tensor.
Internally, it may store single precision or double precision floating point values for each entry.

Let’s add dependencies.

Add the following import statements.

NDArrays operations

We will use Nd4j class, which exposes many static methods to help us with the creation and manipulation of NDArrays.

Creating Vectors

We can create NDArray which represents a vector from an Array.

Now, let’s draw our newly created 2-dimensional vector.

To create a vector of all zeros we can use the zeros function.

Adding two vectors

Two vectors of the same size can be added by adding the corresponding elements.

Scalar-vector multiplication

Scalar-vector multiplication is an operation in which every element of the vector is multiplied by a scalar.

Vector transpose

The transpose of a vector changes a column vector to a row vector or vice versa.

We used shape method to check the size of each dimension, the shape has changed from 1×3 (1 row 3 columns) to 3X1 (3 rows 1 column).

Vector dot product

Vector dot product is one of the most important operations in the whole machine learning.
It’s defined as a sum of corresponding elements of two vectors of the same size. We can think of a dot product as a measure of similarity between two vectors.

Creating matrices

Matrix is a two-dimensional array.  We can create a matrix from a two dimensional  Array.

We can now check the shape of our newly created matrix.

Accessing and setting matrix elements

NDArray supports multidimensional indexing for multidimensional arrays, to access or set a particular element we need to specify its row and column number.

Matrix addition and subtraction

Two matrices can be added or subtracted if, and only if, they have the same dimensions. To add (or subtract) two matrices of the same dimensions, just add (or subtract) the corresponding entries.

Matrix product

The matrix dot product is an operation that produces a matrix from two matrices. The number of columns of the 1st matrix must equal the number of rows of the 2nd. This is how we define the dot product of two matrices, A \((2 \times 3)\) and B \((3 \times 2)\).

$$
\begin{pmatrix}
a_{11} & a_{12} & a_{13} \\
a_{21}& a_{22} & a_{23} \\
\end{pmatrix} \circ \begin{pmatrix}
b_{11} & b_{12} \\
b_{21}& b_{22} \\
b_{31}& b_{32} \\
\end{pmatrix} =
$$
$$
\begin{pmatrix}
a_{11} b_{11} + a_{12} b_{21} + a_{13} b_{31} & a_{11} b_{12} + a_{12} b_{22} + a_{13} b_{32} \\
a_{21} b_{11} + a_{22} b_{21} + a_{23} b_{31} & a_{21} b_{12} + a_{22} b_{22} + a_{23} b_{32} \\
\end{pmatrix}
$$

Matrix element-wise multiplication

Element-wise matrix multiplication takes two matrices of the same dimensions and produces another matrix with elements that are a product of corresponding elements.