Vectors and Matrices | AI Pathshala

Definition

How can a photo or a sentence go into a formula? It has to become numbers first.

A scalar is one number. A vector is an ordered list of numbers. A matrix is a grid of numbers with rows and columns. A tensor is the general idea: an array of numbers with any number of dimensions, so a scalar is a 0D tensor, a vector 1D and a matrix 2D.

In simple words

One student's marks in one subject is a scalar: 85. That student's marks in three subjects is a vector: [85, 70, 92]. The whole class's marks form a matrix, with one row per student and one column per subject. Marks for every class in a school, in every exam, make a tensor.

Why it matters

Every model works only on numbers. A customer becomes a vector, a table of customers becomes a matrix, and a colour image becomes a 3D tensor. All the training code you'll ever write comes down to operations on these objects, and most errors in real projects are shape errors.

How it works

Object	Dimensions	Shape example	In AI
Scalar	0	()	A loss value, a learning rate
Vector	1	(3,)	One customer's features
Matrix	2	(100, 3)	100 customers, 3 features each
3D tensor	3	(224, 224, 3)	A colour image: height, width, RGB
4D tensor	4	(32, 224, 224, 3)	A batch of 32 colour images

Notation you'll see in papers

Lowercase italic x is a scalar. Lowercase bold x is a vector. Uppercase A is a matrix.
x_i is element i of a vector. A_ij is the element in row i and column j of a matrix.
The symbol ℝ means the real numbers, so ℝⁿ is the set of all vectors with n numbers.

The maths

A vector and a matrix

x = x_{1} x_{2} ⋮ x_{n} \in R^{n}, A = a_{11} ⋮ a_{m 1} \dots ⋱ \dots a_{1 n} ⋮ a_{mn} \in R^{m \times n}

$n$: How many numbers the vector holds, or how many columns the matrix has
$m$: How many rows the matrix has
$a_{ij}$: The number in row i, column j

In code

shapes.py

import numpy as np

scalar = np.array(85)
vector = np.array([85, 70, 92])
matrix = np.array([[85, 70, 92], [60, 75, 88]])
tensor = np.zeros((32, 224, 224, 3))

for name, value in [("scalar", scalar), ("vector", vector), ("matrix", matrix), ("tensor", tensor)]:
    print(name, "dimensions:", value.ndim, "shape:", value.shape)

print("Row 1, column 2 of the matrix:", matrix[0, 1])

Output

scalar dimensions: 0 shape: ()
vector dimensions: 1 shape: (3,)
matrix dimensions: 2 shape: (2, 3)
tensor dimensions: 4 shape: (32, 224, 224, 3)
Row 1, column 2 of the matrix: 70

Common mistakes

Mixing up rows and columns. In ML data, rows are almost always examples and columns are features.
Confusing a vector of shape (3,) with a matrix of shape (3, 1). They look alike but behave differently in code.
Counting from 1 in code. Maths usually starts at 1, while Python starts at 0.

Check yourself

1. A dataset has 500 rows and 8 columns. What is its shape?

2. How many dimensions does a batch of colour images have?

3. Which is a tensor?

Summary

Scalar: 0D. Vector: 1D. Matrix: 2D. Tensor: any number of dimensions.
All data must become these before a model can use it.
Shape tells you what the data is. Rows are examples, columns are features.

Finished reading?

Sources: Deisenroth, Faisal & Ong, Mathematics for Machine Learning