What is Tensor?
In layman’s terms, a tensor is a way of representing the data in deep learning. A tensor can be a 1-dimensional, a 2-dimensional, a 3-dimensional array, etc. You can think of a tensor as a multidimensional array. In machine learning and deep learning, you have datasets that are high dimensional, in which each dimension represents a different feature of that dataset.
A tensor, is the mathematical representation of a physical entity that may be characterized by magnitude and multiple directions.
And, just like you represent a scalar with a single number and a vector with a sequence of three numbers in a 3-dimensional space, for example, a tensor can be represented by an array of 3R numbers in a 3-dimensional space.
The “R” in this notation represents the rank of the tensor: this means that in a 3-dimensional space, a second-rank tensor can be represented by 3 to the power of 2 or 9 numbers. In an N-dimensional space, scalars will still require only one number, while vectors will require N numbers, and tensors will require N^R numbers. This explains why you often hear that scalars are tensors of rank 0: since they have no direction, you can represent them with one number.
With this in mind, it’s relatively easy to recognize scalars, vectors, and tensors and to set them apart: scalars can be represented by a single number, vectors by an ordered set of numbers, and tensors by an array of numbers with rank.
Consider the following example of a dog versus cat classification problem, where the dataset you’re working with has multiple varieties of both cats and dogs images. Now, in order to correctly classify a dog or a cat when given an image, the network has to learn discriminative features like color, face structure, ears, eyes, the shape of the tail, etc. These features are incorporated by the tensors.
How are tensors different from matrices?
A matrix is a two-dimensional grid of size n×m that contains numbers: you can add and subtract matrices of the same size, multiply one matrix with another as long as the sizes are compatible ((n×m)×(m×p)=n×p), and multiply an entire matrix by a constant.
However, a vector is a matrix with just one row or column. A tensor is often thought of as a generalized matrix. The dimension of the tensor is called its rank. Any rank-2 tensor can be represented as a matrix, but not every matrix is a rank-2 tensor.
Example of Tensors
a) Scalar – A scalar or “rank-0” tensor contains a single value, and no “axes”
rank_0_tensor = tf.constant(4)
print(rank_0_tensor)
Output:
tf.Tensor(4, shape=(), dtype=int32)
b) Vector – A “vector” or “rank-1” tensor is like a list of values. A vector has one axis
rank_1_tensor = tf.constant([2.0, 3.0, 4.0])
print(rank_1_tensor)
Output:
tf.Tensor([2. 3. 4.], shape=(3,), dtype=float32)
c) Matrix – A “rank-2” tensor has two axes
rank_2_tensor = tf.constant([[1, 2],
[3, 4],
[5, 6]], dtype=tf.float16)
print(rank_2_tensor)
Output –
tf.Tensor(
[[1. 2.]
[3. 4.]
[5. 6.]], shape=(3, 2), dtype=float16)
d) Higher dimensional structure – Tensors may have more axes; here is a tensor with three axes
rank_3_tensor = tf.constant([
[[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]],
[[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]],
[[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]],])
print(rank_3_tensor)
Output –
tf.Tensor(
[[[ 0 1 2 3 4]
[ 5 6 7 8 9]]
[[10 11 12 13 14]
[15 16 17 18 19]]
[[20 21 22 23 24]
[25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)
