Pytorch——ArgMax and Reduction Tensor Ops

Tensor reduction operations

A reduction operation on a tensor is an operation that reduces the number of elements contained within the tensor.

1. Reshaping operations gave us the ability to position our elements along particular axes.
2. Element-wise operations allow us to perform operations on elements between two tensors.
3. Reduction operations allow us to perform operations on elements within a single tensor.

Reduction operation example

Suppose we the following 3 x 3 rank-2 tensor:

> t = torch.tensor([
    [0,1,0],
    [2,0,2],
    [0,3,0]
], dtype=torch.float32)

# Let’s look at our first reduction operation, a summation:

> t.sum()
tensor(8.)

> t.numel()
9

> t.sum().numel()
1

# We can see that

> t.sum().numel() < t.numel()
True

Since the number of elements have been reduced by the operation, we can conclude that the sum() method is a reduction operation.
As you may expect, here are some other common reduction functions:

> t.sum()
tensor(8.)

> t.prod()
tensor(0.)

> t.mean()
tensor(.8889)

> t.std()
tensor(1.1667)

Reducing tensors by axes

> t = torch.tensor([
    [1,1,1,1],
    [2,2,2,2],
    [3,3,3,3]
], dtype=torch.float32)

This is a 3 x 4 rank-2 tensor. Having different lengths for the two axes will help us understand these reduce operations
.Let’s consider the sum() method again. Only, this time, we will specify a dimension to reduce. We have two axes so we’ll do both. Check it out.

> t.sum(dim=0)  # 在维度0上进行加和，也就是数组相加，故1+2+3=6
tensor([6., 6., 6., 6.])

> t.sum(dim=1)  # 在维度1上进行加和，也就是每个数组内的数相加，1+1+1+1=4，2+2+2+2=8等
tensor([ 4.,  8., 12.])

When we sum across the first axis（第一个轴就是第零个维度）, we are taking the summation of all the elements of the first axis.

​> t[0]
tensor([1., 1., 1., 1.])

> t[1]
tensor([2., 2., 2., 2.])

> t[2]
tensor([3., 3., 3., 3.])

> t[0] + t[1] + t[2]
tensor([6., 6., 6., 6.])

The second axis in this tensor contains numbers that come in groups of four. Since we have three groups of four numbers, we get three sums.

> t[0].sum()
tensor(4.)

> t[1].sum()
tensor(8.)

> t[2].sum()
tensor(12.)

> t.sum(dim=1)
tensor([ 4.,  8., 12.])

Argmax tensor reduction operation

Argmax returns the index location of the maximum value inside a tensor.
When we call the argmax() method on a tensor, the tensor is reduced to a new tensor that contains an index value indicating where the max value is inside the tensor.

t = torch.tensor([
    [1,0,0,2],
    [0,3,3,0],
    [4,0,0,5]
], dtype=torch.float32)

> t.max()  # 最大值为5
tensor(5.)

> t.argmax()
tensor(11)

> t.flatten()
tensor([1., 0., 0., 2., 0., 3., 3., 0., 4., 0., 0., 5.])

We’ll have a look at the flattened output for this tensor. If we don’t specific an axis to the argmax() method, it returns the index location of the max value from the flattened tensor, which in this case is indeed 11.
Let’s see how we can work with specific axes now.

> t.max(dim=0)  # 数组间进行比较
(tensor([4., 3., 3., 5.]), tensor([2, 1, 1, 2]))

> t.argmax(dim=0)
tensor([2, 1, 1, 2])

> t.max(dim=1)  # 数组内进行比较
(tensor([2., 3., 5.]), tensor([3, 1, 3]))

> t.argmax(dim=1)
tensor([3, 1, 3])

Notice how the call to the max() method returns two tensors. The first tensor contains the max values and the second tensor contains the index locations for the max values.

Accessing elements inside tensors

> t = torch.tensor([
    [1,2,3],
    [4,5,6],
    [7,8,9]
], dtype=torch.float32)

> t.mean()
tensor(5.)

> t.mean().item()
5.0

When we call mean on this 3 x 3 tensor, the reduced output is a scalar valued tensor. If we want to actually get the value as a number, we use the item() tensor method. This works for scalar valued tensors.
Have a look at how we do it with multiple values:

> t.mean(dim=0).tolist()
[4.0, 5.0, 6.0]

> t.mean(dim=0).numpy()
array([4., 5., 6.], dtype=float32)

When we compute the mean across the first axis, multiple values are returned, and we can access the numeric values by transforming the output tensor into a Python list or a NumPy array.