Skip to main content

Tensors and Operations

Introduction to Tensors in PyTorch

PyTorch is an open-source library for deep learning and other compute-intensive tasks, and it's built around tensors. Tensors are a type of data structure used in linear algebra, which is essential for machine learning and deep learning. They are, in essence, multi-dimensional arrays. This article aims to provide a comprehensive understanding of PyTorch Tensors and operations that can be performed on them.

What is a Tensor?

A Tensor in PyTorch is very similar to NumPy’s ndarray, with the addition being that Tensors can also be used on a GPU to accelerate computing. It is a multi-dimensional matrix containing elements of a single data type. They can be of various sizes and are used to encode the inputs and outputs of a model, as well as the model’s parameters.

Creating Tensors

Tensors can be created in several ways. Let's see a few examples:

  1. Directly from data:
import torch

data = [[1, 2],[3, 4]]
x_data = torch.tensor(data)
  1. From a NumPy array:
import numpy as np

np_array = np.array(data)
x_np = torch.from_numpy(np_array)
  1. From another tensor:
x_ones = torch.ones_like(x_data) # retains the properties of x_data
print(f"Ones Tensor: \n {x_ones} \n")

x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data
print(f"Random Tensor: \n {x_rand} \n")

Tensor Attributes

Tensors have attributes like shape, datatype, and device they are stored on. Let's print these attributes:

tensor = torch.rand(3,4)

print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device tensor is stored on: {tensor.device}")

Tensor Operations

Over a hundred tensor operations, including arithmetic, linear algebra, matrix manipulation, sampling, etc., are comprehensively described here. Most of these operations have GPU-accelerated versions.

Let's see a few examples:

  1. Standard numpy-like indexing and slicing:
tensor = torch.ones(4, 4)
tensor[:,1] = 0
print(tensor)
  1. Joining tensors. You can use torch.cat to concatenate a sequence of tensors along a given dimension:
t1 = torch.cat([tensor, tensor, tensor], dim=1)
print(t1)
  1. Multiplying tensors. This computes the element-wise product:
print(f"tensor.mul(tensor) \n {tensor.mul(tensor)} \n")
  1. Matrix multiplication:
print(f"tensor.matmul(tensor.T) \n {tensor.matmul(tensor.T)} \n")

Tensors on the GPU

Tensors can be moved onto any device using the .to method.

# We move our tensor to the GPU if available
if torch.cuda.is_available():
  tensor = tensor.to('cuda')

Conclusion

Tensors are the fundamental data structure in PyTorch and understanding them is necessary for implementing any kind of model in PyTorch. This article provided an introduction to PyTorch Tensors, their creation, attributes, operations, and how they can be used on a GPU. Happy learning!