A Quick Introduction to PyTorch

PyTorch is an open source machine learning library for Python. In this tutorial, we will provide an introduction to the main PyTorch features, tensor library, and autograd –  automatic differentiation package.

Tensor Library

The core data structure in PyTorch is a tensor, which is a multi-dimensional array like NumPy’s  nd-arrays but it offers GPU support.

First, we need to import torch package.

Tensor operations

Creating tensor with all zeros.

Creating tensor with random numbers from a uniform distribution on the interval [0,1).

Reshaping tensor.

The matrix dot product.

Element-wise matrix multiplication.

The transpose of a matrix.


GPU support

PyTorch uses separate backends for CPU and GPU.

Using a GPU in PyTorch is incredibly easy, we just need to call cuda method.

NumPy bridge

Import NumPy

Creating ndarray from tensor.

Creating tensor from NumPy ndarray.

Going back and forth between PyTorch tensor and NumPy ndarray is very efficient because they share same data.

Autograd Library

We need to import Variable from autograd package.

Autograd is an automatic differentiation library which lets you compute derivatives for all operations on tensors. To support automatic differentation, Pytorch introduces Variables, which are wrappers around tensors that provide the ability to define a computational graph, so that we can automatically compute gradients. You can retrieve a tensor held by the Variable, using the data attribute.

Variable represents a node in a computational graph and it’s able to accumulate gradient, which will be stored in grad attribute. In order to create Variable that supports gradient accumulation, we need to specify requires_grad=True.

Computation Graph

PyTorch uses a define-by-run paradigm, you run computation on actual tensors with graphs generated on-the-fly. Let’s define and run simple computation graph.

We can now compute gradients by running backward method, grad attribute contains computed gradients.

Dynamic computation graph gives us a lot of flexibility, we can use control-flow primitives, like for and if to define the graph in runtime, no special DSL is needed, and debugging PyTorch code is as easy as debugging python code.



Leave a Reply

Your email address will not be published. Required fields are marked *