Deep Learning PyTorch Course, Architecture of PyTorch

Hello, today we will take a deep dive into the architecture of PyTorch, a deep learning framework.

1. What is PyTorch?

PyTorch is an open-source machine learning framework developed by Facebook AI Research. It is designed for gradient computation, automatic differentiation, and tensor operations. PyTorch is highly useful for research-oriented tasks and has a more intuitive and Pythonic syntax compared to other frameworks like TensorFlow.

2. Basic Concepts of PyTorch

2.1 Tensor

The fundamental data structure in PyTorch is a tensor. A tensor is a multi-dimensional array similar to a numpy array but can perform computations faster on a GPU. Tensors can be created as follows:

import torch

# Create a 1D tensor
tensor_1d = torch.tensor([1, 2, 3, 4, 5])
print(tensor_1d)

# Create a 2D tensor
tensor_2d = torch.tensor([[1, 2], [3, 4]])
print(tensor_2d)

2.2 Autograd

Autograd is the automatic differentiation feature of PyTorch. PyTorch can compute gradients by setting the requires_grad attribute for any tensor:

x = torch.ones(2, 2, requires_grad=True)
y = x + 2
z = y * y * 3
out = z.mean()

print(out)

# Backpropagation
out.backward()
print(x.grad)

3. Exploring PyTorch Architecture

PyTorch is composed of several components for creating deep neural networks. This allows users to design and train new models efficiently.

3.1 Module

A module is the basic building block of PyTorch, implemented by inheriting the nn.Module class. Each neural network layer is implemented as a module:

import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc1 = nn.Linear(in_features=2, out_features=2)

    def forward(self, x):
        return self.fc1(x)

3.2 Loss Function

A loss function is a metric to evaluate the performance of the model by calculating the difference between predicted and actual values. There are various loss functions available in PyTorch:

loss_fn = nn.MSELoss()

# Predicted and actual values
y_pred = torch.tensor([0.0, 1.0])
y_true = torch.tensor([0.5, 0.5])
loss = loss_fn(y_pred, y_true)
print(loss)

3.3 Optimization

This is the process of updating the model’s parameters. PyTorch allows efficient learning using various optimization techniques:

import torch.optim as optim

model = MyModel()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Training process
for epoch in range(100):
    optimizer.zero_grad()  # Gradient reset
    y_pred = model(torch.tensor([[1.0, 2.0]]))  # Model prediction
    loss = loss_fn(y_pred, torch.tensor([[0.0, 1.0]]))  # Loss calculation
    loss.backward()  # Backpropagation
    optimizer.step()  # Parameter update

4. Practice: Creating a Simple Neural Network

Now, based on the content described above, let’s create a simple neural network. This neural network will perform digit image classification using the MNIST dataset.

4.1 Preparing the Dataset

from torchvision import datasets, transforms

transform = transforms.Compose([transforms.ToTensor()])
train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=32, shuffle=True)

4.2 Defining the Neural Network Architecture

class SimpleNN(nn.Module):
        def __init__(self):
            super(SimpleNN, self).__init__()
            self.fc1 = nn.Linear(784, 128)  # 28*28=784
            self.fc2 = nn.Linear(128, 10)

        def forward(self, x):
            x = x.view(-1, 784)  # Flatten
            x = torch.relu(self.fc1(x))
            x = self.fc2(x)
            return x

4.3 Training the Model

model = SimpleNN()
optimizer = optim.Adam(model.parameters(), lr=0.001)
loss_fn = nn.CrossEntropyLoss()

for epoch in range(5):
    for images, labels in train_loader:
        optimizer.zero_grad()
        outputs = model(images)
        loss = loss_fn(outputs, labels)
        loss.backward()
        optimizer.step()

    print(f'Epoch [{epoch + 1}/5], Loss: {loss.item():.4f}')

5. Conclusion

In this lecture, we explored the architecture of PyTorch and various components that constitute a neural network model. Due to its intuitive ideas and flexibility, PyTorch is very effective for deep learning research and development. We hope you will challenge yourself with various projects utilizing PyTorch in the future.