Deep Learning is a field of machine learning based on artificial neural networks, focused on automatically learning useful patterns from various data. In this course, we will explain in detail the process of building and training deep learning models using PyTorch. Depending on the data that needs to be learned and the business requirements, various network architectures can be designed, and PyTorch is a very useful tool for this.
1. What is PyTorch?
PyTorch is an open-source machine learning library developed by the Facebook AI Research group, primarily used for deep learning research and production.
It provides tensor calculations and automatic differentiation features that facilitate model training and gradient-based optimization.
Additionally, it integrates well with Python to support intuitive code writing.
2. Installing PyTorch
There are several methods to install PyTorch, and you can install it using Conda or pip through the commands below.
# If using Anaconda
conda install pytorch torchvision torchaudio cpuonly -c pytorch
# If using pip
pip install torch torchvision torchaudio
After installation, run the following code to check if the installation has been completed successfully.
import torch
print(torch.__version__)
3. Basic Concepts of Deep Learning
The main concepts of deep learning are as follows:
- Neural Network: A data processing structure composed of input layers, hidden layers, and output layers.
- Tensor: The basic data structure in PyTorch, referring to multi-dimensional arrays.
- Activation Function: Determines how each node in the neural network is activated through activation.
- Loss Function: A function that measures the error between the model’s predictions and the actual values.
- Optimizer: An algorithm that updates the weights of the network based on the loss function.
4. Building Deep Learning Models with PyTorch
Let’s build a simple neural network using PyTorch. The dataset we will use is the famous MNIST handwritten digit dataset. This dataset consists of black and white images containing digits from 0 to 9.
4.1 Downloading the Dataset
PyTorch makes it easy to download and preprocess various image datasets through the torchvision library.
The code for downloading the MNIST dataset and setting up the DataLoader is as follows.
import torch
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
# Data preprocessing: convert images to tensors and normalize
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.5,), (0.5,))
])
# Download dataset
train_dataset = datasets.MNIST(root='data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST(root='data', train=False, download=True, transform=transform)
# Set up DataLoader
train_loader = DataLoader(dataset=train_dataset, batch_size=64, shuffle=True)
test_loader = DataLoader(dataset=test_dataset, batch_size=64, shuffle=False)
4.2 Defining the Neural Network Model
Now let’s define a simple neural network model. The code below represents a neural network with two hidden layers.
import torch.nn as nn
import torch.nn.functional as F
class SimpleNet(nn.Module):
def __init__(self):
super(SimpleNet, self).__init__()
self.fc1 = nn.Linear(28 * 28, 128) # Input layer
self.fc2 = nn.Linear(128, 64) # Hidden layer 1
self.fc3 = nn.Linear(64, 10) # Output layer
def forward(self, x):
x = x.view(-1, 28 * 28) # Flatten
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
4.3 Setting the Loss Function and Optimizer
To train the model, we need to define the loss function and optimizer. In this case, we will use cross-entropy loss as the loss function and the Adam optimizer as the optimizer.
model = SimpleNet()
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
4.4 Training the Model
The code below shows the process of training the model. Data is fetched in mini-batches using the data loader, and for each batch, the model’s output is calculated, followed by loss calculation and weight updates.
num_epochs = 5
for epoch in range(num_epochs):
for images, labels in train_loader:
# Zero the gradients
optimizer.zero_grad()
# Forward pass
outputs = model(images)
loss = criterion(outputs, labels)
# Backward pass
loss.backward()
optimizer.step()
print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
4.5 Evaluating the Model
After the model is trained, we evaluate its performance using the test dataset. The code below shows how to measure accuracy on the test dataset.
model.eval() # Switch to evaluation mode
correct = 0
total = 0
with torch.no_grad(): # Disable gradient calculation
for images, labels in test_loader:
outputs = model(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
accuracy = 100 * correct / total
print(f'Accuracy on the test set: {accuracy:.2f}%')
5. Hyperparameter Tuning in Deep Learning
Hyperparameter tuning is an important step in improving the performance of deep learning models. Hyperparameters include learning rate, batch size, size and number of hidden layers, type of activation function, dropout rate, etc.
Generally, techniques such as Grid Search, Random Search, and Bayesian Optimization are used for hyperparameter tuning, each method evaluates various combinations to explore the optimal settings.
6. Conclusion
In this course, we introduced the process of building and training basic deep learning models using PyTorch. We covered key steps in deep learning such as dataset preparation, model definition, training, and evaluation.
Various theories and techniques were also explained to help understand deep learning, so we encourage you to take on more complex models and diverse applications based on this foundation.