Deep learning is a field of machine learning based on artificial neural networks, which is used to learn patterns from data and perform tasks such as prediction or classification. In this course, we will explain the basic concepts of deep learning along with the learning algorithms using a deep learning framework called PyTorch.
Basic Concepts of Deep Learning
The core of deep learning is neural networks. A neural network is a structure composed of units called nodes that are connected in layers, receiving input data and applying weights and biases to generate output data.
Each node performs a nonlinear transformation, which is accomplished through an activation function.
Structure of Neural Networks
Generally, neural networks consist of an input layer, hidden layers, and an output layer.
- Input Layer: The place where the model receives data
- Hidden Layers: Internal layers that process the input data, which can have multiple layers
- Output Layer: The layer that outputs the final prediction value or class
Activation Functions
Activation functions play the role of introducing non-linearity in nodes. Here are the activation functions commonly used.
- ReLU (Rectified Linear Unit): $f(x) = max(0, x)$
- Sigmoid: $f(x) = \frac{1}{1 + e^{-x}}$
- Tanh: $f(x) = \tanh(x) = \frac{e^{x} – e^{-x}}{e^{x} + e^{-x}}$
Deep Learning Learning Algorithms
To train a deep learning model, a dataset is required. The data consists of inputs and targets (outputs).
The learning process of the model proceeds through the following steps.
1. Forward Pass
The input data is passed through the model to compute the predicted values. At this time, the weights and biases of the neural network are used to generate the output.
2. Loss Calculation
The loss is calculated as the difference between the model’s predictions and the actual target values. Common loss functions include Mean Squared Error (MSE) and Cross-Entropy.
3. Backpropagation
This process adjusts weights and biases based on the loss, using Gradient Descent to update the model’s parameters. The backpropagation algorithm calculates the gradient of the loss for each weight using the chain rule.
4. Weight Update
The calculated gradients are used to update the weights and biases. The update formula is as follows.
w = w - learning_rate * gradient b = b - learning_rate * gradient
Implementation in PyTorch
Now, based on the explanations above, let’s implement a simple deep learning model in PyTorch. This example uses the MNIST handwritten digit recognition dataset to classify handwritten digits.
Install and Import Required Libraries
pip install torch torchvision
import torch import torch.nn as nn import torch.optim as optim from torchvision import datasets, transforms from torch.utils.data import DataLoader
Load and Preprocess Dataset
Load the MNIST dataset and perform normalization on the image data.
# Data Preprocessing
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.5,), (0.5,))
])
# Load Dataset
train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
test_dataset = datasets.MNIST(root='./data', train=False, download=True, transform=transform)
# Create Data Loaders
train_loader = DataLoader(dataset=train_dataset, batch_size=64, shuffle=True)
test_loader = DataLoader(dataset=test_dataset, batch_size=64, shuffle=False)
Define the Model
Define a simple neural network model. The input size is 28×28 (MNIST image size), and it has two hidden layers. The output layer is set to 10 (digits 0 to 9).
class SimpleNN(nn.Module):
def __init__(self):
super(SimpleNN, self).__init__()
self.fc1 = nn.Linear(28 * 28, 128) # Input layer -> Hidden layer
self.fc2 = nn.Linear(128, 64) # Hidden layer -> Hidden layer
self.fc3 = nn.Linear(64, 10) # Hidden layer -> Output layer
self.activation = nn.ReLU() # Activation function
def forward(self, x):
x = x.view(-1, 28 * 28) # Reshape image to 1D tensor
x = self.activation(self.fc1(x)) # Forward pass
x = self.activation(self.fc2(x))
x = self.fc3(x)
return x
Initialize the Model and Set Loss Function and Optimizer
# Initialize Model model = SimpleNN() # Set Loss Function and Optimizer criterion = nn.CrossEntropyLoss() optimizer = optim.Adam(model.parameters(), lr=0.001)
Train the Model
Train the model iteratively while recording the loss and periodically evaluate the model’s performance.
# Train the Model
num_epochs = 5
for epoch in range(num_epochs):
for images, labels in train_loader:
optimizer.zero_grad() # Initialize gradients
outputs = model(images) # Forward pass
loss = criterion(outputs, labels) # Calculate loss
loss.backward() # Backpropagation
optimizer.step() # Update weights
print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
Evaluate the Model
Evaluate the accuracy of the model using the test dataset.
# Evaluate the Model
model.eval() # Set to evaluation mode
with torch.no_grad(): # Disable gradient calculation
correct = 0
total = 0
for images, labels in test_loader:
outputs = model(images)
_, predicted = torch.max(outputs.data, 1) # Predicted classes
total += labels.size(0) # Total sample count
correct += (predicted == labels).sum().item() # Count correct predictions
print(f'Accuracy of the model on the test images: {100 * correct / total:.2f}%')
Conclusion
In this course, we covered the basic concepts of deep learning and implemented a simple neural network model using PyTorch. Through hands-on practice, we learned about data preprocessing, model definition, training, and evaluation processes.
This provided an opportunity to gain a deep understanding of how deep learning works.
Furthermore, we can explore the world of deep learning by dealing with complex architectures, advanced optimization techniques, and various datasets.
Thank you!