Deep Learning PyTorch Course, Deep Neural Networks

In this course, we will start with the basic concepts of deep learning and learn how to implement Deep Neural Networks (DNN) using PyTorch. Deep Neural Networks are a crucial element in solving various problems in the field of artificial intelligence. Through this course, you will learn about the structure of deep neural networks, learning methods, and the basic usage of PyTorch.

1. What is Deep Learning?

Deep Learning is a field of Artificial Intelligence (AI) that processes and predicts data based on Artificial Neural Networks. A Deep Neural Network consists of multiple hidden layers, allowing it to learn complex patterns.

1.1. Key Features of Deep Learning

• Large Amounts of Data: Deep learning learns features from large amounts of data.
• Unsupervised Learning: Typically, deep learning learns the relationships between inputs and outputs through unsupervised learning.
• Complex Models: It can model non-linearity through a hierarchical structure.

2. Structure of Deep Neural Networks

A Deep Neural Network consists of an input layer, several hidden layers, and an output layer. Each layer consists of multiple nodes, and each node calculates the output value of that layer.

2.1. Components

2.1.1. Node

A node receives input, applies weights and biases, passes the result through an activation function, and generates the output.

2.1.2. Activation Function

An activation function is a function that non-linearly transforms the output of a node. Common activation functions include Sigmoid, Tanh, and ReLU (Rectified Linear Unit).

2.1.3. Forward Propagation

The forward propagation process is the procedure of passing input data through the network to calculate the output. In this process, the nodes of all hidden layers receive input values, apply weights and biases, and generate results through the activation function.

2.1.4. Backward Propagation

The backward propagation process is the procedure of adjusting weights and biases to reduce the error between the network’s output and the actual target values. We update the weights using Gradient Descent.

2.2. Formulas for Deep Neural Networks

The output of a deep neural network can be expressed as follows:

y = f(W * x + b)

Here, y represents the output, f is the activation function, W is the weight, x is the input, and b is the bias.

3. Basics of PyTorch

PyTorch is an open-source machine learning library developed by Facebook (now Meta). It features ease of use and dynamic computation graphs (Define-by-Run). We will learn how to implement deep neural networks with PyTorch.

3.1. Installation

PyTorch can be easily installed using pip.

pip install torch torchvision torchaudio

3.2. Basic Data Structure

The tensor provided by PyTorch is similar to a numpy array but supports GPU operations, making it optimized for deep learning. Here’s how to create tensors:

import torch

# 1D tensor
x = torch.tensor([1.0, 2.0, 3.0])
print(x)

# 2D tensor
y = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
print(y)

4. Implementing Deep Neural Networks

4.1. Preparing the Dataset

To practice deep learning, we can use an empty dataset. Here, we will use the MNIST dataset. MNIST is a handwritten digit dataset consisting of numbers from 0 to 9.

from torchvision import datasets, transforms

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

# Download MNIST dataset
train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=64, shuffle=True)

4.2. Defining the Model

Next, we define the deep neural network model. The nn.Module class allows you to easily create a custom neural network class.

import torch.nn as nn
import torch.nn.functional as F

class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(28 * 28, 128)
        self.fc2 = nn.Linear(128, 64)
        self.fc3 = nn.Linear(64, 10)

    def forward(self, x):
        x = x.view(-1, 28 * 28)  # Flatten the 2D tensor to 1D.
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)  # No activation function at the output layer
        return x

model = SimpleNN()

4.3. Setting Loss Function and Optimizer

We set the loss function and optimizer for model training. Here, we will use Cross Entropy Loss and Stochastic Gradient Descent (SGD) optimizer.

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

4.4. Training Loop

Finally, we will write a loop for model training. For each batch, we will perform forward propagation, loss calculation, and backward propagation.

num_epochs = 5

for epoch in range(num_epochs):
    for images, labels in train_loader:
        # Zero gradients
        optimizer.zero_grad()
        
        # Forward propagation
        outputs = model(images)
        
        # Calculate loss
        loss = criterion(outputs, labels)
        
        # Backward propagation
        loss.backward()
        
        # Update weights
        optimizer.step()
    
    print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')

5. Performance Evaluation

Once the model training is complete, we use the test dataset to evaluate the model’s performance and calculate accuracy. This will help verify how well the model has learned.

# Prepare test dataset
test_dataset = datasets.MNIST(root='./data', train=False, download=True, transform=transform)
test_loader = torch.utils.data.DataLoader(dataset=test_dataset, batch_size=64, shuffle=False)

model.eval()  # Switch to evaluation mode
correct = 0
total = 0

with torch.no_grad():
    for images, labels in test_loader:
        outputs = model(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print(f'Accuracy: {100 * correct / total:.2f}%')

Conclusion

In this course, we explored the basic concepts of deep neural networks and how to implement them using PyTorch. After preparing the dataset and defining the model, we built the model through actual training and evaluation processes. Based on your understanding of deep learning and PyTorch, I encourage you to try more complex networks or various models.

Research and application of deep neural networks will continue to develop, contributing to the advancement of the fields of machine learning and artificial intelligence.

Please continue your in-depth learning through additional materials and references. We wish all readers a successful journey in deep learning!