Deep Learning PyTorch Course, Difference between Using CPU and GPU

Deep learning has rapidly advanced in recent years, and this development relies heavily on powerful hardware. In particular, CPUs and GPUs play a vital role in the training and inference performance of deep learning models. This course will explore the structure, operating principles of CPUs and GPUs, and how to efficiently train deep learning models through PyTorch example code.

Structural Differences Between CPU and GPU

The CPU (Central Processing Unit) is the central processing unit of a computer, known for its excellent capability to perform complex calculations and handle various tasks. On the other hand, the GPU (Graphics Processing Unit) is hardware optimized for massive data parallel processing. Each of these processors has the following characteristics:

CPU: Typically has 4-16 cores, making it strong in multitasking by handling multiple programs simultaneously. However, due to the high performance of each core, it is very fast for single-threaded tasks.
GPU: Consists of thousands of small cores that excel at processing large datasets concurrently and performing repetitive calculations. Therefore, it is highly suitable for image and video processing as well as deep learning operations.

Usage of CPU and GPU in Deep Learning

In deep learning model training, thousands of parameters need to be optimized, and this process involves numerous matrix operations. In this case, the GPU demonstrates its capability for parallel processing by handling massive amounts of data at once, thus reducing training time. For example, training with a GPU can be tens to hundreds of times faster than with a CPU.

Using CPU and GPU in PyTorch

In PyTorch, users can easily choose between CPU and GPU. By default, the CPU is used, but when a GPU is available, it can be utilized with just a few simple changes in the code. Let’s take a look at this through the example code below.

Example: Training a Simple Neural Network Model


import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms

# Data preparation
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

train_dataset = datasets.MNIST(root='./data', train=True, transform=transform, download=True)
train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=64, shuffle=True)

# Neural network model definition
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(28 * 28, 128)
        self.fc2 = nn.Linear(128, 10)
    
    def forward(self, x):
        x = x.view(-1, 28 * 28)  # flatten
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Device configuration
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = SimpleNN().to(device)

# Loss function and optimizer configuration
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Model training
for epoch in range(5):  # Number of training epochs
    for images, labels in train_loader:
        images, labels = images.to(device), labels.to(device)  # Move data to GPU
        optimizer.zero_grad()   # Gradient initialization
        outputs = model(images) # Predictions
        loss = criterion(outputs, labels) # Loss calculation
        loss.backward()         # Backpropagation
        optimizer.step()        # Weight update
    
    print(f'Epoch [{epoch + 1}/5], Loss: {loss.item():.4f}')

Code Explanation

Data preparation: Loads and preprocesses the MNIST dataset into a DataLoader.
Neural network model definition: Defines a simple two-layer structure neural network.
Device configuration: Uses the GPU if available; otherwise, it uses the CPU.
Model training: Trains using the defined data and model, ensuring to move data to the GPU.

Performance Comparison of CPU and GPU

The performance advantage of using a GPU can be confirmed through various measurements. Typically, both CPU and GPU show differences in terms of training time and accuracy. Below is an example of training time when using CPU and GPU:


import time

# CPU performance test
device_cpu = torch.device('cpu')
model_cpu = SimpleNN().to(device_cpu)

start_time = time.time()
for epoch in range(5):
    for images, labels in train_loader:
        images, labels = images.to(device_cpu), labels.to(device_cpu)
        optimizer.zero_grad()
        outputs = model_cpu(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
end_time = time.time()
print(f'CPU Training Time: {end_time - start_time:.2f} seconds')

# GPU performance test
device_gpu = torch.device('cuda')
model_gpu = SimpleNN().to(device_gpu)

start_time = time.time()
for epoch in range(5):
    for images, labels in train_loader:
        images, labels = images.to(device_gpu), labels.to(device_gpu)
        optimizer.zero_grad()
        outputs = model_gpu(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
end_time = time.time()
print(f'GPU Training Time: {end_time - start_time:.2f} seconds')

Running the code above allows us to compare the training times of CPU and GPU. Generally, the GPU demonstrates faster training performance, but the complexity of the model, size of the data, and hardware performance can lead to differences.

Conclusion

To train deep learning models efficiently, it is essential to understand the characteristics and advantages of CPUs and GPUs. While the CPU provides versatility, the GPU is optimized for effectively handling massive data processing. Therefore, if you choose the hardware that suits your project and write code accordingly using PyTorch, you will be able to build deep learning models more efficiently.

Additionally, when utilizing GPUs, it is important to recognize the limitations of GPU memory and, if necessary, adjust mini-batches to suit your needs. These considerations will enhance the utility of PyTorch and deep learning.