Today, artificial intelligence (AI) and machine learning (ML) play a crucial role in various industries and research fields. In particular, deep learning has established itself as a powerful tool for learning and predicting complex data patterns. PyTorch is an open-source deep learning library that helps build these deep learning models easily and intuitively. In this course, we will closely examine the basic concepts and implementation methods of machine learning algorithms using PyTorch.
1. Overview of Machine Learning
Machine learning is a collection of algorithms that analyze and learn from data to make predictions or decisions. One important classification of machine learning is supervised learning. In supervised learning, input and corresponding correct answers (labels) are provided to train the model. Here, we will explain linear regression, a representative machine learning algorithm, as an example.
2. Linear Regression
Linear regression is a method for modeling the linear relationship between input features and outputs. Mathematically, it is expressed as follows:
y = wx + b
Here, y represents the predicted value, w represents the weight, x represents the input value, and b represents the bias. The goal during the learning process is to find the optimal w and b. To do this, a loss function is defined and minimized. Generally, Mean Squared Error (MSE) is used.
2.1. Implementing Linear Regression with PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
# Data generation
np.random.seed(42)
x_numpy = np.random.rand(100, 1) * 10 # Random numbers from 0 to 10
y_numpy = 2.5 * x_numpy + np.random.randn(100, 1) # y = 2.5x + noise
# Convert NumPy arrays to PyTorch tensors
x_train = torch.FloatTensor(x_numpy)
y_train = torch.FloatTensor(y_numpy)
# Define linear regression model
model = nn.Linear(1, 1)
# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)
# Learning process
num_epochs = 100
for epoch in range(num_epochs):
model.train()
# Calculate predicted values
y_pred = model(x_train)
# Calculate loss
loss = criterion(y_pred, y_train)
# Print elapsed loss
if (epoch + 1) % 10 == 0:
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')
# Initialize gradients, backpropagate and update weights
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Visualizing predictions
plt.scatter(x_numpy, y_numpy, label='Data')
plt.plot(x_numpy, model(x_train).detach().numpy(), color='red', label='Prediction')
plt.legend()
plt.show()
The code above creates a linear regression model, trains it based on data, and finally visualizes the prediction results. As learning progresses, the loss decreases. This indicates that the model is successfully learning the patterns in the data.
3. Deep Neural Networks
In deep learning, multiple layers of artificial neural networks are used to learn more complex data patterns. Such deep learning models can be implemented using a simple Multi-Layer Perceptron (MLP) structure. An MLP consists of an input layer, hidden layers, and an output layer, with each layer made up of nodes. Each node is connected to the nodes of the previous layer, and nonlinearity is introduced through an activation function.
3.1. Implementing an MLP Model
class NeuralNetwork(nn.Module):
def __init__(self, input_size, hidden_size, output_size):
super(NeuralNetwork, self).__init__()
self.fc1 = nn.Linear(input_size, hidden_size) # First hidden layer
self.fc2 = nn.Linear(hidden_size, output_size) # Output layer
self.relu = nn.ReLU() # Activation function
def forward(self, x):
out = self.fc1(x)
out = self.relu(out)
out = self.fc2(out)
return out
# Prepare dataset
from sklearn.datasets import make_moons
x, y = make_moons(n_samples=1000, noise=0.2)
# Convert NumPy arrays to PyTorch tensors
x_train = torch.FloatTensor(x)
y_train = torch.FloatTensor(y).view(-1, 1)
# Define model, loss function, and optimizer
input_size = 2
hidden_size = 10
output_size = 1
model = NeuralNetwork(input_size, hidden_size, output_size)
criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
# Learning process
num_epochs = 1000
for epoch in range(num_epochs):
model.train()
# Calculate predicted values
y_pred = model(x_train)
# Calculate loss
loss = criterion(y_pred, y_train)
# Initialize gradients, backpropagate and update weights
optimizer.zero_grad()
loss.backward()
optimizer.step()
# Print elapsed loss
if (epoch + 1) % 100 == 0:
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')
The code above defines a basic Multi-Layer Perceptron model and demonstrates how to train it on a ‘make_moons’ dataset containing 1000 samples. ‘BCEWithLogitsLoss’ is a commonly used loss function for binary classification. You can observe that the loss decreases as the model learns.
4. Convolutional Neural Networks (CNN)
CNNs are primarily used for 2D data such as images. CNNs are composed of convolutional layers and pooling layers, which are effective in extracting features from images. Convolutional layers capture local characteristics from images, while pooling layers reduce the size of images to decrease computational workload.
4.1. Implementing a CNN Model
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1) # First convolutional layer
self.pool = nn.MaxPool2d(kernel_size=2, stride=2) # Pooling layer
self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1) # Second convolutional layer
self.fc1 = nn.Linear(64 * 7 * 7, 128) # First fully connected layer
self.fc2 = nn.Linear(128, 10) # Output layer
def forward(self, x):
x = self.pool(F.relu(self.conv1(x))) # First convolution + pooling
x = self.pool(F.relu(self.conv2(x))) # Second convolution + pooling
x = x.view(-1, 64 * 7 * 7) # Flatten
x = F.relu(self.fc1(x)) # First fully connected layer
x = self.fc2(x) # Output layer
return x
# Load example data (MNIST)
import torchvision.transforms as transforms
from torchvision import datasets
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])
train_dataset = datasets.MNIST(root='./data', train=True, transform=transform, download=True)
train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=64, shuffle=True)
# Define model, loss function, and optimizer
model = CNN()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
# Learning process
num_epochs = 5
for epoch in range(num_epochs):
for images, labels in train_loader:
optimizer.zero_grad() # Initialize gradients
outputs = model(images) # Calculate predicted values
loss = criterion(outputs, labels) # Calculate loss
loss.backward() # Backpropagate
optimizer.step() # Update weights
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')
The code above constructs a simple CNN model and demonstrates training it on the MNIST dataset. CNNs effectively learn features of images through convolution and pooling operations.
5. Conclusion
In this course, we have implemented linear regression in machine learning and various deep learning models using PyTorch. Deep learning is very useful for learning complex data, and PyTorch is a valuable tool in that process. I hope you experiment with various models using PyTorch and gain a deeper understanding of the data.
6. Additional Resources
If you would like to know more, please refer to the following materials: