Deep Learning PyTorch Course, Logistic Regression and Linear Regression

In this course, we will explore two important regression analysis techniques, logistic regression and linear regression, which are fundamental concepts in deep learning. In this process, we will implement both models using PyTorch and examine how each technique is utilized.

Table of Contents

1. 1. Linear Regression
2. 2. Logistic Regression
3. 3. Conclusion

1. Linear Regression

Linear regression is a statistical method that models the relationship between input variables and output variables using a straight line. It performs predictions by finding the optimal line for a given set of data points.

1.1 Linear Regression Model Equation

The linear regression model can be represented by the following equation:

y = β0 + β1*x1 + β2*x2 + ... + βn*xn

Here, y is the predicted value, β0 is the intercept, and β1, β2, ..., βn are the regression coefficients. These regression coefficients are learned from the data.

1.2 Implementing Linear Regression in PyTorch

Now, let’s implement the linear regression model using PyTorch. We will create a simple linear regression model with the code below.

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt

# Generate data
x_data = np.array([[1], [2], [3], [4], [5]])
y_data = np.array([[2], [3], [5], [7], [11]])

# Convert to tensor
x_tensor = torch.Tensor(x_data)
y_tensor = torch.Tensor(y_data)

# Define linear regression model
model = nn.Linear(1, 1)

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Train the model
for epoch in range(100):
    model.train()
    
    optimizer.zero_grad()
    y_pred = model(x_tensor)
    loss = criterion(y_pred, y_tensor)
    loss.backward()
    optimizer.step()

# Visualize the results
plt.scatter(x_data, y_data, color='blue', label='Actual Data')
plt.plot(x_data, model(x_tensor).detach().numpy(), color='red', label='Regression Line')
plt.legend()
plt.title('Linear Regression Prediction')
plt.xlabel('x')
plt.ylabel('y')
plt.show()

The code above is an example of training a linear regression model using a simple dataset. After training the model, we can visualize and compare the actual data with the predicted values.

2. Logistic Regression

Logistic regression is a linear model designed to handle classification problems. It is primarily used in binary classification, where it applies a sigmoid function (logistic function) to the linear combination of inputs, converting the output into a probability value between 0 and 1.

2.1 Logistic Regression Model Equation

The logistic regression model can be represented by the following equations:

y = 1 / (1 + e^(-z))
z = β0 + β1*x1 + β2*x2 + ... + βn*xn

Here, z is the linear combination, and y is the probability of the class.

2.2 Implementing Logistic Regression in PyTorch

Now, let’s implement the logistic regression model using PyTorch. The code below is an example solving a simple binary classification problem.

# Generate data (binary classification)
from sklearn.datasets import make_classification
import torch.nn.functional as F

# Create binary classification dataset
X, y = make_classification(n_samples=100, n_features=2, n_classes=2, n_informative=2, n_redundant=0, random_state=42)
X_tensor = torch.Tensor(X)
y_tensor = torch.Tensor(y).view(-1, 1)

# Define logistic regression model
class LogisticRegressionModel(nn.Module):
    def __init__(self):
        super(LogisticRegressionModel, self).__init__()
        self.linear = nn.Linear(2, 1)

    def forward(self, x):
        return torch.sigmoid(self.linear(x))

# Define model, loss function, and optimizer
model = LogisticRegressionModel()
criterion = nn.BCELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Train the model
for epoch in range(100):
    model.train()
    
    optimizer.zero_grad()
    y_pred = model(X_tensor)
    
    loss = criterion(y_pred, y_tensor)
    loss.backward()
    optimizer.step()

# Predictions
model.eval()
with torch.no_grad():
    y_pred = model(X_tensor)

# Visualize predictions
predicted_classes = (y_pred.numpy() > 0.5).astype(int)
plt.scatter(X[y[:, 0] == 0][:, 0], X[y[:, 0] == 0][:, 1], color='blue', label='Class 0')
plt.scatter(X[y[:, 0] == 1][:, 0], X[y[:, 0] == 1][:, 1], color='red', label='Class 1')
plt.scatter(X[predicted_classes[:, 0] == 1][:, 0], X[predicted_classes[:, 0] == 1][:, 1], color='green', label='Predicted')
plt.legend()
plt.title('Logistic Regression Prediction')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.show()

The code above is an example of training a logistic regression model to solve a simple binary classification problem. After training the model, we can visualize and compare the actual classes with the predicted classes.

3. Conclusion

In this course, we covered logistic regression and linear regression. Both models were implemented using PyTorch with a simple dataset, highlighting the differences in the types of problems they address and their significance. These techniques form the foundation of machine learning and deep learning and are widely used in solving real-world problems. I hope this course broadens your understanding.