Deep Learning PyTorch Course, Feature Map Visualization

In deep learning, it is important to understand how models learn from data. In particular,
feature maps are low-dimensional data generated in the intermediate layers of neural networks,
which serve as important indicators of what features the model is extracting from the input data.
In this article, we will explain in detail how to visualize feature maps using PyTorch,
and we will understand the process through hands-on practice.

1. What is a feature map?

A feature map refers to the output that is mapped through various filters generated by a convolutional neural network (CNN).
Each filter plays a role in detecting specific patterns or features in the input image, visually expressing
what information the model is learning in this process.

2. Why visualize feature maps?

The reasons for visualizing feature maps are as follows:

• To understand the model’s decision-making and enhance interpretability.
• To determine how sensitive the model is to specific features.
• To provide insights for error analysis and model improvement.

3. Installing required libraries

To visualize feature maps, PyTorch and a few additional libraries are needed.
You can install the necessary libraries using the command below.

pip install torch torchvision matplotlib

4. Preparing the dataset

We will use the MNIST dataset to process handwritten digit images. To do this,
we will load the data using the datasets module from torchvision.

import torch
import torchvision.transforms as transforms
from torchvision import datasets
from torch.utils.data import DataLoader

# Define transformations
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

# Download MNIST dataset
mnist_data = datasets.MNIST(root='./data', train=True, transform=transform, download=True)
data_loader = DataLoader(mnist_data, batch_size=64, shuffle=True)
    

5. Building a simple CNN model

We will build a simple CNN model to classify images. The model consists of the following:
Convolutional Layer -> ReLU -> Max Pooling -> Fully Connected Layer

import torch.nn as nn
import torch.nn.functional as F

class SimpleCNN(nn.Module):
    def __init__(self):
        super(SimpleCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 16, kernel_size=5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(16, 32, kernel_size=5)
        self.fc1 = nn.Linear(32 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 32 * 4 * 4)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

model = SimpleCNN()
    

6. Visualizing feature maps

We can extract and visualize feature maps from a specific layer of the model. Here, we will visualize
the feature maps of the first convolutional layer.

import matplotlib.pyplot as plt

# Get a batch of virtual images
data_iter = iter(data_loader)
images, labels = next(data_iter)

# Pass through the model and extract feature maps
with torch.no_grad():
    feature_maps = model.conv1(images)

# Visualize feature maps
def show_feature_maps(feature_maps):
    feature_maps = feature_maps[0].detach().numpy()
    num_feature_maps = feature_maps.shape[0]

    plt.figure(figsize=(15, 15))
    for i in range(num_feature_maps):
        plt.subplot(8, 8, i + 1)
        plt.imshow(feature_maps[i], cmap='gray')
        plt.axis('off')
    plt.show()

show_feature_maps(feature_maps)
    

7. Interpreting results

Through the visualization of feature maps, we can see what features the model is extracting from each input image.
Each feature map is the result of applying different filters, with specific patterns or shapes highlighted.
This allows us to gain insights into the model’s learning process.

8. Conclusion

In this tutorial, we introduced how to implement a simple CNN model using PyTorch and how to visualize
feature maps from the first convolutional layer. Visualizing feature maps is a useful tool for understanding
the internal workings of the model and gaining insights into the patterns being generated.

The fields of machine learning and deep learning continue to evolve, and these visualization techniques can
help us explain and improve complex models.
I encourage you to continue learning by tackling various topics similar to this one.