\n GAN is a model proposed by Ian Goodfellow in 2014, consisting of two neural networks: the generator and the discriminator, that compete with each other. Through this competition, the generator produces data that looks real.\n <\/p>\n
\n GAN consists of two neural networks. The generator takes a random noise vector as input and generates fake data, while the discriminator distinguishes whether the input data is real or generated. The generator and discriminator are trained with their respective objectives.\n <\/p>\n
\n The loss function of GAN is used to evaluate the performance of the generator and the discriminator. The generator tries to fool the discriminator, and the discriminator works to distinguish between the two.
\n \\[
\n \\text{Loss}_D = – \\mathbb{E}_{x \\sim p_{data}(x)}[\\log(D(x))] – \\mathbb{E}_{z \\sim p_z(z)}[\\log(1 – D(G(z)))]
\n \\]
\n \\[
\n \\text{Loss}_G = – \\mathbb{E}_{z \\sim p_z(z)}[\\log(D(G(z)))]
\n \\]\n <\/p>\n
The following is a simple GAN implemented using PyTorch:<\/p>\n
\n \nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torchvision import datasets, transforms\n\n# Define hyperparameters\nlatent_size = 100\nbatch_size = 64\nnum_epochs = 200\nlearning_rate = 0.0002\n\n# Load dataset\ntransform = transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.5,), (0.5,))\n])\n\ndataset = datasets.MNIST(root='.\/data', train=True, download=True, transform=transform)\ndata_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=True)\n\n# Define generator\nclass Generator(nn.Module):\n def __init__(self):\n super(Generator, self).__init__()\n self.model = nn.Sequential(\n nn.Linear(latent_size, 256),\n nn.ReLU(),\n nn.Linear(256, 512),\n nn.ReLU(),\n nn.Linear(512, 1024),\n nn.ReLU(),\n nn.Linear(1024, 28*28),\n nn.Tanh()\n )\n\n def forward(self, z):\n return self.model(z).view(-1, 1, 28, 28)\n\n# Define discriminator\nclass Discriminator(nn.Module):\n def __init__(self):\n super(Discriminator, self).__init__()\n self.model = nn.Sequential(\n nn.Linear(28*28, 1024),\n nn.LeakyReLU(0.2),\n nn.Linear(1024, 512),\n nn.LeakyReLU(0.2),\n nn.Linear(512, 256),\n nn.LeakyReLU(0.2),\n nn.Linear(256, 1),\n nn.Sigmoid()\n )\n\n def forward(self, img):\n return self.model(img.view(-1, 28*28))\n\n# Initialize models\ngenerator = Generator()\ndiscriminator = Discriminator()\n\n# Define loss function and optimizers\ncriterion = nn.BCELoss()\noptimizer_G = optim.Adam(generator.parameters(), lr=learning_rate)\noptimizer_D = optim.Adam(discriminator.parameters(), lr=learning_rate)\n\n# Training process\nfor epoch in range(num_epochs):\n for i, (imgs, _) in enumerate(data_loader):\n # Real and fake image labels\n real_imgs = imgs\n real_labels = torch.ones(imgs.size(0), 1) # Real labels\n fake_labels = torch.zeros(imgs.size(0), 1) # Fake labels\n\n # Train discriminator\n optimizer_D.zero_grad()\n outputs = discriminator(real_imgs)\n d_loss_real = criterion(outputs, real_labels)\n\n z = torch.randn(imgs.size(0), latent_size)\n fake_imgs = generator(z)\n outputs = discriminator(fake_imgs.detach())\n d_loss_fake = criterion(outputs, fake_labels)\n\n d_loss = d_loss_real + d_loss_fake\n d_loss.backward()\n optimizer_D.step()\n\n # Train generator\n optimizer_G.zero_grad()\n outputs = discriminator(fake_imgs)\n g_loss = criterion(outputs, real_labels)\n g_loss.backward()\n optimizer_G.step()\n\n print(f\"Epoch [{epoch}\/{num_epochs}], d_loss: {d_loss.item()}, g_loss: {g_loss.item()}\")\n <\/code>\n <\/pre>\n2. Autoencoder<\/h2>\n
\n Autoencoders are an unsupervised learning method that compresses and reconstructs input data. They aim to produce outputs that are the same as the inputs while learning features to compress the data.\n <\/p>\n
2.1 Structure of Autoencoder<\/h3>\n
\n An autoencoder is divided into two parts: an encoder and a decoder. The encoder transforms the input into a low-dimensional latent representation, while the decoder uses this latent representation to reconstruct the original input.\n <\/p>\n
2.2 Loss Function of Autoencoder<\/h3>\n
Autoencoders mainly use Mean Squared Error (MSE) as the loss function to minimize the difference between the inputs and outputs.
\n \\[
\n \\text{Loss} = \\frac{1}{N} \\sum_{i=1}^N (x_i – \\hat{x}_i)^2
\n \\]\n <\/p>\n
2.3 Autoencoder Example Code<\/h3>\n
The following is a simple implementation of an autoencoder using PyTorch:<\/p>\n
\n \nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torchvision import datasets, transforms\n\n# Define hyperparameters\nbatch_size = 64\nnum_epochs = 20\nlearning_rate = 0.001\n\n# Load dataset\ntransform = transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.5,), (0.5,))\n])\n\ndataset = datasets.MNIST(root='.\/data', train=True, download=True, transform=transform)\ndata_loader = torch.utils.data.DataLoader(dataset, batch_size=batch_size, shuffle=True)\n\n# Define autoencoder\nclass Autoencoder(nn.Module):\n def __init__(self):\n super(Autoencoder, self).__init__()\n self.encoder = nn.Sequential(\n nn.Linear(28*28, 128),\n nn.ReLU(),\n nn.Linear(128, 64),\n nn.ReLU()\n )\n self.decoder = nn.Sequential(\n nn.Linear(64, 128),\n nn.ReLU(),\n nn.Linear(128, 28*28),\n nn.Sigmoid()\n )\n\n def forward(self, x):\n x = x.view(-1, 28*28)\n encoded = self.encoder(x)\n reconstructed = self.decoder(encoded)\n return reconstructed.view(-1, 1, 28, 28)\n\n# Initialize model\nautoencoder = Autoencoder()\n\n# Define loss function and optimizer\ncriterion = nn.MSELoss()\noptimizer = optim.Adam(autoencoder.parameters(), lr=learning_rate)\n\n# Training process\nfor epoch in range(num_epochs):\n for imgs, _ in data_loader:\n optimizer.zero_grad()\n outputs = autoencoder(imgs)\n loss = criterion(outputs, imgs)\n loss.backward()\n optimizer.step()\n\n print(f\"Epoch [{epoch}\/{num_epochs}], Loss: {loss.item()}\")\n <\/code>\n <\/pre>\n3. Conclusion<\/h2>\n
\n GANs and autoencoders are powerful deep learning techniques for image generation, data representation, and compression. By understanding and practicing their structures and training methods, one can build a higher level of deep learning knowledge.
\n These models can be applied to various fields and can yield better results with customized architectures.\n <\/p>\n
<\/body><\/p>\n","protected":false},"excerpt":{"rendered":"