{"id":36537,"date":"2024-11-01T09:49:21","date_gmt":"2024-11-01T09:49:21","guid":{"rendered":"http:\/\/atmokpo.com\/w\/?p=36537"},"modified":"2024-11-01T11:52:48","modified_gmt":"2024-11-01T11:52:48","slug":"deep-learning-pytorch-course-performance-optimization-using-dropout","status":"publish","type":"post","link":"https:\/\/atmokpo.com\/w\/36537\/","title":{"rendered":"Deep Learning PyTorch Course, Performance Optimization using Dropout"},"content":{"rendered":"<p><body><\/p>\n<p>Overfitting is one of the common problems encountered when building deep learning models. Overfitting occurs when a model is excessively fitted to the training data, resulting in a decreased ability to generalize to new data. Although there are various methods to address this issue, <strong>Dropout<\/strong> is known to be particularly effective. In this post, we will explore the concept of dropout and how to implement it in PyTorch.<\/p>\n<h2>What is Dropout?<\/h2>\n<p>Dropout is a method for optimizing the learning process of a neural network by randomly deactivating some neurons. This prevents the model from relying too heavily on specific neurons, thereby preventing overfitting and creating a more generalized model. Specifically, dropout operates in the following manner:<\/p>\n<ul>\n<li>During training, the output of each neuron is probabilistically set to 0.<\/li>\n<li>The dropout rate (p) represents the proportion of neurons to which dropout is applied, typically using values between 0.2 and 0.5.<\/li>\n<li>When the model is evaluated, all neurons are used, and the output is scaled according to the dropout rate.<\/li>\n<\/ul>\n<h2>Effects of Dropout<\/h2>\n<p>Applying dropout provides the following advantages:<\/p>\n<ol>\n<li><strong>Prevention of Overfitting:<\/strong> By randomly deactivating neurons, it prevents the model from learning to fit specific patterns.<\/li>\n<li><strong>Ensemble Effect:<\/strong> Dropout provides the effect of training different sub-models, resulting in performance similar to ensemble models.<\/li>\n<li><strong>Simple Implementation:<\/strong> It can be applied relatively easily, making it widely used in various models.<\/li>\n<\/ol>\n<h2>PyTorch Example Code Using Dropout<\/h2>\n<p>Now, let&#8217;s learn how to train a deep learning model using dropout. In the following example, we will implement a digit classification model using the MNIST dataset.<\/p>\n<h3>1. Preparing the Dataset<\/h3>\n<pre><code>import torch\nimport torchvision.transforms as transforms\nimport torchvision.datasets as datasets\nfrom torch.utils.data import DataLoader\n\n# Define transformations for the dataset\ntransform = transforms.Compose([transforms.ToTensor(),\n                                transforms.Normalize((0.5,), (0.5,))])\n\n# Load the MNIST dataset\ntrain_dataset = datasets.MNIST(root='.\/data', train=True, transform=transform, download=True)\ntest_dataset = datasets.MNIST(root='.\/data', train=False, transform=transform, download=True)\n\n# Define data loaders\ntrain_loader = DataLoader(dataset=train_dataset, batch_size=64, shuffle=True)\ntest_loader = DataLoader(dataset=test_dataset, batch_size=64, shuffle=False)<\/code><\/pre>\n<h3>2. Defining the Model<\/h3>\n<p>Next, we define a neural network model that includes dropout.<\/p>\n<pre><code>import torch.nn as nn\nimport torch.nn.functional as F\n\nclass SimpleCNN(nn.Module):\n    def __init__(self):\n        super(SimpleCNN, self).__init__()\n        self.conv1 = nn.Conv2d(1, 32, kernel_size=3)\n        self.conv2 = nn.Conv2d(32, 64, kernel_size=3)\n        self.fc1 = nn.Linear(64*6*6, 128)\n        self.fc2 = nn.Linear(128, 10)\n        self.dropout = nn.Dropout(p=0.5)  # Setting dropout rate\n\n    def forward(self, x):\n        x = F.relu(self.conv1(x))\n        x = F.max_pool2d(x, 2)\n        x = F.relu(self.conv2(x))\n        x = F.max_pool2d(x, 2)\n        x = x.view(x.size(0), -1)  # flatten\n        x = F.relu(self.fc1(x))\n        x = self.dropout(x)  # Apply dropout\n        x = self.fc2(x)\n        return x<\/code><\/pre>\n<h3>3. Training the Model<\/h3>\n<p>To train the model, we define a loss function and optimizer, and conduct training over multiple epochs.<\/p>\n<pre><code>import torch.optim as optim\n\n# Define the model, loss function, and optimizer\nmodel = SimpleCNN()\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=0.001)\n\n# Training function\ndef train(model, train_loader, criterion, optimizer, epochs=5):\n    model.train()\n    for epoch in range(epochs):\n        running_loss = 0.0\n        for images, labels in train_loader:\n            optimizer.zero_grad()\n            outputs = model(images)\n            loss = criterion(outputs, labels)\n            loss.backward()\n            optimizer.step()\n            running_loss += loss.item()\n        print(f'Epoch [{epoch+1}\/{epochs}], Loss: {running_loss\/len(train_loader):.4f}')\n\n# Train the model\ntrain(model, train_loader, criterion, optimizer, epochs=5)<\/code><\/pre>\n<h3>4. Evaluating the Model<\/h3>\n<p>We evaluate the trained model to assess its performance.<\/p>\n<pre><code>def test(model, test_loader):\n    model.eval()\n    correct = 0\n    total = 0\n    with torch.no_grad():\n        for images, labels in test_loader:\n            outputs = model(images)\n            _, predicted = torch.max(outputs.data, 1)\n            total += labels.size(0)\n            correct += (predicted == labels).sum().item()\n    print(f'Accuracy of the model on the test images: {100 * correct \/ total:.2f}%')\n\n# Evaluate the model\ntest(model, test_loader)<\/code><\/pre>\n<h2>Conclusion<\/h2>\n<p>Dropout is an effective method for preventing overfitting and enhancing performance in deep learning models. This post demonstrated the implementation of dropout using PyTorch through an example of classifying the MNIST dataset. This is just a basic example; in practice, various architectures and dropout rates can be adjusted to design more complex models.<\/p>\n<h2>References<\/h2>\n<ul>\n<li><a href=\"https:\/\/pytorch.org\/docs\/stable\/index.html\">PyTorch Documentation<\/a><\/li>\n<li><a href=\"http:\/\/yann.lecun.com\/exdb\/mnist\/\">MNIST Dataset<\/a><\/li>\n<\/ul>\n<p><\/body><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overfitting is one of the common problems encountered when building deep learning models. Overfitting occurs when a model is excessively fitted to the training data, resulting in a decreased ability to generalize to new data. Although there are various methods to address this issue, Dropout is known to be particularly effective. In this post, we &hellip; <a href=\"https:\/\/atmokpo.com\/w\/36537\/\" class=\"more-link\">\ub354 \ubcf4\uae30<span class=\"screen-reader-text\"> &#8220;Deep Learning PyTorch Course, Performance Optimization using Dropout&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[149],"tags":[],"class_list":["post-36537","post","type-post","status-publish","format-standard","hentry","category-pytorch-study"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Deep Learning PyTorch Course, Performance Optimization using Dropout - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/atmokpo.com\/w\/36537\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Deep Learning PyTorch Course, Performance Optimization using Dropout - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\" \/>\n<meta property=\"og:description\" content=\"Overfitting is one of the common problems encountered when building deep learning models. Overfitting occurs when a model is excessively fitted to the training data, resulting in a decreased ability to generalize to new data. Although there are various methods to address this issue, Dropout is known to be particularly effective. In this post, we &hellip; \ub354 \ubcf4\uae30 &quot;Deep Learning PyTorch Course, Performance Optimization using Dropout&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/atmokpo.com\/w\/36537\/\" \/>\n<meta property=\"og:site_name\" content=\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\" \/>\n<meta property=\"article:published_time\" content=\"2024-11-01T09:49:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-11-01T11:52:48+00:00\" \/>\n<meta name=\"author\" content=\"root\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@bebubo4\" \/>\n<meta name=\"twitter:site\" content=\"@bebubo4\" \/>\n<meta name=\"twitter:label1\" content=\"\uae00\uc4f4\uc774\" \/>\n\t<meta name=\"twitter:data1\" content=\"root\" \/>\n\t<meta name=\"twitter:label2\" content=\"\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04\" \/>\n\t<meta name=\"twitter:data2\" content=\"3\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/atmokpo.com\/w\/36537\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/atmokpo.com\/w\/36537\/\"},\"author\":{\"name\":\"root\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7\"},\"headline\":\"Deep Learning PyTorch Course, Performance Optimization using Dropout\",\"datePublished\":\"2024-11-01T09:49:21+00:00\",\"dateModified\":\"2024-11-01T11:52:48+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/atmokpo.com\/w\/36537\/\"},\"wordCount\":371,\"publisher\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#organization\"},\"articleSection\":[\"PyTorch Study\"],\"inLanguage\":\"ko-KR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/atmokpo.com\/w\/36537\/\",\"url\":\"https:\/\/atmokpo.com\/w\/36537\/\",\"name\":\"Deep Learning PyTorch Course, Performance Optimization using Dropout - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\",\"isPartOf\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#website\"},\"datePublished\":\"2024-11-01T09:49:21+00:00\",\"dateModified\":\"2024-11-01T11:52:48+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/atmokpo.com\/w\/36537\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/atmokpo.com\/w\/36537\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/atmokpo.com\/w\/36537\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\ud648\",\"item\":\"https:\/\/atmokpo.com\/w\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Deep Learning PyTorch Course, Performance Optimization using Dropout\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/atmokpo.com\/w\/#website\",\"url\":\"https:\/\/atmokpo.com\/w\/\",\"name\":\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/atmokpo.com\/w\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/atmokpo.com\/w\/#organization\",\"name\":\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\",\"url\":\"https:\/\/atmokpo.com\/w\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png\",\"contentUrl\":\"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png\",\"width\":400,\"height\":400,\"caption\":\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\"},\"image\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/bebubo4\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7\",\"name\":\"root\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g\",\"caption\":\"root\"},\"sameAs\":[\"http:\/\/atmokpo.com\/w\"],\"url\":\"https:\/\/atmokpo.com\/w\/author\/root\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Deep Learning PyTorch Course, Performance Optimization using Dropout - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/atmokpo.com\/w\/36537\/","og_locale":"ko_KR","og_type":"article","og_title":"Deep Learning PyTorch Course, Performance Optimization using Dropout - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","og_description":"Overfitting is one of the common problems encountered when building deep learning models. Overfitting occurs when a model is excessively fitted to the training data, resulting in a decreased ability to generalize to new data. Although there are various methods to address this issue, Dropout is known to be particularly effective. In this post, we &hellip; \ub354 \ubcf4\uae30 \"Deep Learning PyTorch Course, Performance Optimization using Dropout\"","og_url":"https:\/\/atmokpo.com\/w\/36537\/","og_site_name":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","article_published_time":"2024-11-01T09:49:21+00:00","article_modified_time":"2024-11-01T11:52:48+00:00","author":"root","twitter_card":"summary_large_image","twitter_creator":"@bebubo4","twitter_site":"@bebubo4","twitter_misc":{"\uae00\uc4f4\uc774":"root","\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04":"3\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/atmokpo.com\/w\/36537\/#article","isPartOf":{"@id":"https:\/\/atmokpo.com\/w\/36537\/"},"author":{"name":"root","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7"},"headline":"Deep Learning PyTorch Course, Performance Optimization using Dropout","datePublished":"2024-11-01T09:49:21+00:00","dateModified":"2024-11-01T11:52:48+00:00","mainEntityOfPage":{"@id":"https:\/\/atmokpo.com\/w\/36537\/"},"wordCount":371,"publisher":{"@id":"https:\/\/atmokpo.com\/w\/#organization"},"articleSection":["PyTorch Study"],"inLanguage":"ko-KR"},{"@type":"WebPage","@id":"https:\/\/atmokpo.com\/w\/36537\/","url":"https:\/\/atmokpo.com\/w\/36537\/","name":"Deep Learning PyTorch Course, Performance Optimization using Dropout - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","isPartOf":{"@id":"https:\/\/atmokpo.com\/w\/#website"},"datePublished":"2024-11-01T09:49:21+00:00","dateModified":"2024-11-01T11:52:48+00:00","breadcrumb":{"@id":"https:\/\/atmokpo.com\/w\/36537\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/atmokpo.com\/w\/36537\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/atmokpo.com\/w\/36537\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\ud648","item":"https:\/\/atmokpo.com\/w\/en\/"},{"@type":"ListItem","position":2,"name":"Deep Learning PyTorch Course, Performance Optimization using Dropout"}]},{"@type":"WebSite","@id":"https:\/\/atmokpo.com\/w\/#website","url":"https:\/\/atmokpo.com\/w\/","name":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","description":"","publisher":{"@id":"https:\/\/atmokpo.com\/w\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/atmokpo.com\/w\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/atmokpo.com\/w\/#organization","name":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","url":"https:\/\/atmokpo.com\/w\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/","url":"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png","contentUrl":"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png","width":400,"height":400,"caption":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8"},"image":{"@id":"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/bebubo4"]},{"@type":"Person","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7","name":"root","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g","caption":"root"},"sameAs":["http:\/\/atmokpo.com\/w"],"url":"https:\/\/atmokpo.com\/w\/author\/root\/"}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts\/36537","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/comments?post=36537"}],"version-history":[{"count":1,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts\/36537\/revisions"}],"predecessor-version":[{"id":36538,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts\/36537\/revisions\/36538"}],"wp:attachment":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/media?parent=36537"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/categories?post=36537"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/tags?post=36537"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}