{"id":36057,"date":"2024-11-01T09:45:20","date_gmt":"2024-11-01T09:45:20","guid":{"rendered":"http:\/\/atmokpo.com\/w\/?p=36057"},"modified":"2024-11-01T09:45:20","modified_gmt":"2024-11-01T09:45:20","slug":"huggingface-transformer-training-course-bart-inference","status":"publish","type":"post","link":"https:\/\/atmokpo.com\/w\/36057\/","title":{"rendered":"huggingface transformer training course, BART inference"},"content":{"rendered":"<p><body><\/p>\n<p>With the advancement of machine learning, particularly in natural language processing (NLP), transformer models have shown innovative results in word embedding, sentence generation, and various other tasks. Among them, BART (Bidirectional and Auto-Regressive Transformers) has garnered attention as a model that demonstrates excellent performance across multiple NLP tasks such as text generation, summarization, and translation.<\/p>\n<h2>Introduction to BART<\/h2>\n<p>BART is a model developed by Facebook AI Research (Fair) that fundamentally combines two architectures: the Encoder-Decoder structure and modified language modeling. BART is trained by symmetrically transforming input text to restore the original text from noisy text. Thanks to this characteristic, BART is suitable for performing various language tasks that require adaptability, such as sentence summarization, translation, and question answering.<\/p>\n<h3>Main Features of BART<\/h3>\n<ul>\n<li><strong>Bidirectional Encoder:<\/strong> BART&#8217;s encoder can consider context information from both directions, thanks to the bidirectionality of the Transformer model.<\/li>\n<li><strong>Auto-Regressive Decoder:<\/strong> The decoder operates by considering all previous words to predict the next word.<\/li>\n<li><strong>Noise Removal:<\/strong> The model is trained by randomly masking or transforming parts of the text to remove this noise.<\/li>\n<\/ul>\n<h2>Hugging Face Transformers Library<\/h2>\n<p>The Hugging Face Transformers library provides an API that allows easy use of various transformer models, such as BART. The advantages of this library include:<\/p>\n<ul>\n<li>A variety of pre-trained models available<\/li>\n<li>An easy and intuitive API<\/li>\n<li>Advanced feature support for various NLP tasks<\/li>\n<\/ul>\n<h3>Installation Instructions<\/h3>\n<p>To install the Transformers library, use the following pip command:<\/p>\n<pre><code>pip install transformers<\/code><\/pre>\n<h2>Example of Using the BART Model<\/h2>\n<p>This section will explain how to perform text summarization using the BART model. Below is a simple example code:<\/p>\n<h3>Example Code<\/h3>\n<div class=\"example\">\n<pre><code>from transformers import BartTokenizer, BartForConditionalGeneration\n\n# Load the BART model and tokenizer\ntokenizer = BartTokenizer.from_pretrained('facebook\/bart-large-cnn')\nmodel = BartForConditionalGeneration.from_pretrained('facebook\/bart-large-cnn')\n\n# Text to summarize\ntext = \"\"\"\nDeep learning is a subset of machine learning that is based on artificial neural networks.\nIt is used for various applications such as image recognition, natural language processing,\nand more. Deep learning allows computers to learn from data in a hierarchical manner,\nenabling them to achieve high accuracy in various tasks.\n\"\"\"\n\n# Tokenize the text and input it to the summarization model\ninputs = tokenizer(text, return_tensors='pt', max_length=1024, truncation=True)\n\n# Generate the summary\nsummary_ids = model.generate(inputs['input_ids'], max_length=50, min_length=25, length_penalty=2.0, num_beams=4, early_stopping=True)\n\n# Decode the summarized text\nsummary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)\nprint(\"Summary:\", summary)\n<\/code><\/pre>\n<\/div>\n<h3>Code Explanation<\/h3>\n<ol>\n<li><strong>Import Libraries:<\/strong> Import the BART model and tokenizer from the `transformers` library.<\/li>\n<li><strong>Loading the Model and Tokenizer:<\/strong> Retrieve the pre-trained model and tokenizer from `facebook\/bart-large-cnn`.<\/li>\n<li><strong>Input Text:<\/strong> Set the long text to be summarized.<\/li>\n<li><strong>Tokenizing:<\/strong> Tokenize the input text and convert it to a tensor.<\/li>\n<li><strong>Generating Summary:<\/strong> Use the `generate` method to create the summary.<\/li>\n<li><strong>Output Result:<\/strong> Decode and print the generated summary.<\/li>\n<\/ol>\n<h2>Explanation of Control Parameters<\/h2>\n<p>The above code allows for adjusting the quality of the summary through various control parameters. Each parameter plays the following role:<\/p>\n<ul>\n<li><strong>max_length:<\/strong> The maximum length of the generated summary.<\/li>\n<li><strong>min_length:<\/strong> The minimum length of the generated summary.<\/li>\n<li><strong>length_penalty:<\/strong> A penalty for the length of the summary, giving lower scores for longer outputs to adjust the length.<\/li>\n<li><strong>num_beams:<\/strong> The number of beams to use in beam search, with higher values exploring more candidates.<\/li>\n<li><strong>early_stopping:<\/strong> Determines whether to stop the process when the optimal summary is generated.<\/li>\n<\/ul>\n<h2>Various Applications of BART<\/h2>\n<p>BART can be used for various NLP tasks beyond summarization. Here are some key use cases of BART:<\/p>\n<h3>1. Machine Translation<\/h3>\n<p>BART is effectively used in translation tasks to convert input text into another language, allowing users to perform translations from the source language to the target language.<\/p>\n<h3>2. Question Answering<\/h3>\n<p>BART demonstrates strong performance in generating answers to given questions.<\/p>\n<h3>3. Text Generation<\/h3>\n<p>BART can also be used to generate high-quality text in free-form text generation tasks.<\/p>\n<h2>Evaluating Model Performance<\/h2>\n<p>Various metrics can be used to evaluate the performance of the BART model. The ROUGE metric is commonly used. ROUGE measures the similarity between machine-generated summaries and human summaries, providing various metrics such as F1 score and recall.<\/p>\n<h3>Calculating ROUGE Scores<\/h3>\n<p>Below is an example of how to calculate ROUGE scores using Python:<\/p>\n<div class=\"example\">\n<pre><code>from rouge import Rouge\n\n# Human summary and model summary\nhuman_summary = \"Deep learning is a subset of machine learning.\"\nmodel_summary = summary  # The model summary generated above\n\n# Create a ROUGE evaluator object\nrouge = Rouge()\n\n# Calculate ROUGE scores\nscores = rouge.get_scores(model_summary, human_summary)\nprint(\"ROUGE Scores:\", scores)\n<\/code><\/pre>\n<\/div>\n<h2>Conclusion<\/h2>\n<p>The BART model is a highly useful tool applicable to various natural language processing tasks such as effective text summarization, translation, and question answering. It can be easily used through the Hugging Face Transformers library, and many researchers and developers are leveraging it to achieve innovative results in the field of NLP. Through this tutorial, I hope you gain an understanding of the basic concepts of BART and its usage, and acquire experience in performing text summarization.<\/p>\n<h2>References<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/transformers\/model_doc\/bart.html\">Hugging Face BART Documentation<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/1910.13461\">BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Processing<\/a><\/li>\n<\/ul>\n<p><\/body><\/p>\n","protected":false},"excerpt":{"rendered":"<p>With the advancement of machine learning, particularly in natural language processing (NLP), transformer models have shown innovative results in word embedding, sentence generation, and various other tasks. Among them, BART (Bidirectional and Auto-Regressive Transformers) has garnered attention as a model that demonstrates excellent performance across multiple NLP tasks such as text generation, summarization, and translation. &hellip; <a href=\"https:\/\/atmokpo.com\/w\/36057\/\" class=\"more-link\">\ub354 \ubcf4\uae30<span class=\"screen-reader-text\"> &#8220;huggingface transformer training course, BART inference&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[108],"tags":[],"class_list":["post-36057","post","type-post","status-publish","format-standard","hentry","category---en"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>huggingface transformer training course, BART inference - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/atmokpo.com\/w\/36057\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"huggingface transformer training course, BART inference - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\" \/>\n<meta property=\"og:description\" content=\"With the advancement of machine learning, particularly in natural language processing (NLP), transformer models have shown innovative results in word embedding, sentence generation, and various other tasks. Among them, BART (Bidirectional and Auto-Regressive Transformers) has garnered attention as a model that demonstrates excellent performance across multiple NLP tasks such as text generation, summarization, and translation. &hellip; \ub354 \ubcf4\uae30 &quot;huggingface transformer training course, BART inference&quot;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/atmokpo.com\/w\/36057\/\" \/>\n<meta property=\"og:site_name\" content=\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\" \/>\n<meta property=\"article:published_time\" content=\"2024-11-01T09:45:20+00:00\" \/>\n<meta name=\"author\" content=\"root\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@bebubo4\" \/>\n<meta name=\"twitter:site\" content=\"@bebubo4\" \/>\n<meta name=\"twitter:label1\" content=\"\uae00\uc4f4\uc774\" \/>\n\t<meta name=\"twitter:data1\" content=\"root\" \/>\n\t<meta name=\"twitter:label2\" content=\"\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04\" \/>\n\t<meta name=\"twitter:data2\" content=\"4\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/atmokpo.com\/w\/36057\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/atmokpo.com\/w\/36057\/\"},\"author\":{\"name\":\"root\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7\"},\"headline\":\"huggingface transformer training course, BART inference\",\"datePublished\":\"2024-11-01T09:45:20+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/atmokpo.com\/w\/36057\/\"},\"wordCount\":655,\"publisher\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#organization\"},\"articleSection\":[\"Using Hugging Face\"],\"inLanguage\":\"ko-KR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/atmokpo.com\/w\/36057\/\",\"url\":\"https:\/\/atmokpo.com\/w\/36057\/\",\"name\":\"huggingface transformer training course, BART inference - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\",\"isPartOf\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#website\"},\"datePublished\":\"2024-11-01T09:45:20+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/atmokpo.com\/w\/36057\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/atmokpo.com\/w\/36057\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/atmokpo.com\/w\/36057\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\ud648\",\"item\":\"https:\/\/atmokpo.com\/w\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"huggingface transformer training course, BART inference\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/atmokpo.com\/w\/#website\",\"url\":\"https:\/\/atmokpo.com\/w\/\",\"name\":\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/atmokpo.com\/w\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/atmokpo.com\/w\/#organization\",\"name\":\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\",\"url\":\"https:\/\/atmokpo.com\/w\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png\",\"contentUrl\":\"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png\",\"width\":400,\"height\":400,\"caption\":\"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8\"},\"image\":{\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/bebubo4\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7\",\"name\":\"root\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/atmokpo.com\/w\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g\",\"caption\":\"root\"},\"sameAs\":[\"http:\/\/atmokpo.com\/w\"],\"url\":\"https:\/\/atmokpo.com\/w\/author\/root\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"huggingface transformer training course, BART inference - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/atmokpo.com\/w\/36057\/","og_locale":"ko_KR","og_type":"article","og_title":"huggingface transformer training course, BART inference - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","og_description":"With the advancement of machine learning, particularly in natural language processing (NLP), transformer models have shown innovative results in word embedding, sentence generation, and various other tasks. Among them, BART (Bidirectional and Auto-Regressive Transformers) has garnered attention as a model that demonstrates excellent performance across multiple NLP tasks such as text generation, summarization, and translation. &hellip; \ub354 \ubcf4\uae30 \"huggingface transformer training course, BART inference\"","og_url":"https:\/\/atmokpo.com\/w\/36057\/","og_site_name":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","article_published_time":"2024-11-01T09:45:20+00:00","author":"root","twitter_card":"summary_large_image","twitter_creator":"@bebubo4","twitter_site":"@bebubo4","twitter_misc":{"\uae00\uc4f4\uc774":"root","\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04":"4\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/atmokpo.com\/w\/36057\/#article","isPartOf":{"@id":"https:\/\/atmokpo.com\/w\/36057\/"},"author":{"name":"root","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7"},"headline":"huggingface transformer training course, BART inference","datePublished":"2024-11-01T09:45:20+00:00","mainEntityOfPage":{"@id":"https:\/\/atmokpo.com\/w\/36057\/"},"wordCount":655,"publisher":{"@id":"https:\/\/atmokpo.com\/w\/#organization"},"articleSection":["Using Hugging Face"],"inLanguage":"ko-KR"},{"@type":"WebPage","@id":"https:\/\/atmokpo.com\/w\/36057\/","url":"https:\/\/atmokpo.com\/w\/36057\/","name":"huggingface transformer training course, BART inference - \ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","isPartOf":{"@id":"https:\/\/atmokpo.com\/w\/#website"},"datePublished":"2024-11-01T09:45:20+00:00","breadcrumb":{"@id":"https:\/\/atmokpo.com\/w\/36057\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/atmokpo.com\/w\/36057\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/atmokpo.com\/w\/36057\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\ud648","item":"https:\/\/atmokpo.com\/w\/en\/"},{"@type":"ListItem","position":2,"name":"huggingface transformer training course, BART inference"}]},{"@type":"WebSite","@id":"https:\/\/atmokpo.com\/w\/#website","url":"https:\/\/atmokpo.com\/w\/","name":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","description":"","publisher":{"@id":"https:\/\/atmokpo.com\/w\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/atmokpo.com\/w\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/atmokpo.com\/w\/#organization","name":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8","url":"https:\/\/atmokpo.com\/w\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/","url":"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png","contentUrl":"https:\/\/atmokpo.com\/w\/wp-content\/uploads\/2024\/11\/logo.png","width":400,"height":400,"caption":"\ub77c\uc774\ube0c\uc2a4\ub9c8\ud2b8"},"image":{"@id":"https:\/\/atmokpo.com\/w\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/bebubo4"]},{"@type":"Person","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/person\/91b6b3b138fbba0efb4ae64b1abd81d7","name":"root","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/atmokpo.com\/w\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/708197b41fc6435a7ce22d951b25d4a47e9e904270cb1f04682d4f025066f80c?s=96&d=mm&r=g","caption":"root"},"sameAs":["http:\/\/atmokpo.com\/w"],"url":"https:\/\/atmokpo.com\/w\/author\/root\/"}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts\/36057","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/comments?post=36057"}],"version-history":[{"count":1,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts\/36057\/revisions"}],"predecessor-version":[{"id":36058,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/posts\/36057\/revisions\/36058"}],"wp:attachment":[{"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/media?parent=36057"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/categories?post=36057"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/atmokpo.com\/w\/wp-json\/wp\/v2\/tags?post=36057"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}