{"id":29700,"date":"2024-10-28T01:56:09","date_gmt":"2024-10-28T01:56:09","guid":{"rendered":"http:\/\/atmokpo.com\/w\/?p=29700"},"modified":"2024-11-26T06:51:40","modified_gmt":"2024-11-26T06:51:40","slug":"%ed%97%88%ea%b9%85%ed%8e%98%ec%9d%b4%ec%8a%a4-%ed%8a%b8%eb%a0%8c%ec%8a%a4%ed%8f%ac%eb%a8%b8-%ed%99%9c%ec%9a%a9%ea%b0%95%ec%a2%8c-%ed%8c%8c%ec%9d%b4%ed%86%a0%ec%b9%98-%ec%82%ac%ec%a0%84%ed%95%99","status":"publish","type":"post","link":"https:\/\/atmokpo.com\/w\/29700\/","title":{"rendered":"\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ud65c\uc6a9\uac15\uc88c, \ud30c\uc774\ud1a0\uce58 \uc0ac\uc804\ud559\uc2b5"},"content":{"rendered":"

<\/p>\n

\ub525\ub7ec\ub2dd\uc758 \ubc1c\uc804\uacfc \ud568\uaed8 \uc790\uc5f0\uc5b4 \ucc98\ub9ac(NLP) \ubd84\uc57c\uc5d0\uc11c\ub3c4 \ud070 \ud601\uc2e0\uc774 \uc77c\uc5b4\ub098\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uadf8 \uc911\uc5d0\uc11c\ub3c4 \ud5c8\uae45\ud398\uc774\uc2a4(Hugging Face)\uc5d0\uc11c \uc81c\uacf5\ud558\ub294 \ud2b8\ub80c\uc2a4\ud3ec\uba38(Transformers) \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 \uc0ac\uc804\ud559\uc2b5(pre-trained)\ub41c \uc5b8\uc5b4 \ubaa8\ub378\ub4e4\uc744 \uc190\uc27d\uac8c \ud65c\uc6a9\ud560 \uc218 \uc788\ub3c4\ub85d \ub3c4\uc640\uc90d\ub2c8\ub2e4. \ubcf8 \uae00\uc5d0\uc11c\ub294 \ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38\uc758 \uae30\ubcf8 \uac1c\ub150\uacfc \uc0ac\uc6a9 \ubc29\ubc95\uc5d0 \ub300\ud574 \uc18c\uac1c\ud558\uace0, \ud30c\uc774\ud1a0\uce58(PyTorch) \uae30\ubc18\uc758 \uc0ac\uc804\ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \ud65c\uc6a9\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \uc790\uc138\ud788 \uc124\uba85\ud558\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n

1. \ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub780?<\/h2>\n

\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 \ub2e4\uc591\ud55c \uc790\uc5f0\uc5b4 \ucc98\ub9ac \ud0dc\uc2a4\ud06c\ub97c \uc704\ud55c \uc0ac\uc804\ud559\uc2b5\ub41c \ubaa8\ub378\ub4e4\uc744 \ud3ec\ud568\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub7ec\ud55c \ubaa8\ub378\ub4e4\uc740 \uc608\ub97c \ub4e4\uc5b4 \ubc88\uc5ed, \ud14d\uc2a4\ud2b8 \ubd84\ub958, \uc694\uc57d, \uc9c8\ubb38 \uc751\ub2f5 \ub4f1\uacfc \uac19\uc740 \ub2e4\uc591\ud55c NLP \uc791\uc5c5\uc744 \uc218\ud589\ud560 \uc218 \uc788\ub3c4\ub85d \ud6c8\ub828\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4. \ud2b8\ub80c\uc2a4\ud3ec\uba38 \uc544\ud0a4\ud14d\ucc98\ub294 \uc140\ud504 \uc5b4\ud150\uc158(self-attention) \uba54\ucee4\ub2c8\uc998\uc744 \ud1b5\ud574 \uc785\ub825 \uc2dc\ud000\uc2a4\uc758 \uac01 \ub2e8\uc5b4\uc758 \uc911\uc694\ub3c4\ub97c \uacc4\uc0b0\ud558\uc5ec \ubb38\ub9e5\uc744 \uc774\ud574\ud558\ub294 \ub370 \ud0c1\uc6d4\ud55c \uc131\ub2a5\uc744 \ubcf4\uc5ec\uc90d\ub2c8\ub2e4.<\/p>\n

1.1 \uc124\uce58\ud558\uae30<\/h3>\n

\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc124\uce58\ud558\uae30 \uc704\ud574\uc11c\ub294 \uba3c\uc800 \ud30c\uc774\uc36c \ud658\uacbd\uc744 \uc124\uc815\ud574\uc57c \ud569\ub2c8\ub2e4. \uc544\ub798\uc758 \uba85\ub839\uc5b4\ub97c \ud1b5\ud574 \uc124\uce58\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

pip install transformers torch<\/code><\/pre>\n
2. \uc0ac\uc804\ud559\uc2b5 \ubaa8\ub378 \uc774\ud574\ud558\uae30<\/h2>\n
\uc0ac\uc804\ud559\uc2b5\ub41c \ubaa8\ub378\uc740 \ub300\uaddc\ubaa8 \ub370\uc774\ud130\uc14b\uc73c\ub85c \ud6c8\ub828\ub418\uc5b4 \ud2b9\uc815 \ud0dc\uc2a4\ud06c\uc5d0 \ub300\ud55c \uc131\ub2a5\uc744 \ub192\uc774\uae30 \uc704\ud574 \ubbf8\uc138\uc870\uc815(fine-tuning)\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4 BERT, GPT-2, RoBERTa \ub4f1\uacfc \uac19\uc740 \ubaa8\ub378\ub4e4\uc774 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub7ec\ud55c \ubaa8\ub378\ub4e4\uc740 \uae30\ubcf8\uc801\uc778 \uc5b8\uc5b4 \uc774\ud574\ub97c \uc704\ud574 \uc124\uacc4\ub418\uc5c8\uc73c\uba70, \ub2e4\uc591\ud55c \ub370\uc774\ud130\uc14b\uc5d0 \ub300\ud574 \uc0ac\uc804\ud559\uc2b5\uc744 \uac70\ucce4\uc2b5\ub2c8\ub2e4.<\/p>\n
2.1 BERT \ubaa8\ub378<\/h3>\n
BERT(Bidirectional Encoder Representations from Transformers) \ubaa8\ub378\uc740 \uc591 \ubc29\ud5a5\uc131 \uc2dc\ud000\uc2a4 \uc778\ucf54\ub354\ub85c, \ubb38\ub9e5\uc744 \ubc18\uc601\ud558\uc5ec \ub2e8\uc5b4\uc758 \uc758\ubbf8\ub97c \ud30c\uc545\ud569\ub2c8\ub2e4. \uc218\ub9ce\uc740 \uc790\uc5f0\uc5b4 \ucc98\ub9ac \ud0dc\uc2a4\ud06c\uc5d0\uc11c \ub6f0\uc5b4\ub09c \uc131\ub2a5\uc744 \ubcf4\uc774\ub294 \uc774 \ubaa8\ub378\uc740 \ubb38\uc7a5 \ub0b4 \ub2e8\uc5b4\ub4e4\uc744 \ub458\ub7ec\uc2fc \ubb38\ub9e5\uc744 \ub3d9\uc2dc\uc5d0 \uc774\ud574\ud560 \uc218 \uc788\ub294 \ub2a5\ub825\uc744 \uac00\uc9c0\uace0 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
2.2 GPT-2 \ubaa8\ub378<\/h3>\n
GPT-2(Generative Pre-trained Transformer 2)\ub294 \uc8fc\ub85c \ud14d\uc2a4\ud2b8 \uc0dd\uc131 \uc791\uc5c5\uc5d0 \uc0ac\uc6a9\ub418\ub294 \ubaa8\ub378\ub85c, \uc21c\ucc28\uc801\uc778 \ub370\uc774\ud130 \ucc98\ub9ac\ub97c \ud1b5\ud574 \ubb38\ub9e5\uc744 \uc774\ud574\ud558\uace0 \uc0dd\uc131\ud558\ub294 \ub370 \uac15\uc810\uc744 \ubcf4\uc785\ub2c8\ub2e4. \uc774 \ubaa8\ub378\uc740 \ud2b9\uc815 \uc8fc\uc81c\ub098 \uc2a4\ud0c0\uc77c\uc758 \ubb38\uc11c\ub97c \uc0dd\uc131\ud558\ub294 \ub370 \ub110\ub9ac \ud65c\uc6a9\ub429\ub2c8\ub2e4.<\/p>\n
3. \ud30c\uc774\ud1a0\uce58(PyTorch)\uc640 \ud568\uaed8 \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95<\/h2>\n
\ud30c\uc774\ud1a0\uce58\ub294 \ub525\ub7ec\ub2dd \ubaa8\ub378\uc744 \uad6c\uc131\ud558\uace0 \ud559\uc2b5\ud558\ub294 \ub370 \ub9e4\uc6b0 \uc720\uc6a9\ud55c \ud504\ub808\uc784\uc6cc\ud06c\uc785\ub2c8\ub2e4. \ud5c8\uae45\ud398\uc774\uc2a4\uc758 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 \ud30c\uc774\ud1a0\uce58\uc640 \ud568\uaed8 \uc0ac\uc6a9\ud560 \uc218 \uc788\ub3c4\ub85d \uc124\uacc4\ub418\uc5b4 \uc788\uc5b4, \ubaa8\ub378\uc744 \uc27d\uac8c \ub85c\ub4dc\ud558\uace0 \ud65c\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
3.1 \ud14d\uc2a4\ud2b8 \ubd84\ub958 \uc608\uc81c<\/h3>\n
\uc774\ubc88 \uc139\uc158\uc5d0\uc11c\ub294 BERT \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\uc5ec \uac04\ub2e8\ud55c \ud14d\uc2a4\ud2b8 \ubd84\ub958 \uc791\uc5c5\uc744 \uc218\ud589\ud558\ub294 \uc608\uc81c\ub97c \ubcf4\uc5ec\ub4dc\ub9ac\uaca0\uc2b5\ub2c8\ub2e4. Text Classification\uc744 \uc704\ud574 IMDB \uc601\ud654 \ub9ac\ubdf0 \ub370\uc774\ud130\uc14b\uc744 \uc0ac\uc6a9\ud560 \uc608\uc815\uc785\ub2c8\ub2e4.<\/p>\n
3.1.1 \ub370\uc774\ud130\uc14b \uc900\ube44<\/h4>\n
\uba3c\uc800 IMDB \ub370\uc774\ud130\uc14b\uc744 \ub2e4\uc6b4\ub85c\ub4dc\ud558\uace0 \uc0ac\uc804\ucc98\ub9ac\ud569\ub2c8\ub2e4. \uc774\ub97c \uc704\ud574 pandas \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc0ac\uc6a9\ud560 \uac83\uc785\ub2c8\ub2e4.<\/p>\n
import pandas as pd\n\n# IMDB \uc601\ud654 \ub9ac\ubdf0 \ub370\uc774\ud130 \ub2e4\uc6b4\ub85c\ub4dc\nurl = \"https:\/\/ai.stanford.edu\/~amaas\/data\/sentiment\/aclImdb_v1.tar.gz\"\nimdb_data = pd.read_csv(url)\n\n# \ub370\uc774\ud130 \uc804\ucc98\ub9ac\ndf = pd.DataFrame(imdb_data)\ndf['label'] = df['label'].map({'pos': 1, 'neg': 0})\ndf.head()<\/code><\/pre>\n
3.1.2 \ub370\uc774\ud130\uc14b \ub85c\ub4dc\uc640 \ubd84\ub9ac<\/h4>\n
\ub370\uc774\ud130\uc14b\uc744 \ub85c\ub4dc\ud55c \ud6c4 \ud559\uc2b5 \ub370\uc774\ud130\uc640 \uac80\uc99d \ub370\uc774\ud130\ub97c \ubd84\ub9ac\ud569\ub2c8\ub2e4.<\/p>\n
from sklearn.model_selection import train_test_split\n\n# train\/test split\ntrain_data, test_data = train_test_split(df, test_size=0.2, random_state=42)\n\n# \ubd84\ub9ac\ub41c \ub370\uc774\ud130 \ud655\uc778\nprint(f'\ud6c8\ub828 \ub370\uc774\ud130 \ud06c\uae30: {len(train_data)}, \ud14c\uc2a4\ud2b8 \ub370\uc774\ud130 \ud06c\uae30: {len(test_data)}')<\/code><\/pre>\n
3.1.3 \ud5c8\uae45\ud398\uc774\uc2a4 \ub370\uc774\ud130\uc14b \uc124\uc815<\/h4>\n
\uc774\uc81c Hugging Face\uc758 Dataset<\/code> \ud074\ub798\uc2a4\ub97c \uc0ac\uc6a9\ud558\uc5ec \ub370\uc774\ud130\ub97c \ubd88\ub7ec\uc624\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
from transformers import Dataset\n\n# Hugging Face Dataset \ud615\uc2dd\uc73c\ub85c \ubcc0\ud658\ntrain_dataset = Dataset.from_pandas(train_data)\ntest_dataset = Dataset.from_pandas(test_data)<\/code><\/pre>\n
3.1.4 \ubaa8\ub378 \ub85c\ub4dc\ud558\uae30<\/h4>\n
\uc0ac\uc804\ud559\uc2b5\ub41c BERT \ubaa8\ub378\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4. \uc774\ub7ec\ud55c \ubaa8\ub378\ub4e4\uc740 AutoModelForSequenceClassification<\/code> \ud074\ub798\uc2a4\ub97c \ud1b5\ud574 \ud638\ucd9c\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
from transformers import AutoTokenizer, AutoModelForSequenceClassification\n\n# \uc0ac\uc804\ud559\uc2b5\ub41c BERT \ubaa8\ub378\uacfc \ud1a0\ud06c\ub098\uc774\uc800 \ub85c\ub4dc\nmodel_name = \"bert-base-uncased\"\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)<\/code><\/pre>\n
3.1.5 \ub370\uc774\ud130 \uc804\ucc98\ub9ac<\/h4>\n
\ud14d\uc2a4\ud2b8 \ub370\uc774\ud130\ub97c BERT \ubaa8\ub378\uc774 \ucc98\ub9ac\ud560 \uc218 \uc788\ub294 \uc785\ub825 \ud615\uc2dd\uc73c\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.<\/p>\n
def preprocess_function(examples):\n    return tokenizer(examples['text'], truncation=True, padding='max_length', max_length=512)\n\ntokenized_train_data = train_dataset.map(preprocess_function, batched=True)\ntokenized_test_data = test_dataset.map(preprocess_function, batched=True)<\/code><\/pre>\n
3.1.6 \ubaa8\ub378 \ud559\uc2b5\ud558\uae30<\/h4>\n
\ubaa8\ub378\uc744 \ud559\uc2b5\uc2dc\ud0a4\uae30 \uc704\ud574 Trainer<\/code> \ud074\ub798\uc2a4\ub97c \uc0ac\uc6a9\ud569\ub2c8\ub2e4.<\/p>\n
from transformers import Trainer, TrainingArguments\n\n# \ud6c8\ub828 \uc778\uc790 \uc124\uc815\ntraining_args = TrainingArguments(\n    output_dir='.\/results',\n    evaluation_strategy='epoch',\n    learning_rate=2e-5,\n    per_device_train_batch_size=16,\n    per_device_eval_batch_size=16,\n    num_train_epochs=3,\n    weight_decay=0.01,\n)\n\ntrainer = Trainer(\n    model=model,\n    args=training_args,\n    train_dataset=tokenized_train_data,\n    eval_dataset=tokenized_test_data\n)\n\n# \ubaa8\ub378 \ud6c8\ub828\ntrainer.train()<\/code><\/pre>\n
3.1.7 \ubaa8\ub378 \ud3c9\uac00\ud558\uae30<\/h4>\n
\ud6c8\ub828\ub41c \ubaa8\ub378\uc758 \uc131\ub2a5\uc744 \ud3c9\uac00\ud569\ub2c8\ub2e4.<\/p>\n
<\/code#><\/pre>\n
4. \uacb0\ub860<\/h2>\n
\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub80c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \ud65c\uc6a9\ud558\uba74, \uc0ac\uc804\ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \uc190\uc27d\uac8c \ub2e4\ub8f0 \uc218 \uc788\uc2b5\ub2c8\ub2e4. BERT\uc640 \uac19\uc740 \ubaa8\ub378\uc740 \ud14d\uc2a4\ud2b8 \ubd84\ub958 \uc678\uc5d0\ub3c4 \ub2e4\uc591\ud55c \uc790\uc5f0\uc5b4 \ucc98\ub9ac \uc791\uc5c5\uc5d0 \ud65c\uc6a9\ub420 \uc218 \uc788\uc73c\uba70, \uc5f0\uad6c \ubc0f \uc0b0\uc5c5\uc5d0\uc11c \uc911\uc694\ud55c \uc5ed\ud560\uc744 \ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. \ubcf8 \ube14\ub85c\uadf8\uc5d0\uc11c\ub294 \ubaa8\ub378\uc758 \uc124\uce58\ubd80\ud130 \ud559\uc2b5, \ud3c9\uac00\uae4c\uc9c0\uc758 \uacfc\uc815\uc744 \ud568\uaed8 \uc0b4\ud3b4\ubcf4\uc558\uc2b5\ub2c8\ub2e4. \uc2e4\ubb34\uc5d0 \uc801\uc6a9\ud560 \uc218 \uc788\ub294 \uc720\uc6a9\ud55c \uc9c0\uc2dd\uc744 \uc2b5\ub4dd\ud558\uc168\uae30\ub97c \ubc14\ub78d\ub2c8\ub2e4.<\/p>\n
5. \ucc38\uace0\uc790\ub8cc<\/h2>\n