\ub525\ub7ec\ub2dd\uacfc \uc790\uc5f0\uc5b4 \ucc98\ub9ac(NLP) \ubd84\uc57c\uc5d0\uc11c \ud5c8\uae45\ud398\uc774\uc2a4(Hugging Face)<\/strong>\uc758 \ud2b8\ub79c\uc2a4\ud3ec\uba38(Transformers) \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 \ub9e4\uc6b0 \uc720\uc6a9\ud55c \ub3c4\uad6c\uc785\ub2c8\ub2e4. \uc774 \uac15\uc88c\uc5d0\uc11c\ub294 \ud5c8\uae45\ud398\uc774\uc2a4\uc758 \uc774\uc81c \uc704\uc758 \ucf54\ub4dc\ub294 \ud6c8\ub828 \ud30c\ub77c\ubbf8\ud130\ub97c \uc124\uc815\ud55c \ud6c4, \uc704\uc758 \ucf54\ub4dc\uc5d0\uc11c\ub294 \ub2e4\uc74c \ub2e8\uacc4\ub97c \uc218\ud589\ud569\ub2c8\ub2e4:<\/p>\n TrainingArguments\uc758 \uc778\uc790\ub294 \ub2e4\uc591\ud558\uc9c0\ub9cc, \uc790\uc8fc \uc0ac\uc6a9\ud558\ub294 \uba87 \uac00\uc9c0 \uc124\uc815\uc744 \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:<\/p>\n \ubaa8\ub378 \ud6c8\ub828 \uc911 \uba54\ubaa8\ub9ac \uc81c\ud55c \ub54c\ubb38\uc5d0 \ud070 \ubc30\uce58\ub97c \ud6c8\ub828\ud558\uae30 \uc5b4\ub824\uc6b8 \uacbd\uc6b0, gradient accumulation\uc744 \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, \ubc30\uce58 \ud06c\uae30\ub97c 32\ub85c \uc124\uc815\ud558\uace0 4\ubc88\uc758 \ubc30\uce58\uc5d0 \ub300\ud574 \uae30\uc6b8\uae30\ub97c \ucd95\uc801\ud558\uc5ec \ud559\uc2b5\ud558\uba74 \ucd1d \ubc30\uce58 \ud06c\uae30\ub294 128\uc774 \ub429\ub2c8\ub2e4.<\/p>\n GPU\uac00 Mixed Precision Training\uc744 \uc9c0\uc6d0\ud558\ub294 \uacbd\uc6b0, \ud6c8\ub828\uc744 \uac00\uc18d\ud654\ud558\uace0 \uba54\ubaa8\ub9ac \uc0ac\uc6a9\uc744 \uc904\uc77c \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \uacbd\uc6b0 \ud6c8\ub828 \ub3c4\uc911 \uc131\ub2a5 \ud5a5\uc0c1\uc774 \uc5c6\uc744 \uacbd\uc6b0 \uc870\uae30 \uc885\ub8cc\ub97c \uc124\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774\ub97c \ud1b5\ud574 \ubd88\ud544\uc694\ud55c \ud6c8\ub828\uc744 \ubc29\uc9c0\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc774 \uac15\uc88c\uc5d0\uc11c\ub294 \ud5c8\uae45\ud398\uc774\uc2a4\uc758 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\uc5d0\uc11c \ub525\ub7ec\ub2dd \ubaa8\ub378\uc744 \ub354\uc6b1 \ud6a8\uacfc\uc801\uc73c\ub85c \ud6c8\ub828\ud558\uae30 \uc704\ud574\uc11c\ub294 \ucd94\uac00\uc801\uc73c\ub85c \uad81\uae08\ud55c \uc810\uc774\ub098 \ub354 \uc54c\uace0 \uc2f6\uc740 \uc810\uc774 \uc788\ub2e4\uba74 \ub313\uae00\ub85c \ub0a8\uaca8\uc8fc\uc2dc\uba74 \uc131\uc2ec\uaecf \ub2f5\ubcc0\ud574\ub4dc\ub9ac\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\nTrainer<\/code> API\uc5d0\uc11c \uc0ac\uc6a9\ud558\ub294 TrainingArguments<\/code> \ud074\ub798\uc2a4\uc5d0 \ub300\ud574 \uc790\uc138\ud788 \uc124\uba85\ud558\uace0, \uadf8 \uc124\uc815 \ubc29\ubc95\uacfc \uc2e4\uc81c \ucf54\ub4dc \uc608\uc81c\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\nTrainingArguments\ub780?<\/h2>\n
TrainingArguments<\/code> \ud074\ub798\uc2a4\ub294 \ubaa8\ub378 \ud6c8\ub828\uc758 \ub2e4\uc591\ud55c \ud558\uc774\ud37c\ud30c\ub77c\ubbf8\ud130\uc640 \uc124\uc815\uc744 \uc815\uc758\ud558\uae30 \uc704\ud574 \uc0ac\uc6a9\ub429\ub2c8\ub2e4. \uc774 \ud074\ub798\uc2a4\ub294 \ubaa8\ub378 \ud6c8\ub828, \uac80\uc99d \ubc0f \ub85c\uae45 \uc694\uad6c \uc0ac\ud56d\uc744 \ud3ec\ud568\ud558\ub294 \uc5ec\ub7ec \uc778\uc790\ub97c \uc124\uc815\ud560 \uc218 \uc788\ub3c4\ub85d \ud574\uc90d\ub2c8\ub2e4.<\/p>\nTrainingArguments\uc758 \uc8fc\uc694 \ub9e4\uac1c\ubcc0\uc218<\/h3>\n
\n
output_dir<\/code>: \ubaa8\ub378 \uccb4\ud06c\ud3ec\uc778\ud2b8\uac00 \uc800\uc7a5\ub420 \ub514\ub809\ud1a0\ub9ac \uacbd\ub85c\uc785\ub2c8\ub2e4.<\/li>\nnum_train_epochs<\/code>: \uc804\uccb4 \ud6c8\ub828 \ub370\uc774\ud130\uc14b\uc744 \uba87 \ubc88 \ubc18\ubcf5\ud560 \uac83\uc778\uc9c0 \uc124\uc815\ud569\ub2c8\ub2e4.<\/li>\nper_device_train_batch_size<\/code>: \uac01 \uc7a5\uce58(\uc608: GPU)\uc5d0\uc11c \uc0ac\uc6a9\ud560 \ubc30\uce58 \uc0ac\uc774\uc988\uc785\ub2c8\ub2e4.<\/li>\nlearning_rate<\/code>: \ud559\uc2b5\ub960\uc744 \uc124\uc815\ud569\ub2c8\ub2e4.<\/li>\nevaluation_strategy<\/code>: \ud3c9\uac00 \uc804\ub7b5\uc744 \uc124\uc815\ud569\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, “epoch” \ub610\ub294 “steps”\uc640 \uac19\uc740 \uc635\uc158\uc774 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\nlogging_dir<\/code>: \ub85c\uadf8 \ud30c\uc77c\uc774 \uc800\uc7a5\ub420 \ub514\ub809\ud1a0\ub9ac \uacbd\ub85c\uc785\ub2c8\ub2e4.<\/li>\nweight_decay<\/code>: \uac00\uc911\uce58 \uac10\uc18c\ub97c \uc0ac\uc6a9\ud558\uc5ec \uc815\uaddc\ud654\ub97c \uc801\uc6a9\ud569\ub2c8\ub2e4.<\/li>\nsave_total_limit<\/code>: \uc800\uc7a5\ud560 \uccb4\ud06c\ud3ec\uc778\ud2b8\uc758 \ucd5c\ub300 \uac1c\uc218\ub97c \uc81c\ud55c\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\nTrainingArguments \uc124\uc815\ud558\uae30<\/h2>\n
TrainingArguments<\/code>\ub97c \ud65c\uc6a9\ud574 \uc2e4\uc81c\ub85c \ud6c8\ub828\uc5d0 \ud544\uc694\ud55c \ub9e4\uac1c\ubcc0\uc218\ub97c \uc124\uc815\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \uc544\ub798 \uc608\uc81c \ucf54\ub4dc\uc5d0\uc11c\ub294 \uc774 \ud074\ub798\uc2a4\ub97c \uc5b4\ub5bb\uac8c \uc0ac\uc6a9\ud558\uace0 \uac01\uac01\uc758 \ub9e4\uac1c\ubcc0\uc218\uc758 \uc5ed\ud560\uc774 \ubb34\uc5c7\uc778\uc9c0 \uc124\uba85\ud569\ub2c8\ub2e4.<\/p>\n\ud30c\uc774\uc36c \uc608\uc81c \ucf54\ub4dc<\/h3>\n
from transformers import TrainingArguments\n\n# TrainingArguments \uac1d\uccb4 \uc0dd\uc131\ntraining_args = TrainingArguments(\n output_dir='.\/results', # \uccb4\ud06c\ud3ec\uc778\ud2b8\ub97c \uc800\uc7a5\ud560 \ub514\ub809\ud1a0\ub9ac \uacbd\ub85c\n num_train_epochs=3, # \ud6c8\ub828\ud560 \uc5d0\ud3ed \uc218\n per_device_train_batch_size=16, # \uac01 \ub514\ubc14\uc774\uc2a4\uc5d0\uc11c \uc0ac\uc6a9\ud560 \ubc30\uce58 \uc0ac\uc774\uc988\n per_device_eval_batch_size=64, # \ud3c9\uac00 \uc2dc \uc0ac\uc6a9\ud560 \ubc30\uce58 \uc0ac\uc774\uc988\n learning_rate=2e-5, # \ud559\uc2b5\ub960\n evaluation_strategy=\"epoch\", # \ud3c9\uac00 \uc804\ub7b5\n logging_dir='.\/logs', # \ub85c\uadf8 \ud30c\uc77c \uc800\uc7a5 \uacbd\ub85c\n weight_decay=0.01, # \uac00\uc911\uce58 \uac10\uc18c\n save_total_limit=2 # \ucd5c\ub300 \uc800\uc7a5 \uccb4\ud06c\ud3ec\uc778\ud2b8 \uc218\n)\n\nprint(training_args)\n<\/code><\/pre>\n\ucf54\ub4dc \uc124\uba85<\/h3>\n
TrainingArguments<\/code> \uac1d\uccb4\ub97c \uc0dd\uc131\ud558\ub294 \uc608\uc81c\uc785\ub2c8\ub2e4. \uac01 \ub9e4\uac1c\ubcc0\uc218\uc5d0 \ub300\ud574 \uad6c\uccb4\uc801\uc73c\ub85c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:<\/p>\n\n
output_dir='.\/results'<\/code>: \ud6c8\ub828 \ud6c4 \ubaa8\ub378 \uccb4\ud06c\ud3ec\uc778\ud2b8\uac00 \uc800\uc7a5\ub420 \ud3f4\ub354\ub97c \uc9c0\uc815\ud569\ub2c8\ub2e4.<\/li>\nnum_train_epochs=3<\/code>: \uc804\uccb4 \ub370\uc774\ud130\uc14b\uc744 3\ubc88 \ubc18\ubcf5\ud558\uc5ec \ud6c8\ub828\ud569\ub2c8\ub2e4.<\/li>\nper_device_train_batch_size=16<\/code>: \uac01 \uc7a5\uce58\uc5d0\uc11c 16\uac1c\uc758 \uc0d8\ud50c\uc744 \ubc30\uce58\ub85c \uc0ac\uc6a9\ud558\uc5ec \ud6c8\ub828\ud569\ub2c8\ub2e4.<\/li>\nper_device_eval_batch_size=64<\/code>: \ud3c9\uac00 \uc2dc\uc5d0\ub294 \uac01 \uc7a5\uce58\uc5d0\uc11c 64\uac1c\uc758 \uc0d8\ud50c\uc744 \ubc30\uce58\ub85c \ucc98\ub9ac\ud569\ub2c8\ub2e4.<\/li>\nlearning_rate=2e-5<\/code>: \ud6c8\ub828 \uc2dc\uc791 \uc2dc\uc758 \ud559\uc2b5\ub960\uc744 \uc124\uc815\ud569\ub2c8\ub2e4.<\/li>\nevaluation_strategy=\"epoch\"<\/code>: \uac01 \uc5d0\ud3ed \uc885\ub8cc \ud6c4 \ubaa8\ub378\uc744 \ud3c9\uac00\ud558\ub3c4\ub85d \uc124\uc815\ud569\ub2c8\ub2e4.<\/li>\nlogging_dir='.\/logs'<\/code>: \ud6c8\ub828 \ub85c\uadf8\ub97c \uc800\uc7a5\ud560 \ub514\ub809\ud1a0\ub9ac\uc785\ub2c8\ub2e4.<\/li>\nweight_decay=0.01<\/code>: 1%\uc758 \uac00\uc911\uce58 \uac10\uc18c\ub97c \uc801\uc6a9\ud558\uc5ec \ubaa8\ub378 \uacfc\uc801\ud569\uc744 \ubc29\uc9c0\ud569\ub2c8\ub2e4.<\/li>\nsave_total_limit=2<\/code>: \uc800\uc7a5\ub418\ub294 \uccb4\ud06c\ud3ec\uc778\ud2b8\uc758 \ucd5c\ub300 \uc218\ub97c 2\ub85c \uc81c\ud55c\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\nTrainingArguments\uc640 Trainer API \ud1b5\ud569\ud558\uae30<\/h2>\n
Trainer<\/code> API\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc744 \ud6c8\ub828\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \uc544\ub798\ub294 Trainer<\/code> \ud074\ub798\uc2a4\uc640 TrainingArguments<\/code>\ub97c \ud1b5\ud569\ud558\ub294 \ubc29\ubc95\uc744 \ubcf4\uc5ec\uc8fc\ub294 \uc608\uc2dc\uc785\ub2c8\ub2e4.<\/p>\nfrom transformers import Trainer, TrainingArguments, AutoModelForSequenceClassification, AutoTokenizer\n\n# \ubaa8\ub378\uacfc \ud1a0\ud06c\ub098\uc774\uc800 \ubd88\ub7ec\uc624\uae30\nmodel = AutoModelForSequenceClassification.from_pretrained(\"bert-base-uncased\", num_labels=2)\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n\n# \ud6c8\ub828 \ubc0f \ud3c9\uac00 \ub370\uc774\ud130\uc14b \uc900\ube44 (\uc608\uc2dc\ub294 \uc0dd\ub7b5)\ntrain_dataset = ...\neval_dataset = ...\n\n# Trainer \uac1d\uccb4 \uc0dd\uc131\ntrainer = Trainer(\n model=model,\n args=training_args,\n train_dataset=train_dataset,\n eval_dataset=eval_dataset\n)\n\n# \ubaa8\ub378 \ud6c8\ub828\ntrainer.train()\n<\/code><\/pre>\n\ucf54\ub4dc \uc124\uba85<\/h3>\n
\n
AutoModelForSequenceClassification<\/code>\uc744 \uc0ac\uc6a9\ud558\uc5ec \ubd84\ub958 \uc791\uc5c5\uc744 \uc704\ud55c BERT \ubaa8\ub378\uc744 \ubd88\ub7ec\uc635\ub2c8\ub2e4.<\/li>\nAutoTokenizer<\/code>\ub97c \uc0ac\uc6a9\ud558\uc5ec BERT\uc5d0 \ub9de\ub294 \ud1a0\ud06c\ub098\uc774\uc800\ub3c4 \ubd88\ub7ec\uc635\ub2c8\ub2e4.<\/li>\nTrainer<\/code> \uac1d\uccb4\ub97c \uc0dd\uc131\ud569\ub2c8\ub2e4. \uc774 \uac1d\uccb4\uc5d0\ub294 \ubaa8\ub378, \ud6c8\ub828 \uc778\uc790, \ud6c8\ub828 \ub370\uc774\ud130\uc14b \ubc0f \uac80\uc99d \ub370\uc774\ud130\uc14b\uc744 \uc804\ub2ec\ud569\ub2c8\ub2e4.<\/li>\ntrainer.train()<\/code>\uc744 \ud638\ucd9c\ud558\uc5ec \ubaa8\ub378 \ud6c8\ub828\uc744 \uc2dc\uc791\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\nTrainingArguments\uc5d0\uc11c \uc790\uc8fc \uc0ac\uc6a9\ud558\ub294 \uc124\uc815<\/h2>\n
1. Gradient Accumulation<\/h3>\n
training_args = TrainingArguments(\n per_device_train_batch_size=8,\n gradient_accumulation_steps=4, # 4\uac1c\uc758 \ubc30\uce58\uc5d0 \ub300\ud574 \uae30\uc6b8\uae30 \ucd95\uc801\n)\n<\/code><\/pre>\n2. Mixed Precision Training<\/h3>\n
fp16=True<\/code> \uc124\uc815\uc744 \ucd94\uac00\ud558\uc5ec \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\ntraining_args = TrainingArguments(\n fp16=True, # Mixed precision training\n)\n<\/code><\/pre>\n3. Early Stopping<\/h3>\n
EarlyStoppingCallback<\/code>\uc640 \uacb0\ud569\ud574\uc57c \ud569\ub2c8\ub2e4.<\/p>\nfrom transformers import EarlyStoppingCallback\n\ntrainer = Trainer(\n ...\n callbacks=[EarlyStoppingCallback(early_stopping_patience=3)], # 3 \uc5d0\ud3ed \ub3d9\uc548 \ud5a5\uc0c1\uc774 \uc5c6\uc73c\uba74 \uc885\ub8cc\n)\n<\/code><\/pre>\n\uacb0\ub860<\/h2>\n
TrainingArguments<\/code> \ud074\ub798\uc2a4\ub97c \uc124\uc815\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \uc790\uc138\ud788 \uc124\uba85\ud588\uc2b5\ub2c8\ub2e4. \ub2e4\uc591\ud55c \ud558\uc774\ud37c\ud30c\ub77c\ubbf8\ud130\ub97c \ud1b5\ud574 \ubaa8\ub378 \ud6c8\ub828\uc744 \ucd5c\uc801\ud654\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\nTrainingArguments<\/code>\uc758 \ub2e4\uc591\ud55c \ub9e4\uac1c\ubcc0\uc218\ub97c \uc798 \ud65c\uc6a9\ud558\ub294 \uac83\uc774 \uc911\uc694\ud569\ub2c8\ub2e4. \uc2e4\ud5d8\uc744 \ud1b5\ud574 \ucd5c\uc801\uc758 \ud558\uc774\ud37c\ud30c\ub77c\ubbf8\ud130\ub97c \ucc3e\uace0, \ubaa8\ub378\uc758 \uc131\ub2a5\uc744 \uc9c0\uc18d\uc801\uc73c\ub85c \uac1c\uc120\ud574 \ub098\uac00\uae38 \ubc14\ub78d\ub2c8\ub2e4.<\/p>\n