\ub525\ub7ec\ub2dd \ubd84\uc57c\uc5d0\uc11c \uc790\uc5f0\uc5b4 \ucc98\ub9ac(NLP)\ub294 \ub9e4\uc6b0 \uc911\uc694\ud55c \uc5ed\ud560\uc744 \ud569\ub2c8\ub2e4. \ud2b9\ud788 BERT(Bidirectional Encoder Representations from Transformers) \ubaa8\ub378\uc740 NLP \ubd84\uc57c\uc5d0\uc11c \ub110\ub9ac \uc0ac\uc6a9\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4.
\n \uc774\ubc88 \uac15\uc88c\uc5d0\uc11c\ub294 \ud5c8\uae45\ud398\uc774\uc2a4(Hugging Face)\uc758 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc0ac\uc6a9\ud558\uc5ec Mobile BERT \ubaa8\ub378\uc744 \uc124\uce58\ud558\uace0, \uc0ac\uc804 \ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \ubd88\ub7ec\uc624\ub294 \ubc29\ubc95\uc744 \uc124\uba85\ud558\uaca0\uc2b5\ub2c8\ub2e4.
\n Mobile BERT\ub294 \uacbd\ub7c9\ud654\ub41c BERT \ubaa8\ub378\ub85c, \ubaa8\ubc14\uc77c \uae30\uae30\uc5d0\uc11c\ub3c4 \ud6a8\uc728\uc801\uc73c\ub85c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub2e4\ub294 \uc7a5\uc810\uc774 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub294 \ub2e4\uc591\ud55c \ucd5c\uc2e0 NLP \ubaa8\ub378\uc744 \uc190\uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub3c4\ub85d \ub3c4\uc640\uc8fc\ub294 \ud30c\uc774\uc36c \ub77c\uc774\ube0c\ub7ec\ub9ac\uc785\ub2c8\ub2e4.
\n \uc774 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \ud1b5\ud574 BERT, GPT-2, T5 \ub4f1 \ub2e4\uc591\ud55c \uc0ac\uc804 \ud559\uc2b5 \ubaa8\ub378\uc744 \ubd88\ub7ec\uc624\uace0, \uc5ec\ub7ec NLP \ud0dc\uc2a4\ud06c\uc5d0 \uc801\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ub610\ud55c, \uc0ac\uc6a9\uc790 \ub9de\ucda4\ud615 \ud6c8\ub828\uc744 \uc704\ud55c API\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n
Mobile BERT\ub294 Google\uc774 \uac1c\ubc1c\ud55c \uacbd\ub7c9\ud654\ub41c BERT \ubaa8\ub378\uc785\ub2c8\ub2e4. \uc804\ud1b5\uc801\uc778 BERT \ubaa8\ub378\uc740 \ub300\uaddc\ubaa8 \ub370\uc774\ud130\uc14b\uc5d0\uc11c \uc0ac\uc804 \ud559\uc2b5\ub418\uc5b4 \uac15\ub825\ud55c \uc131\ub2a5\uc744 \ubcf4\uc774\uc9c0\ub9cc,
\n \uadf8\ub9cc\ud07c \ubaa8\ub378\uc758 \ud06c\uae30\uac00 \ucee4\uc11c \ubaa8\ubc14\uc77c \uae30\uae30\ub098 \uc784\ubca0\ub514\ub4dc \uc2dc\uc2a4\ud15c\uc5d0\uc11c \uc0ac\uc6a9\ud558\ub294 \ub370 \uc81c\uc57d\uc774 \uc788\uc2b5\ub2c8\ub2e4.
\n \uc774\uc5d0 \ubc18\ud574 Mobile BERT\ub294 \ud06c\uae30\ub97c \uc904\uc774\uace0\ub3c4 BERT\uc758 \uc131\ub2a5\uc744 \ucd5c\ub300\ud55c \uc720\uc9c0\ud558\ub3c4\ub85d \uc124\uacc4\ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uc774\ub7ec\ud55c \ud2b9\uc131 \ub355\ubd84\uc5d0 Mobile BERT\ub294 \uc5ec\ub7ec NLP \ud0dc\uc2a4\ud06c\uc5d0 \ud65c\uc6a9\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
Mobile BERT\ub97c \uc0ac\uc6a9\ud558\uae30 \uc704\ud574\uc11c\ub294 \uba3c\uc800 \ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\uc640 \uae30\ud0c0 \ud544\uc218 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4.
\n \ub2e4\uc74c \uba85\ub839\uc5b4\ub97c \ud1b5\ud574 pip\ub97c \uc774\uc6a9\ud574 \ud544\uc694\ud55c \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc124\uce58\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4:<\/p>\n
pip install transformers torch<\/code><\/pre>\n\uc704 \uba85\ub839\uc5b4\ub85c \uc124\uce58\uac00 \uc644\ub8cc\ub418\uba74, Python \ud658\uacbd\uc5d0\uc11c Mobile BERT\ub97c \uc0ac\uc6a9\ud560 \uc900\ube44\uac00 \uc644\ub8cc\ub429\ub2c8\ub2e4.<\/p>\n
\n\ucc38\uace0:<\/strong> PyTorch\ub97c \uc124\uce58\ud558\uc9c0 \uc54a\uc558\ub2e4\uba74 \uc124\uce58\ud574\uc57c \ud569\ub2c8\ub2e4. CUDA\ub97c \uc9c0\uc6d0\ud558\ub294 GPU\ub97c \uc0ac\uc6a9\ud558\ub294 \uacbd\uc6b0,
\n \uacf5\uc2dd \uc6f9\uc0ac\uc774\ud2b8\uc5d0\uc11c CUDA\uc5d0 \ub9de\ub294 PyTorch \ubc84\uc804\uc744 \uc120\ud0dd\ud558\uc5ec \uc124\uce58\ud558\uc138\uc694.\n <\/div>\n4. \uc0ac\uc804 \ud559\uc2b5 \ubaa8\ub378 \ubd88\ub7ec\uc624\uae30<\/h2>\n
\uc774\uc81c Mobile BERT \ubaa8\ub378\uc744 \ubd88\ub7ec\uc640\ubcf4\uaca0\uc2b5\ub2c8\ub2e4. \ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\uc5d0\uc11c\ub294 \ubbf8\ub9ac \ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \uc190\uc27d\uac8c \uc0ac\uc6a9\ud560 \uc218 \uc788\ub3c4\ub85d
\n \ud558\uae30 \uc704\ud574 \uba87 \uac00\uc9c0 \ud074\ub798\uc2a4\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n
4.1 \ucf54\ub4dc \uc608\uc81c<\/h3>\n
\ub2e4\uc74c \ucf54\ub4dc\ub97c \ud1b5\ud574 Mobile BERT\ub97c \ubd88\ub7ec\uc635\ub2c8\ub2e4:<\/p>\n
from transformers import MobileBertTokenizer, MobileBertForSequenceClassification\nimport torch\n\n# Mobile BERT \ubaa8\ub378\uacfc \ud1a0\ud06c\ub098\uc774\uc800 \ubd88\ub7ec\uc624\uae30\nmodel_name = \"google\/mobilebert-uncased\"\ntokenizer = MobileBertTokenizer.from_pretrained(model_name)\nmodel = MobileBertForSequenceClassification.from_pretrained(model_name)\n\n# \ud14c\uc2a4\ud2b8\ud560 \ubb38\uc7a5\ninput_text = \"\ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub9e4\uc6b0 \uc720\uc6a9\ud574\uc694!\"\n\n# \uc785\ub825 \ubb38\uc7a5\uc744 \ud1a0\ud06c\ub098\uc774\uc988\ud558\uace0 \ud150\uc11c\ub85c \ubcc0\ud658\ninputs = tokenizer(input_text, return_tensors=\"pt\")\n\n# \ubaa8\ub378\uc5d0 \uc785\ub825\ud558\uace0 \uc608\uce21\ud558\uae30\nwith torch.no_grad():\n logits = model(**inputs).logits\n\npredicted_class = torch.argmax(logits, dim=-1).item()\nprint(f\"\uc608\uce21\ub41c \ud074\ub798\uc2a4: {predicted_class}\")<\/code><\/pre>\n4.2 \ucf54\ub4dc \uc124\uba85<\/h3>\n
\ucf54\ub4dc\uc758 \uac01 \uc694\uc18c\ub97c \uc0b4\ud3b4\ubcf4\uba74 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4:<\/p>\n
\nfrom transformers import MobileBertTokenizer, MobileBertForSequenceClassification<\/code>:
\n Mobile BERT \ubaa8\ub378\uacfc \ud1a0\ud06c\ub098\uc774\uc800\ub97c \ubd88\ub7ec\uc635\ub2c8\ub2e4.<\/li>\nmodel_name = \"google\/mobilebert-uncased\"<\/code>: \uc0ac\uc6a9\ud560 \uc0ac\uc804 \ud559\uc2b5 \ubaa8\ub378\uc758 \uc774\ub984\uc744 \uc124\uc815\ud569\ub2c8\ub2e4.<\/li>\ntokenizer = MobileBertTokenizer.from_pretrained(model_name)<\/code>: \ubaa8\ub378\uc5d0 \ub300\ud55c \ud1a0\ud06c\ub098\uc774\uc800\ub97c \ucd08\uae30\ud654\ud569\ub2c8\ub2e4.<\/li>\nmodel = MobileBertForSequenceClassification.from_pretrained(model_name)<\/code>: \ubaa8\ub378\uc744 \ucd08\uae30\ud654\ud569\ub2c8\ub2e4.
\n \uc774 \ub54c \ud574\ub2f9 \ubaa8\ub378\uc740 \ubb38\uc7a5 \ubd84\ub958 \uc791\uc5c5\uc5d0 \uc801\ud569\ud569\ub2c8\ub2e4.<\/li>\ninputs = tokenizer(input_text, return_tensors=\"pt\")<\/code>: \uc785\ub825\ub41c \ubb38\uc7a5\uc744 \ud1a0\ud06c\ub098\uc774\uc988\ud558\uace0 PyTorch \ud150\uc11c\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.<\/li>\nwith torch.no_grad():<\/code>: \ud150\uc11c\uc758 \ubcc0\ud654\ub3c4 \ucd94\uc801\ud558\uc9c0 \uc54a\ub3c4\ub85d \uc124\uc815\ud558\uc5ec \uba54\ubaa8\ub9ac \ud6a8\uc728\uc131\uc744 \ub192\uc785\ub2c8\ub2e4.<\/li>\nlogits = model(**inputs).logits<\/code>: \ubaa8\ub378\uc744 \ud1b5\ud574 \uc608\uce21\ud55c \uacb0\uacfc\ub97c \uac00\uc838\uc635\ub2c8\ub2e4.<\/li>\npredicted_class = torch.argmax(logits, dim=-1).item()<\/code>: \uc608\uce21\ub41c \ud074\ub798\uc2a4 \uc911 \uac00\uc7a5 \ub192\uc740 \ud655\ub960\uc744 \uac00\uc9c4 \ud074\ub798\uc2a4\ub97c \uc120\ud0dd\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n5. Mobile BERT\ub97c \ud65c\uc6a9\ud55c \uc2e4\uc81c \uc608\uc81c<\/h2>\n
Mobile BERT \ubaa8\ub378\uc744 \ud65c\uc6a9\ud558\uc5ec \ubb38\uc7a5 \ubd84\ub958 \uc791\uc5c5\uc744 \uc218\ud589\ud558\ub294 \uc608\uc81c\ub97c \uc0b4\ud3b4\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.
\n \uc811\uadfc \ubc29\uc2dd\uc740 \uc8fc\uc5b4\uc9c4 \ubb38\uc7a5\uc744 \uae0d\uc815\uc801\uc774\uac70\ub098 \ubd80\uc815\uc801\uc778\uc9c0 \ubd84\ub958\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.<\/p>\n
5.1 \ub370\uc774\ud130\uc14b \uc900\ube44<\/h3>\n
\uc6b0\uc120 \uc2e0\ub8b0\uc131 \uc788\ub294 \ub370\uc774\ud130\ub97c \uc900\ube44\ud574\uc57c \ud569\ub2c8\ub2e4. \uc608\ub97c \ub4e4\uc5b4, \uc601\ud654 \ub9ac\ubdf0 \ub370\uc774\ud130\uc14b\uc740 \uae0d\uc815\uc801\/\ubd80\uc815\uc801 \ub9ac\ubdf0\ub85c \ub098\ub258\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.
\n \uc774\ub97c \uc0ac\uc6a9\ud558\uc5ec \ubaa8\ub378\uc744 \ud559\uc2b5\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ub370\uc774\ud130\ub97c \ub85c\ub4dc\ud558\uace0 \uc804\ucc98\ub9ac\ub97c \uc218\ud589\ud558\ub294 \ucf54\ub4dc\ub97c \uc791\uc131\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
import pandas as pd\n\n# \uc0d8\ud50c \ub370\uc774\ud130\ub85c\ub4dc (\uae0d\uc815 \ub9ac\ubdf0 5\uac1c, \ubd80\uc815 \ub9ac\ubdf0 5\uac1c)\ndata = {\n \"text\": [\n \"\ub098\ub294 \uc774 \uc601\ud654\ub97c \uc815\ub9d0 \uc88b\uc544\ud55c\ub2e4.\", \n \"\ucd5c\uace0\uc758 \uc601\ud654\ub2e4!\", \n \"\uc815\ub9d0 \uac10\ub3d9\uc801\uc774\uc5c8\ub2e4.\", \n \"\uc644\ubcbd\ud55c \uba85\uc791\uc774\ub2e4.\", \n \"\uc774 \uc601\ud654\ub294 \ub0b4 \ub9c8\uc74c\uc744 \uc6b8\ub838\ub2e4.\",\n \"\uc774\uac83\uc740 \uc2dc\uac04 \ub0ad\ube44\uc774\ub2e4.\", \n \"\ub098\uc058\uace0 \uc9c0\ub8e8\ud558\ub2e4.\", \n \"\uc815\ub9d0 \uc2e4\ub9dd\uc2a4\ub7ec\uc6e0\ub2e4.\", \n \"\uc808\ub300 \ubcf4\uc9c0 \ub9c8\ub77c.\", \n \"\uc774 \uc601\ud654\ub294 \ucd5c\uc545\uc774\ub2e4.\"\n ],\n \"label\": [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]\n}\n\ndf = pd.DataFrame(data)\nprint(df.head())<\/code><\/pre>\n5.2 \ubaa8\ub378 \ud559\uc2b5<\/h3>\n
\uc774\uc81c \ub370\uc774\ud130\ub97c \ubaa8\ub378\uc5d0 \ud559\uc2b5\uc2dc\ud0a4\uae30 \uc704\ud55c \uacfc\uc815\uc785\ub2c8\ub2e4. \uac04\ub2e8\ud55c \ud6c8\ub828 \ub8e8\ud504\ub97c \uc791\uc131\ud558\uc5ec \ubaa8\ub378\uc744 \ud559\uc2b5\uc2dc\ud0ac \uc218 \uc788\uc2b5\ub2c8\ub2e4.
\n \uadf8\ub7ec\ub098 \uc5ec\uae30\uc11c\ub294 \ud6c8\ub828 \uacfc\uc815\uc5d0 \ub300\ud55c \ubcf4\ub2e4 \uae4a\uc740 \ub0b4\uc6a9\uc744 \ub2e4\ub8e8\uc9c0 \uc54a\uace0, \uc8fc\uc5b4\uc9c4 \ub370\uc774\ud130\ub85c \uac04\ub2e8\ud55c \uc804\uc774 \ud559\uc2b5\uc744 \uc9c4\ud589\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
from torch.utils.data import DataLoader, Dataset\n\nclass CustomDataset(Dataset):\n def __init__(self, texts, labels, tokenizer):\n self.texts = texts\n self.labels = labels\n self.tokenizer = tokenizer\n\n def __len__(self):\n return len(self.labels)\n\n def __getitem__(self, idx):\n text = self.texts[idx]\n label = self.labels[idx]\n encoding = self.tokenizer(text, return_tensors='pt', padding='max_length', truncation=True, max_length=128)\n return {'input_ids': encoding['input_ids'].flatten(), 'attention_mask': encoding['attention_mask'].flatten(), 'label': torch.tensor(label, dtype=torch.long)}\n\ndataset = CustomDataset(df['text'].values, df['label'].values, tokenizer)\ndataloader = DataLoader(dataset, batch_size=2, shuffle=True)\n\n# \ub2e8\uc21c \ud6c8\ub828 \ub8e8\ud504\nfor epoch in range(3):\n for batch in dataloader:\n model.train()\n outputs = model(input_ids=batch['input_ids'], attention_mask=batch['attention_mask'], labels=batch['label'])\n loss = outputs.loss\n loss.backward()\n # Optimizer Step \ub4f1\uc740 \uc0dd\ub7b5\n print(f\"Epoch {epoch + 1}, Loss: {loss.item()}\")<\/code><\/pre>\n5.3 \uc608\uce21 \ubc0f \ud3c9\uac00<\/h3>\n
\ubaa8\ub378\uc774 \ud559\uc2b5\ub41c \ud6c4, \uc0c8\ub85c\uc6b4 \ubb38\uc7a5\uc5d0 \ub300\ud574 \uc608\uce21\uc744 \uc218\ud589\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ub2e4\uc74c \uc608\uc81c\ub97c \ud1b5\ud574 \uc774\ub97c \ud655\uc778\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4:<\/p>\n
test_text = \"\uc774 \uc601\ud654\ub294 \uc544\uc8fc \uc88b\uc2b5\ub2c8\ub2e4.\"\ntest_inputs = tokenizer(test_text, return_tensors=\"pt\")\n\nwith torch.no_grad():\n test_logits = model(**test_inputs).logits\n\ntest_predicted_class = torch.argmax(test_logits, dim=-1).item()\nprint(f\"\ud14c\uc2a4\ud2b8 \ubb38\uc7a5 '{test_text}'\uc5d0 \ub300\ud55c \uc608\uce21 \ud074\ub798\uc2a4: {test_predicted_class}\")<\/code><\/pre>\n6. \ub9c8\ubb34\ub9ac<\/h2>\n
\uc774\ubc88 \uac15\uc88c\uc5d0\uc11c\ub294 \ud5c8\uae45\ud398\uc774\uc2a4 \ud2b8\ub79c\uc2a4\ud3ec\uba38 \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uc0ac\uc6a9\ud558\uc5ec Mobile BERT \ubaa8\ub378\uc744 \uc124\uce58\ud558\uace0, \uc0ac\uc804 \ud559\uc2b5\ub41c \ubaa8\ub378\uc744 \ubd88\ub7ec\uc640
\n \ubb38\uc7a5 \ubd84\ub958 \uc791\uc5c5\uc744 \uc218\ud589\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud574 \uc0b4\ud3b4\ubcf4\uc558\uc2b5\ub2c8\ub2e4. Mobile BERT\ub294 \uac00\ubcbc\uc6b4 \ubaa8\ub378\uc774\uae30 \ub54c\ubb38\uc5d0 \ubaa8\ubc14\uc77c \ud658\uacbd\uc774\ub098 \ub9ac\uc18c\uc2a4\uac00 \uc81c\ud55c\ub41c \ud658\uacbd\uc5d0\uc11c\ub3c4 \uc720\uc6a9\ud558\uac8c \ud65c\uc6a9\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.
\n \ub2e4\uc591\ud55c NLP \ud0dc\uc2a4\ud06c\uc5d0 \ub300\ud55c \uc801\uc6a9 \uac00\ub2a5\uc131\uc744 \ub354 \uc5f0\uad6c\ud574\ubcf4\ub294 \uac83\uc744 \uad8c\uc7a5\ud569\ub2c8\ub2e4.<\/p>\n
\uc774 \uac15\uc88c\uac00 \ub3c4\uc6c0\uc774 \ub418\uc5c8\ub2e4\uba74, \uc8fc\ubcc0\uc5d0 \uacf5\uc720\ud574\uc8fc\uc138\uc694! \ucd94\uac00\uc801\uc778 \uc790\ub8cc\ub098 \uc9c8\ubb38\uc774 \uc788\uc73c\uc2dc\uba74 \uc5b8\uc81c\ub4e0\uc9c0 \ub313\uae00\ub85c \ub0a8\uaca8\uc8fc\uc138\uc694.<\/p>\n
<\/body><\/p>\n","protected":false},"excerpt":{"rendered":"