In recent years, with the rapid advancement of artificial intelligence (AI) technologies, significant progress has been made in the field of natural language processing (NLP). In particular, deep learning-based models have shown excellent performance in language understanding and generation. In this article, we will discuss how to classify Naver movie reviews using KoBERT, a model optimized for the Korean language based on BERT (Bidirectional Encoder Representations from Transformers).<\/p>\n
The goal of this project is to classify whether user reviews of Naver movies are positive or negative based on the review data. Through this, participants can understand the basic concepts of natural language processing and how to use the KoBERT model, while also gaining hands-on experience in data preprocessing and model training.<\/p>\n
KoBERT is a model trained on Google’s BERT model, specifically optimized for the Korean language. BERT is based on two main components: the first is the ‘Masked Language Model,’ where certain words in a sentence are randomly masked so the model can predict these words. The second is ‘Next Sentence Prediction,’ which determines whether the second of two provided sentences is the next sentence following the first one. This transfer learning technique has proven effective in many natural language processing tasks.<\/p>\n
In this project, we will use Naver movie review data. This dataset consists of user reviews of movies along with corresponding positive or negative labels for those reviews. The data is provided in CSV format, and we will prepare the dataset after installing the necessary libraries.<\/p>\n
import pandas as pd\n\n# Load the dataset\ndf = pd.read_csv('naver_movie_reviews.csv')\ndf.head()<\/code><\/pre>\nEach column of the dataset contains movie reviews and their corresponding sentiment labels. We need to undergo necessary preprocessing to analyze this data.<\/p>\n
4. Data Preprocessing<\/h3>\n
Data preprocessing is a crucial step in machine learning. To convert review texts into a format suitable for the model, the following tasks are performed:<\/p>\n
\n- Removing Stop Words<\/strong>: Eliminate common words that do not add meaning.<\/li>\n
- Tokenization<\/strong>: Split sentences into words.<\/li>\n
- Normalization<\/strong>: Standardize words with similar meanings.<\/li>\n<\/ul>\n
from sklearn.model_selection import train_test_split\nfrom transformers import BertTokenizer\n\n# Load KoBERT tokenizer\ntokenizer = BertTokenizer.from_pretrained('kykim\/bert-kor-base')\n\n# Separate review texts and labels\nsentences = df['review'].values\nlabels = df['label'].values\n\n# Split into training and testing data\nX_train, X_test, y_train, y_test = train_test_split(sentences, labels, test_size=0.1, random_state=42)<\/code><\/pre>\n5. Define Dataset Class<\/h3>\n
To train the KoBERT model using PyTorch, we define a dataset class. This class serves to transform the input data into a format that the model can process.<\/p>\n
from torch.utils.data import Dataset\n\nclass NaverMovieDataset(Dataset):\n def __init__(self, texts, labels, tokenizer, max_length=128):\n self.texts = texts\n self.labels = labels\n self.tokenizer = tokenizer\n self.max_length = max_length\n\n def __len__(self):\n return len(self.texts)\n\n def __getitem__(self, idx):\n text = self.texts[idx]\n label = self.labels[idx]\n encoding = self.tokenizer(\n text,\n truncation=True,\n padding='max_length',\n max_length=self.max_length,\n return_tensors='pt'\n )\n return {\n 'input_ids': encoding['input_ids'].squeeze(0),\n 'attention_mask': encoding['attention_mask'].squeeze(0),\n 'labels': torch.tensor(label, dtype=torch.long)\n }<\/code><\/pre>\n6. Define Class for Model Training, Evaluation, and Prediction<\/h3>\n
We define a single class for training, evaluating, and predicting with the model to maintain clean code.<\/p>\n
import torch\nfrom torch.utils.data import DataLoader\nfrom transformers import BertForSequenceClassification, AdamW\nfrom sklearn.metrics import classification_report\n\nclass KoBERTSentimentClassifier:\n def __init__(self, model_name='kykim\/bert-kor-base', num_labels=2, learning_rate=1e-5):\n self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n self.model = BertForSequenceClassification.from_pretrained(model_name, num_labels=num_labels).to(self.device)\n self.optimizer = AdamW(self.model.parameters(), lr=learning_rate)\n\n def train(self, train_dataset, batch_size=16, epochs=3):\n train_dataloader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n self.model.train()\n for epoch in range(epochs):\n for batch in train_dataloader:\n self.optimizer.zero_grad()\n input_ids = batch['input_ids'].to(self.device)\n attention_mask = batch['attention_mask'].to(self.device)\n labels = batch['labels'].to(self.device)\n outputs = self.model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n loss = outputs.loss\n loss.backward()\n self.optimizer.step()\n print(f\"Epoch: {epoch + 1}, Loss: {loss.item()}\")\n\n def evaluate(self, test_dataset, batch_size=16):\n test_dataloader = DataLoader(test_dataset, batch_size=batch_size)\n self.model.eval()\n predictions, true_labels = [], []\n with torch.no_grad():\n for batch in test_dataloader:\n input_ids = batch['input_ids'].to(self.device)\n attention_mask = batch['attention_mask'].to(self.device)\n labels = batch['labels'].to(self.device)\n outputs = self.model(input_ids=input_ids, attention_mask=attention_mask)\n logits = outputs.logits\n predictions.extend(torch.argmax(logits, dim=1).cpu().numpy())\n true_labels.extend(labels.cpu().numpy())\n print(classification_report(true_labels, predictions))\n\n def predict(self, texts, tokenizer, max_length=128):\n self.model.eval()\n inputs = tokenizer(\n texts,\n truncation=True,\n padding='max_length',\n max_length=max_length,\n return_tensors='pt'\n ).to(self.device)\n with torch.no_grad():\n outputs = self.model(input_ids=inputs['input_ids'], attention_mask=inputs['attention_mask'])\n predictions = torch.argmax(outputs.logits, dim=1)\n return predictions.cpu().numpy()<\/code><\/pre>\n7. Conclusion<\/h3>\n
In this article, we explored the process of classifying Naver movie reviews using KoBERT. By learning how to process text data using deep learning-based natural language processing models, I hope this has provided a good opportunity to familiarize oneself with the fundamentals of natural language processing. Now, a foundation has been established to proceed with various natural language processing projects based on this technology.<\/p>\n","protected":false},"excerpt":{"rendered":"