With the recent advancements in artificial intelligence and machine learning, deep learning technologies are being utilized in many fields. In particular, in the field of Natural Language Processing (NLP), the Hugging Face Transformers library has made it easy to use various models. In this course, we will explain in detail the data preprocessing techniques using regular expressions along with an example of document classification using Hugging Face Transformers.<\/p>\n
Hugging Face Transformers is a Python library that provides various deep learning models commonly used in Natural Language Processing (NLP). It includes many of the latest models such as BERT, GPT-2, and T5, designed for users to easily access and utilize. This library is written in Python, making it widely used by data scientists and researchers.<\/p>\n
Regular expressions are a very useful tool for finding or transforming specific patterns in strings. By using regular expressions to remove unnecessary characters and perform pattern matching before inputting data into the model, the quality of the data can be improved. Preprocessing directly affects the model’s performance, so it requires sufficient attention.<\/p>\n
First, we will install Hugging Face Transformers and the necessary libraries. Run the command below to install the libraries:<\/p>\n
pip install transformers pandas re<\/code><\/pre>\n4. Preparing the Data<\/h2>\n
In this example, we will use a simple dataset for sentiment analysis. The data consists of sentences that represent positive and negative sentiments.<\/p>\n
import pandas as pd\n\ndata = {\n \"text\": [\n \"This product is really good!\",\n \"Not great. I was very disappointed.\",\n \"It's not a bad product.\",\n \"I hope for a refund.\",\n \"It really exceeded my expectations!\",\n ],\n \"label\": [1, 0, 1, 0, 1] # 1: positive, 0: negative\n}\n\ndf = pd.DataFrame(data)\nprint(df)<\/code><\/pre>\n5. Data Preprocessing Using Regular Expressions<\/h2>\n
Next, we will perform data preprocessing using regular expressions. For example, we will remove special characters or numbers and convert all characters to lowercase.<\/p>\n
import re\n\ndef preprocess_text(text):\n # Convert to lowercase\n text = text.lower()\n # Remove special characters and numbers\n text = re.sub(r'[^a-z\uac00-\ud7a3\\s]', '', text)\n return text\n\ndf['cleaned_text'] = df['text'].apply(preprocess_text)\nprint(df[['text', 'cleaned_text']])<\/code><\/pre>\n6. Training the Model Using Hugging Face Transformers<\/h2>\n
After preprocessing is complete, we will train a model for sentiment analysis using a transformer model. Below is an example code using the BERT model.<\/p>\n
from transformers import BertTokenizer, BertForSequenceClassification\nfrom transformers import Trainer, TrainingArguments\nimport torch\nfrom sklearn.model_selection import train_test_split\n\n# Split the data\nX_train, X_test, y_train, y_test = train_test_split(df['cleaned_text'], df['label'], test_size=0.2, random_state=42)\n\n# Load BERT tokenizer and model\ntokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\nmodel = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)\n\n# Tokenize the data\ntrain_encodings = tokenizer(X_train.tolist(), padding=True, truncation=True, return_tensors='pt')\ntest_encodings = tokenizer(X_test.tolist(), padding=True, truncation=True, return_tensors='pt')\n\n# Define PyTorch dataset class\nclass TextDataset(torch.utils.data.Dataset):\n def __init__(self, encodings, labels):\n self.encodings = encodings\n self.labels = labels\n\n def __getitem__(self, idx):\n item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}\n item['labels'] = torch.tensor(self.labels[idx])\n return item\n\n def __len__(self):\n return len(self.labels)\n\n# Prepare the dataset\ntrain_dataset = TextDataset(train_encodings, y_train.tolist())\ntest_dataset = TextDataset(test_encodings, y_test.tolist())\n\n# Set training arguments\ntraining_args = TrainingArguments(\n output_dir='.\/results',\n num_train_epochs=3,\n per_device_train_batch_size=4,\n per_device_eval_batch_size=4,\n warmup_steps=500,\n weight_decay=0.01,\n logging_dir='.\/logs',\n)\n\n# Define the trainer\ntrainer = Trainer(\n model=model,\n args=training_args,\n train_dataset=train_dataset,\n eval_dataset=test_dataset,\n)\n\n# Train the model\ntrainer.train()<\/code><\/pre>\n7. Model Evaluation<\/h2>\n
After the model training is complete, you can evaluate the model’s performance. Calculate the accuracy and visualize the confusion matrix to analyze the model’s performance.<\/p>\n
from sklearn.metrics import accuracy_score, confusion_matrix\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Perform predictions\npredictions = trainer.predict(test_dataset)\npreds = predictions.predictions.argmax(-1)\n\n# Calculate accuracy\naccuracy = accuracy_score(y_test, preds)\nprint(f'Accuracy: {accuracy:.2f}')\n\n# Visualize the confusion matrix\ncm = confusion_matrix(y_test, preds)\nplt.figure(figsize=(8, 6))\nsns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=['Negative', 'Positive'], yticklabels=['Negative', 'Positive'])\nplt.ylabel('Actual')\nplt.xlabel('Predicted')\nplt.title('Confusion Matrix')\nplt.show()<\/code><\/pre>\n8. Conclusion<\/h2>\n
In this course, we explained how to build a basic sentiment analysis model using the Hugging Face Transformers library. We saw how improving data quality through regular expression preprocessing can lead to high performance when using transformer models. It would be beneficial to continue working on projects utilizing various natural language processing technologies.<\/p>\n
Thank you!<\/p>\n
<\/body><\/p>\n","protected":false},"excerpt":{"rendered":"