{"id":36203,"date":"2024-11-01T09:46:36","date_gmt":"2024-11-01T09:46:36","guid":{"rendered":"http:\/\/atmokpo.com\/w\/?p=36203"},"modified":"2024-11-01T09:46:36","modified_gmt":"2024-11-01T09:46:36","slug":"hugging-face-transformers-tutorial-loading-pre-trained-models","status":"publish","type":"post","link":"https:\/\/atmokpo.com\/w\/36203\/","title":{"rendered":"Hugging Face Transformers Tutorial: Loading Pre-trained Models"},"content":{"rendered":"<\/p>\n\n\n<\/header>\n\n
1. Introduction<\/h2>\n
\n The advancement of deep learning has achieved remarkable results, especially in the field of natural language processing (NLP). At the center of these advancements are \n pre-trained models. Hugging Face provides a powerful library called Transformers<\/strong> that makes it easier to use these pre-trained models. In this course, we will learn in detail how to load pre-trained models using Hugging Face’s \n Transformers library.\n <\/p>\n<\/section>\n\n
2. What is the Hugging Face Transformers Library?<\/h2>\n
\n The Hugging Face Transformers library is a library that provides various natural language processing (NLP) models, \n including BERT<\/strong>, GPT<\/strong>, RoBERTa<\/strong>, T5<\/strong>, and several others. With this library, developers can easily load pre-trained language models and \n perform various NLP tasks based on them.\n <\/p>\n<\/section>\n\n
3. Environment Setup<\/h2>\n
\n Before we get started, we need to install the required libraries. You can install the basic libraries using the command below.\n <\/p>\n
pip install transformers torch<\/code><\/pre>\n
\n Here, transformers<\/strong> is the Hugging Face library, and torch<\/strong> is the \n PyTorch framework. If you want to use TensorFlow instead of PyTorch, you can install TensorFlow.\n <\/p>\n<\/section>\n\n
4. Loading Pre-trained Models<\/h2>\n
\n Now let’s load a pre-trained model. For example, we can understand the meaning of text using the BERT<\/strong> model. Below is a way to load the BERT model using Python code.\n <\/p>\n
from transformers import BertTokenizer, BertModel\n\n# Load BERT model's tokenizer and model\ntokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\nmodel = BertModel.from_pretrained('bert-base-uncased')\n\n# Example sentence\nsentence = \"Hugging Face is creating a tool that democratizes AI.\"\n\n# Tokenize the sentence and convert it to input vectors\ninputs = tokenizer(sentence, return_tensors='pt')\noutputs = model(**inputs)\n\n# Check the output\nprint(outputs)\n<\/code><\/pre>\n
\n In the code above, the BertTokenizer<\/strong> class converts the input sentence into a \n format that the BERT model can understand. The BertModel<\/strong> class loads the actual model and passes \n the transformed input through the model to generate the output.\n <\/p>\n<\/section>\n\n
5. Analyzing Output Results<\/h2>\n
\n The outputs<\/strong> variable in the code above contains two main pieces of information:\n <\/p>\n
\n
last_hidden_state<\/strong>: The last hidden state, showing the vector representation of each token.<\/li>\n
pooler_output<\/strong>: A vector summarizing the entire input sequence, mainly used for classification tasks.<\/li>\n<\/ul>\n
\n The vector representation of each token is very useful information for natural language processing. The hidden state outputted for each token can be accessed as below.\n <\/p>\n
# Accessing the last hidden state\nhidden_states = outputs.last_hidden_state\nprint(hidden_states.shape) # (batch size, sequence length, hidden dimension)\n<\/code><\/pre>\n<\/section>\n\n
6. Using Various Pre-trained Models<\/h2>\n
\n Hugging Face supports several other models in addition to BERT. It provides various models so that users can choose the models suitable for different tasks. The usage of models like GPT-2<\/strong>, RoBERTa<\/strong>, \n T5<\/strong> is quite similar. For example, if you want to use the GPT-2<\/strong> model, you can load it as follows.\n <\/p>\n
from transformers import GPT2Tokenizer, GPT2Model\n\n# Load GPT-2 model's tokenizer and model\ntokenizer = GPT2Tokenizer.from_pretrained('gpt2')\nmodel = GPT2Model.from_pretrained('gpt2')\n\n# Example sentence\nsentence = \"Hugging Face has become a leader in NLP.\"\n\n# Tokenize the sentence and convert it to input vectors\ninputs = tokenizer(sentence, return_tensors='pt')\noutputs = model(**inputs)\n\nprint(outputs)\n<\/code><\/pre>\n<\/section>\n\n
7. Training Your Own Model<\/h2>\n
\n In addition to obtaining pre-trained models, users can also fine-tune models for their datasets. This process involves the following steps:\n <\/p>\n
\n
Data preparation and preprocessing<\/li>\n
Loading a pre-trained model<\/li>\n
Setting up the loss function and optimizer for training<\/li>\n
Training the model<\/li>\n<\/ol>\n
7.1 Data Preparation and Preprocessing<\/h3>\n
\n Data can be prepared in a format such as a CSV file, and a series of processes are required to load and preprocess it.\n <\/p>\n
import pandas as pd\n\n# Load the dataset\ndata = pd.read_csv('data.csv')\nprint(data.head()) # Check the first 5 rows of the dataset\n<\/code><\/pre>\n
7.2 Loading a Pre-trained Model<\/h3>\n
\n You can load the model in the way explained above.\n <\/p>\n
7.3 Setting Up Loss Function and Optimizer<\/h3>\n
\n For model training, the loss function and optimizer need to be set. For example, you can use the \n AdamW<\/strong> optimizer and CrossEntropyLoss<\/strong> loss function.\n <\/p>\n
from transformers import AdamW\n\noptimizer = AdamW(model.parameters(), lr=5e-5) # Set the learning rate\nloss_fn = torch.nn.CrossEntropyLoss()\n<\/code><\/pre>\n
7.4 Training the Model<\/h3>\n
\n You can train the model using the preprocessed data along with the configured loss function and optimizer. \n Typically, you set the number of epochs and iterate to optimize the model.\n <\/p>\n
for epoch in range(num_epochs):\n model.train()\n outputs = model(**inputs) # Model's output\n loss = loss_fn(outputs, labels) # Calculate loss\n loss.backward() # Backpropagation\n optimizer.step() # Update weights\n optimizer.zero_grad() # Reset gradients\n<\/code><\/pre>\n<\/section>\n\n
8. Conclusion<\/h2>\n
\n Through this course, we have learned how to use Hugging Face’s Transformers library to load pre-trained models and perform various tasks based on them. This library serves as a powerful tool in the field of \n natural language processing, especially helping to utilize models with a consistent dataset and easy API. Now you are equipped with the ability to use Hugging Face’s Transformers for your own projects.\n <\/p>\n<\/section>\n