# 文本分类

文本分类是一种常见的 NLP 任务，用于为文本分配标签或类别。许多大型企业在生产环境中运行文本分类，以支持广泛的实际应用。文本分类最流行的形式之一是情感分析，它为一段文本序列分配诸如 🙂 正面、🙁 负面或 😐 中性之类的标签。

本指南将向您展示如何：

1. 在 [IMDb](https://huggingface.co/datasets/stanfordnlp/imdb) 数据集上微调 [DistilBERT](https://huggingface.co/distilbert/distilbert-base-uncased)，以判断电影评论是正面还是负面。
2. 使用微调后的模型进行推理。

要查看与此任务兼容的所有架构和检查点，我们建议查阅[任务页面](https://huggingface.co/tasks/text-classification)。

在开始之前，请确保已安装所有必要的库：

```bash
pip install transformers datasets evaluate accelerate
```

我们建议您登录 Hugging Face 账户，以便上传模型并与社区共享。当系统提示时，请输入您的令牌以完成登录：

```py
>>> from huggingface_hub import notebook_login

>>> notebook_login()
```

## 加载 IMDb 数据集

首先，从 🤗 Datasets 库中加载 IMDb 数据集：

```py
>>> from datasets import load_dataset

>>> imdb = load_dataset("stanfordnlp/imdb")
```

然后来看一个示例：

```py
>>> imdb["test"][0]
{
    "label": 0,
    "text": "I love sci-fi and am willing to put up with a lot. Sci-fi movies/TV are usually underfunded, under-appreciated and misunderstood. I tried to like this, I really did, but it is to good TV sci-fi as Babylon 5 is to Star Trek (the original). Silly prosthetics, cheap cardboard sets, stilted dialogues, CG that doesn't match the background, and painfully one-dimensional characters cannot be overcome with a 'sci-fi' setting. (I'm sure there are those of you out there who think Babylon 5 is good sci-fi TV. It's not. It's clichéd and uninspiring.) While US viewers might like emotion and character development, sci-fi is a genre that does not take itself seriously (cf. Star Trek). It may treat important issues, yet not as a serious philosophy. It's really difficult to care about the characters here as they are not simply foolish, just missing a spark of life. Their actions and reactions are wooden and predictable, often painful to watch. The makers of Earth KNOW it's rubbish as they have to always say \"Gene Roddenberry's Earth...\" otherwise people would not continue watching. Roddenberry's ashes must be turning in their orbit as this dull, cheap, poorly edited (watching it without advert breaks really brings this home) trudging Trabant of a show lumbers into space. Spoiler. So, kill off a main character. And then bring him back as another actor. Jeeez! Dallas all over again.",
}
```

此数据集包含两个字段：

- `text`：电影评论文本。
- `label`：一个值，负面评论为 `0`，正面评论为 `1`。

## 预处理

下一步是加载 DistilBERT 分词器，以预处理 `text` 字段：

```py
>>> from transformers import AutoTokenizer

>>> tokenizer = AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased")
```

创建一个预处理函数，用于对 `text` 进行分词，并将序列截断至不超过 DistilBERT 的最大输入长度：

```py
>>> def preprocess_function(examples):
...     return tokenizer(examples["text"], truncation=True)
```

要对整个数据集应用预处理函数，请使用 🤗 Datasets 的 [map](https://huggingface.co/docs/datasets/v5.2.0/en/package_reference/main_classes#datasets.Dataset.map) 函数。你可以通过设置 `batched=True` 来一次处理数据集中的多个元素，从而加速 `map`：

```py
tokenized_imdb = imdb.map(preprocess_function, batched=True)
```

现在使用 [DataCollatorWithPadding](/docs/transformers/v5.23.0/zh/main_classes/data_collator#transformers.DataCollatorWithPadding) 创建一个示例批次。在整理数据时，将句子*动态填充*到批次中最长的长度，比将整个数据集填充到最大长度更高效。

```py
>>> from transformers import DataCollatorWithPadding

>>> data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
```

## 评估

在训练过程中包含一个指标通常有助于评估模型的性能。你可以使用 🤗 [Evaluate](https://huggingface.co/docs/evaluate/index) 库快速加载一个评估方法。对于此任务，请加载 [accuracy](https://huggingface.co/spaces/evaluate-metric/accuracy) 指标（请参阅 🤗 Evaluate 的 [快速入门](https://huggingface.co/docs/evaluate/a_quick_tour) 以了解如何加载和计算指标）：

```py
>>> import evaluate

>>> accuracy = evaluate.load("accuracy")
```

然后创建一个函数，将你的预测值和标签传递给 [compute](https://huggingface.co/docs/evaluate/v0.4.6/en/package_reference/main_classes#evaluate.EvaluationModule.compute) 以计算准确率：

```py
>>> import numpy as np

>>> def compute_metrics(eval_pred):
...     predictions, labels = eval_pred
...     predictions = np.argmax(predictions, axis=1)
...     return accuracy.compute(predictions=predictions, references=labels)
```

您的 `compute_metrics` 函数现已准备就绪，您将在配置训练时再次使用它。

## 训练

在开始训练模型之前，请使用 `id2label` 和 `label2id` 创建预期 id 与其标签之间的映射：

```py
>>> id2label = {0: "NEGATIVE", 1: "POSITIVE"}
>>> label2id = {"NEGATIVE": 0, "POSITIVE": 1}
```

如果您不熟悉使用 [Trainer](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.Trainer) 对模型进行微调，请查看 [此处](../training) 的基础教程！

现在您可以开始训练模型了！使用 [AutoModelForSequenceClassification](/docs/transformers/v5.23.0/zh/model_doc/auto#transformers.AutoModelForSequenceClassification) 加载 DistilBERT，并指定预期的标签数量以及标签映射：

```py
>>> from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer

>>> model = AutoModelForSequenceClassification.from_pretrained(
...     "distilbert/distilbert-base-uncased", num_labels=2, id2label=id2label, label2id=label2id
... )
```

此时，只剩下三个步骤：

1. 在 [TrainingArguments](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.TrainingArguments) 中定义训练超参数。唯一必需的参数是 `output_dir`，用于指定模型的保存位置。通过设置 `push_to_hub=True`，你可以将该模型推送到 Hub（上传模型需要登录 Hugging Face 账户）。在每个 epoch 结束时，[Trainer](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.Trainer) 会评估准确率并保存训练检查点。
2. 将训练参数与模型、数据集、分词器、数据整理器以及 `compute_metrics` 函数一起传递给 [Trainer](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.Trainer)。
3. 调用 [train()](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.Trainer.train) 来微调你的模型。

```py
>>> training_args = TrainingArguments(
...     output_dir="my_awesome_model",
...     learning_rate=2e-5,
...     per_device_train_batch_size=16,
...     per_device_eval_batch_size=16,
...     num_train_epochs=2,
...     weight_decay=0.01,
...     eval_strategy="epoch",
...     save_strategy="epoch",
...     load_best_model_at_end=True,
...     push_to_hub=True,
... )

>>> trainer = Trainer(
...     model=model,
...     args=training_args,
...     train_dataset=tokenized_imdb["train"],
...     eval_dataset=tokenized_imdb["test"],
...     processing_class=tokenizer,
...     data_collator=data_collator,
...     compute_metrics=compute_metrics,
... )

>>> trainer.train()
```

当你将 `tokenizer` 传递给 [Trainer](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.Trainer) 时，它默认会应用动态填充。在这种情况下，你无需显式指定数据整理器。

训练完成后，使用 [push_to_hub()](/docs/transformers/v5.23.0/zh/main_classes/trainer#transformers.Trainer.push_to_hub) 方法将您的模型共享到 Hub，以便所有人都能使用您的模型：

```py
>>> trainer.push_to_hub()
```

如需了解如何针对文本分类任务微调模型的更详细示例，请参阅对应的 [PyTorch notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/text_classification.ipynb)。

## 推理

很好，现在你已经微调了一个模型，可以使用它进行推理了！

获取一些您想要进行推理的文本：

```py
>>> text = "This was a masterpiece. Not completely faithful to the books, but enthralling from beginning to end. Might be my favorite of the three."
```

尝试使用微调后的模型进行推理的最简单方法是将其用于 [pipeline()](/docs/transformers/v5.23.0/zh/main_classes/pipelines#transformers.pipeline)。使用你的模型实例化一个用于情感分析的 `pipeline`，并将你的文本传入其中：

```py
>>> from transformers import pipeline

>>> classifier = pipeline("sentiment-analysis", model="stevhliu/my_awesome_model")
>>> classifier(text)
[{'label': 'POSITIVE', 'score': 0.9994940757751465}]
```

您也可以手动复现 `pipeline` 的结果：

对文本进行分词并返回 PyTorch 张量：

```py
>>> from transformers import AutoTokenizer

>>> tokenizer = AutoTokenizer.from_pretrained("stevhliu/my_awesome_model")
>>> inputs = tokenizer(text, return_tensors="pt")
```

将输入传递给模型并返回 `logits`：

```py
>>> import torch
>>> from transformers import AutoModelForSequenceClassification

>>> model = AutoModelForSequenceClassification.from_pretrained("stevhliu/my_awesome_model")
>>> with torch.no_grad():
...     logits = model(**inputs).logits
```

获取概率最高的类别，并使用模型的 `id2label` 映射将其转换为文本标签：

```py
>>> predicted_class_id = logits.argmax().item()
>>> model.config.id2label[predicted_class_id]
'POSITIVE'
```

