# How to use Auto Model For SequenceClassification for Multi-Class Text Classification?

**URL:** <https://discuss.huggingface.co/t/how-to-use-auto-model-for-sequenceclassification-for-multi-class-text-classification/18659>\
**Category:** 🤗AutoTrain\
**Created:** [June 2, 2022, 12:55pm UTC](https://discuss.huggingface.co/t/how-to-use-auto-model-for-sequenceclassification-for-multi-class-text-classification/18659 "2022-06-02T12:55:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![aaditya](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aaditya/32/20855_2.png) [@aaditya](https://discuss.huggingface.co/u/aaditya)\
**Post date:** [June 2, 2022, 12:55pm UTC](https://discuss.huggingface.co/t/how-to-use-auto-model-for-sequenceclassification-for-multi-class-text-classification/18659/1 "2022-06-02T12:55:38Z")

</div>

I am trying to use Hugginface’s AutoModelForSequence Classification API for multi-class classification but am confused about its configuration.

My dataset is in one hot encoded and the problem type is multi-class (one label at a time)

What I have tried:

```auto
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased",
                                                           num_labels=6,
                                                           id2label=id2label,
                                                           label2id=label2id)

batch_size = 8
metric_name = "f1"

from transformers import TrainingArguments, Trainer

args = TrainingArguments(
    f"bert-finetuned-english",
    evaluation_strategy = "epoch",
    save_strategy = "epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=batch_size,
    per_device_eval_batch_size=batch_size,
    num_train_epochs=10,
    weight_decay=0.01,
    load_best_model_at_end=True,
    metric_for_best_model=metric_name,
    #push_to_hub=True,
)

trainer = Trainer(
    model,
    args,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["test"],
    tokenizer=tokenizer,
    compute_metrics=compute_metrics
)

```

is it correct?

I am confused about the loss function, when I am printing one forward pass the loss is BinaryCrossEntropyWithLogits

```auto
SequenceClassifierOutput([('loss',
                           tensor(0.6986, grad_fn=<BinaryCrossEntropyWithLogitsBackward0>)),
                          ('logits',
                           tensor([[-0.5496, 0.0793, -0.5429, -0.1162, -0.0551]],
                                  grad_fn=<AddmmBackward0>))])

```

which is used for multi-label or binary classification tasks. It should use nn.CrossEntropyLoss?

How to properly use this API for multiclass and define the loss function?

---

<div class="post-metadata">

**Author:** ![ChanceChallacombe](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@ChanceChallacombe](https://discuss.huggingface.co/u/ChanceChallacombe)\
**Post date:** [February 26, 2023, 6:46am UTC](https://discuss.huggingface.co/t/how-to-use-auto-model-for-sequenceclassification-for-multi-class-text-classification/18659/2 "2023-02-26T06:46:12Z")

</div>

Also looking for this, did you figure it out?