# How to get embeddings from Finetuned BERT classification model?

**URL:** <https://discuss.huggingface.co/t/how-to-get-embeddings-from-finetuned-bert-classification-model/43948>\
**Category:** 🤗Transformers\
**Created:** [June 20, 2023, 4:37pm UTC](https://discuss.huggingface.co/t/how-to-get-embeddings-from-finetuned-bert-classification-model/43948 "2023-06-20T16:37:49Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![aljanaidehosu](https://avatars.discourse-cdn.com/v4/letter/a/41988e/32.png) [@aljanaidehosu](https://discuss.huggingface.co/u/aljanaidehosu)\
**Post date:** [June 20, 2023, 4:37pm UTC](https://discuss.huggingface.co/t/how-to-get-embeddings-from-finetuned-bert-classification-model/43948/1 "2023-06-20T16:37:49Z")

</div>

So I have the code below which finetunes the BERT model for sequence classification. How can I get the finetuned embeddings? I know how to get the pre-trained embeddings but here I want the embeddings AFTER finetuning.

```auto
from transformers import TFDistilBertForSequenceClassification, TFTrainer, TFTrainingArguments

training_args = TFTrainingArguments(
    output_dir='./results', # output directory
    num_train_epochs=3, # total number of training epochs
    per_device_train_batch_size=16, # batch size per device during training
    per_device_eval_batch_size=64, # batch size for evaluation
    warmup_steps=500, # number of warmup steps for learning rate scheduler
    weight_decay=0.01, # strength of weight decay
    logging_dir='./logs', # directory for storing logs
    logging_steps=10,
    eval_steps = 10,
)

with training_args.strategy.scope():
    model = TFDistilBertForSequenceClassification.from_pretrained("distilbert-base-uncased")

trainer = TFTrainer(
    model=model, # the instantiated 🤗 Transformers model to be trained
    args=training_args, # training arguments, defined above
    train_dataset=train_dataset, # training dataset
    eval_dataset=val_dataset,
    compute_metrics=compute_metrics
)

trainer.train()

```