Is last_hidden_state the output of Encoder block?

MahdiA · December 23, 2021, 4:49pm

When we use BertModel.forward() , is the last_hidden_state the output of Encoder in Transformers block?

nielsr · December 23, 2021, 5:47pm

Yes! It’s a tensor of shape (batch_size, seq_len, hidden_size).

Topic		Replies	Views
Question about last_hidden_state of the bert model Beginners	0	373	December 7, 2023
BERT: What is the shape of each Transformer Encoder block in the final hidden state? Intermediate	7	13260	March 16, 2022
Roberta hidden_states[0] == Bert pooler_output? 🤗Transformers	0	573	July 25, 2022
Last layer hidden state: GPT2 🤗Transformers	0	1987	March 23, 2021
How to add encoder's last hidden state to GPT2 as encoder-decoder attention Beginners	0	388	January 31, 2023