# Why does the Bert classification head matrix has such dimension?

**URL:** <https://discuss.huggingface.co/t/why-does-the-bert-classification-head-matrix-has-such-dimension/115812>\
**Category:** Models\
**Created:** [November 5, 2024, 10:54am UTC](https://discuss.huggingface.co/t/why-does-the-bert-classification-head-matrix-has-such-dimension/115812 "2024-11-05T10:54:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![EquinoxElahin](https://avatars.discourse-cdn.com/v4/letter/e/eb9ed0/32.png) [@EquinoxElahin](https://discuss.huggingface.co/u/EquinoxElahin)\
**Post date:** [November 5, 2024, 10:54am UTC](https://discuss.huggingface.co/t/why-does-the-bert-classification-head-matrix-has-such-dimension/115812/1 "2024-11-05T10:54:10Z")

</div>

Hi everyone,  
I downloaded the NLI Bert model : morit/french\_xlm\_xnli

Looking into the classification head. Here is the str representation :

XLMRobertaClassificationHead(  
(dense): Linear(in\_features=768, out\_features=768, bias=True)  
(dropout): Dropout(p=0.1, inplace=False)  
(out\_proj): Linear(in\_features=768, out\_features=3, bias=True)  
)

OK seems nice to me, bert’s internal dimension 768 and I m looking for 3 out features because of NLI (-1,0, 1 basically).  
Exporting the out\_proj matrices I was expecting its shape being (768, 3) But I got the transpose shape (3, 768) and this is I really don’t understand.  
Can someone enlight me ?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [November 6, 2024, 11:54am UTC](https://discuss.huggingface.co/t/why-does-the-bert-classification-head-matrix-has-such-dimension/115812/2 "2024-11-06T11:54:44Z")

</div>

I realized that I didn’t understand.

> <https://stackoverflow.com/questions/65797330/how-does-the-bert-tokenizer-result-in-an-input-tensor-shape-of-b-24-768>
