Use this topic to ask your questions to Jay Alammar during his talk: A gentle visual intro to Transformers models. You can watch it on YouTube or on Twitch at 8:45am PST

Can we get the URL Jay was sharing at the start?

Here is the link from Jay’s talk in the beginning: jalammar.github.io/Simple_Transformer_Language_Model.ipynb at master · jalammar/jalammar.github.io · GitHub

Do you see a future in symbolic learning rather than probabilistic approaches?

Hi, Will these great video presentations be shared offline? So that one can watch later?

Hi, Will these great video presentations be shared offline? So that one can watch later? The live stream can be viewed at any time on YouTube, and we will also edit to share each talk in a separate YouTube video :slight_smile:

That’s great. Thank you so much.

For T5, about 3 approaches have been used to fine-tune the pre-trained model for all the downstream tasks (1- fine-tuning all the pretrained layers (all params), 2- freeze all the pre-trained layers and adding adapter layers, and 3- gradual unfreezing the pre-trained layers over time (and it is clea…

How do we know how much we should fine-tune a pre-trained model? Can you please share some strategies to produce a good fine-tuned pre-trained model (while incorporating that there would be some bias as well originating from the pre-trained model)?

[image] Abirate: For T5, about 3 approaches have been used to fine-tune the pre-trained model for all the downstream tasks (1- fine-tuning all the pretrained layers (all params), 2- freeze all the pre-trained layers and adding adapter layers, and 3- gradual unfreezing the pre-trained layers over…

[image] rtrimana: How do we know how much we should fine-tune a pre-trained model? Can you please share some strategies to produce a good fine-tuned pre-trained model (while incorporating that there would be some bias as well originating from the pre-trained model)? This is answered at 1:11:4…

[Nov 15th Event] Jay Alammar: A gentle visual intro to Transformers models

Course

sgugger November 15, 2021, 5:37pm 13

Could you move this question to the right topic ? Thanks in advance

Topic		Replies	Views
Example of how to pretrain T5? 🤗Transformers	15	16404	March 16, 2023
Unfreeze BERT vs pre-train BERT for Sentiment Analysis Beginners	2	1503	December 24, 2021
Do you train all layers when fine-tuning T5? Beginners	7	7169	September 26, 2023
Fine tuning of Bert model using tensorflow 2.* 🤗Transformers	1	491	May 29, 2022
How could different model be trained same way? Beginners	0	224	July 26, 2021

[Nov 15th Event] Jay Alammar: A gentle visual intro to Transformers models

Related topics