course documentation
Part 1 completed!
0. Setup
1. Transformer models
IntroductionNatural Language Processing and Large Language ModelsTransformers, what can they do?How do Transformers work?How 🤗 Transformers solve tasksTransformer ArchitecturesQuick quizInference with LLMsBias and limitationsSummaryCertification exam
2. Using 🤗 Transformers
IntroductionBehind the pipelineModelsTokenizersHandling multiple sequencesPutting it all togetherBasic usage completed!Optimized Inference DeploymentEnd-of-chapter quiz
3. Fine-tuning a pretrained model
IntroductionProcessing the dataFine-tuning a model with the Trainer APIA full training loopUnderstanding Learning CurvesFine-tuning, Check!End-of-chapter quiz
4. Sharing models and tokenizers
The Hugging Face HubUsing pretrained modelsSharing pretrained modelsBuilding a model cardPart 1 completed!End-of-chapter quiz
5. The 🤗 Datasets library
IntroductionWhat if my dataset isn't on the Hub?Time to slice and diceBig data? 🤗 Datasets to the rescue!Creating your own datasetSemantic search with FAISS🤗 Datasets, check!End-of-chapter quiz
6. The 🤗 Tokenizers library
IntroductionTraining a new tokenizer from an old oneFast tokenizers' special powersFast tokenizers in the QA pipelineNormalization and pre-tokenizationByte-Pair Encoding tokenizationWordPiece tokenizationUnigram tokenizationBuilding a tokenizer, block by blockTokenizers, check!End-of-chapter quiz
7. Classical NLP tasks
IntroductionToken classificationFine-tuning a masked language modelTranslationSummarizationTraining a causal language model from scratchQuestion answeringMastering LLMsEnd-of-chapter quiz
8. How to ask for help
IntroductionWhat to do when you get an errorAsking for help on the forumsDebugging the training pipelineHow to write a good issuePart 2 completed!End-of-chapter quiz
9. Building and sharing demos
Introduction to GradioBuilding your first demoUnderstanding the Interface classSharing demos with othersIntegrations with the Hugging Face HubAdvanced Interface featuresIntroduction to BlocksGradio, check!End-of-chapter quiz
10. Curate high-quality datasets
Introduction to ArgillaSet up your Argilla instanceLoad your dataset to ArgillaAnnotate your datasetUse your annotated datasetArgilla, check!End-of-chapter quiz
11. Fine-tune Large Language Models
IntroductionChat TemplatesFine-Tuning with SFTTrainerLoRA (Low-Rank Adaptation)EvaluationConclusionExam Time!
12. Build Reasoning Models new
IntroductionReinforcement Learning on LLMsThe Aha Moment in the DeepSeek R1 PaperAdvanced Understanding of GRPO in DeepSeekMathImplementing GRPO in TRLPractical Exercise to Fine-tune a model with GRPOPractical Exercise with UnslothComing soon...
Course Events
Part 1 completed!
This is the end of the first part of the course! Part 2 will be released on November 15th with a big community event, see more information here.
You should now be able to fine-tune a pretrained model on a text classification problem (single or pairs of sentences) and upload the result to the Model Hub. To make sure you mastered this first section, you should do exactly that on a problem that interests you (and not necessarily in English if you speak another language)! You can find help in the Hugging Face forums and share your project in this topic once you’re finished.
We can’t wait to see what you will build with this!
Update on GitHub