life2lang-base-pt

This model is a fine-tuned version of khairi/life2lang-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2901

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 192
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10000
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
7.2108	0.0937	500	6.1957
4.4145	0.1874	1000	3.9154
3.8566	0.2811	1500	3.3855
3.4984	0.3749	2000	3.1536
3.3061	0.4686	2500	2.9938
3.0382	0.5623	3000	2.7199
2.8886	0.6560	3500	2.5770
2.7820	0.7497	4000	2.5104
2.7074	0.8434	4500	2.4672
2.6533	0.9372	5000	2.4341
2.6171	1.0307	5500	2.4071
2.5826	1.1245	6000	2.3984
2.5621	1.2182	6500	2.3673
2.5351	1.3119	7000	2.3465
2.5119	1.4056	7500	2.3516
2.4895	1.4993	8000	2.3331
2.4795	1.5930	8500	2.3174
2.4614	1.6868	9000	2.3124
2.4483	1.7805	9500	2.3003
2.4242	1.8742	10000	2.2985
2.4176	1.9679	10500	2.2778
2.4032	2.0615	11000	2.2669
2.3925	2.1552	11500	2.2666
2.3812	2.2489	12000	2.2743
2.3751	2.3426	12500	2.3001
2.3704	2.4363	13000	2.2653
2.3583	2.5301	13500	2.3094
2.3566	2.6238	14000	2.2848
2.3568	2.7175	14500	2.2812
2.3551	2.8112	15000	2.2754
2.3487	2.9049	15500	2.2819
2.3496	2.9986	16000	2.2872
2.3496	3.0	16008	2.2901

Framework versions

• Transformers 5.8.1
• Pytorch 2.11.0+cu128
• Datasets 3.1.0
• Tokenizers 0.22.2