whisper-small-GL-EN

This model is a fine-tuned version of openai/whisper-small on juanjucm/FLEURS-SpeechT-GL-EN. The training dataset has been augmented using train split from juanjucm/OpenHQ-SpeechT-GL-EN

It achieves the following results on the evaluation set (evaluated only on juanjucm/FLEURS-SpeechT-GL-EN):

• Loss: 1.6335
• Wer: 67.2612
• Bleu: 22.2158

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.25e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 32
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Bleu
0.6816	1.0	236	1.6335	67.2612	22.2158
0.1904	2.0	472	1.7234	69.9647	21.0583
0.2177	3.0	708	1.8764	73.2720	19.0086
0.0334	4.0	944	2.0541	72.6774	19.7679
0.0129	5.0	1180	2.1722	70.6708	19.8076
0.011	6.0	1416	2.2637	71.2653	19.7416
0.0062	7.0	1652	2.3214	70.3920	20.3474
0.0067	8.0	1888	2.3405	71.9621	20.1999

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0