Gpt-oss training on A100 - OOM error

clementlr
Fri, 21 Nov 2025 13:43:03 +0000

Dear DhineshR, thanks for your feedback.

I try downsizing:

per_device_train_batch_size=1, gradient_accumulation_steps=1,

But still results in OOM error message.

I also tried to:

model_kwargs = dict(

attn_implementation=“eager”,

torch_dtype=torch.bfloat16,

quantization_config=quantization_config,

use_cache=False,

#device_map=“auto”,

)

as I noticed warning info about model beeing on multiple device (although runnning on A100 single GPU) after:

trainer = SFTTrainer(

… model=peft_model,

… args=training_args,

… train_dataset=dataset,

… processing_class=tokenizer,

… )

The model is already on multiple devices. Skipping the move to device specified in `args`.

But still result in OOM error.

Do you have other thought ?

Right now I am considering testing on H100 but not sure this will make it as VRAM is equal to 80 GB as well.

Kind regards