Gpt-oss training on A100 - OOM error

clementlr
Thu, 20 Nov 2025 21:41:26 +0000

DhineshR:

convert_to_fp32

Dear DhineshR, thanks for your reply.

The convert_to_fp32 you mentioned is part of error message but I did not set such parameter.

The torch dtype of the model config is the one recommended in openAI cookbook:

import torch

from transformers import AutoModelForCausalLM, Mxfp4Config

quantization_config = Mxfp4Config(dequantize=True)

model_kwargs = dict(

attn_implementation=“eager”,

torch_dtype=torch.bfloat16,

quantization_config=quantization_config,

use_cache=False,

device_map=“auto”,

)

I appreciate your feedback.

model = AutoModelForCausalLM.from_pretrained(“openai/gpt-oss-20b”, **model_kwargs)