Hugging Face's logo

openai
/
gpt-oss-20b

Text Generation
Transformers
Safetensors
gpt_oss
vllm
conversational
8-bit precision
mxfp4
Model card Files
xet
Community
111

NVIDIA L40S GPU's for MXFP4 quantization

#100
by lordim - opened
Discussion
lordim

Are NVIDIA L40S GPU's also compatible for MXFP4 quantization? I'm trying to load gpt-oss-20b on this machine, but it seems to default to bf16.

RichardDeetlefs

Did it work with bf16?

entfane

@lordim MXFP4 is supported by Blackwell architecture, so I don't think it is compatible. At least not natively.

Sign up or log in to comment