[Bug?] google/gemma-3n-E4B-it fails with AttributeError: 'Gemma3nConfig' object has no attribute 'vocab_size'
Environment
- vLLM version: 0.9.1
- Transformers version: 4.41.2
- CUDA / GPU: RTX 4080 (driver 550.xx)
Full Traceback
Notes
- The model works in
transformersdirectly. - I suspect the
config.jsonof the model lacksvocab_size, which vLLM expects. - Should we patch the model config or wait for vLLM to adapt?
Hi @EsatKara ,
Gemma-3n-E4Bmodels requires Transformers version4.53.0. Please update your Transformers library to at least version4.53.0to useGemma-3n-E4Band let us know if you still face the same issue. Thank you
I upgraded Transformers to 4.53.0 with vllm of 0.9.1, but encountered the same issue
Hi @EsatKara I’m using the same environment as you. I suspect the root cause is that the config.json does not explicitly define top-level keys like vocab_size, num_hidden_layers, num_attention_heads, etc. Instead, these values are nested under the "audio_config" field, like so:
json
"audio_config": {
"vocab_size": 128,
"hidden_size": 1536,
"conf_num_attention_heads": 8,
"conf_num_hidden_layers": 12,
...
}
In my case, I manually copied relevant values (like vocab_size, num_attention_heads, num_hidden_layers, etc.) from audio_config and added them as top-level fields in config.json. After that, VLLM was able to launch successfully.
However, I’m not sure if this workaround would affect model performance or correctness — since this is not an officially supported or standard method. I also suspect this issue could be related to VLLM not fully supporting models with nested configs, or possibly a version mismatch.
@EsatKara vllm 0.9.1 does not support gemma 3n yet, see https://github.com/vllm-project/vllm/pull/20134 the PR only got merged 4 days ago. You need to wait for new release or build from source.