code in model card return error
i upgraded to latest version transformers
but when i run model card code i get this
$ python qwen3-32b-awq.py
Traceback (most recent call last):
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1967, in _get_module
return importlib.import_module("." + module_name, self.name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "", line 995, in exec_module
File "", line 488, in _call_with_frames_removed
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 41, in
from ...modeling_utils import ALL_ATTENTION_FUNCTIONS, PreTrainedModel
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/modeling_utils.py", line 53, in
from torchao.quantization import Int4WeightOnlyConfig
ImportError: cannot import name 'Int4WeightOnlyConfig' from 'torchao.quantization' (/home/user/miniconda3/lib/python3.12/site-packages/torchao/quantization/init.py). Did you mean: 'Int4WeightOnlyQuantizer'?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/Downloads/qwen3-32b-awq.py", line 7, in
model = AutoModelForCausalLM.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 568, in from_pretrained
model_class = _get_model_class(config, cls._model_mapping)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 388, in _get_model_class
supported_models = model_mapping[type(config)]
~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 770, in getitem
return self._load_attr_from_module(model_type, model_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 784, in _load_attr_from_module
return getattribute_from_module(self._modules[module_name], attr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 700, in getattribute_from_module
if hasattr(module, attr):
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1955, in getattr
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 1969, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.models.qwen3.modeling_qwen3 because of the following error (look up to see its traceback):
cannot import name 'Int4WeightOnlyConfig' from 'torchao.quantization' (/home/user/miniconda3/lib/python3.12/site-packages/torchao/quantization/init.py)