vllm woes: 'MistralTokenizer' object has no attribute 'init_kwargs'
#15
by
mratsim
- opened
Besides https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506/discussions/7, after copying over files from Mistral 3.1 I now get
AttributeError: 'MistralTokenizer' object has no attribute 'init_kwargs'
This is with mistral_common==1.6.2 and vllm commit 0f9e7354f508af3fe314cfb709babaaa668f1b04 built from source on 2025-06-25.
Tekkenizer v11 is used but it seems like the image handling has a different tokenizer.
Full backtrace:
.2-24B.w4a16-gptq', 'tokenizer_mode': 'mistral', 'trust_remote_code': True, 'max_model_len': 92500, 'served_model_name': ['mistral3.2-24b'], 'generation_config': '/workspace/local_models/Mistral-3.2-24B.w4a16-gptq', 'gpu_memory_utilization': 0.95, 'enable_prefix_caching': True, 'max_num_seqs': 64, 'enable_chunked_prefill': True}
INFO 06-25 15:47:53 [config.py:839] This model supports multiple tasks: {'reward', 'score', 'generate', 'embed', 'classify'}. Defaulting to 'generate'.
INFO 06-25 15:47:53 [config.py:1453] Using max model len 92500
INFO 06-25 15:47:54 [config.py:2197] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 06-25 15:47:56 [__init__.py:244] Automatically detected platform cuda.
INFO 06-25 15:47:57 [core.py:459] Waiting for init message from front-end.
INFO 06-25 15:47:57 [core.py:69] Initializing a V1 LLM engine (v0.9.2.dev247+g0f9e7354f.d20250625) with config: model='/workspace/local_models/Mistral-3.2-24B.w4a16-gptq', speculative_config=None, tokenizer='/workspace/local_models/Mistral-3.2-24B.w4a16-gptq', skip_tokenizer_init=False, tokenizer_mode=mistral, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=92500, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=mistral3.2-24b, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":["none"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":512,"local_cache_dir":null}
WARNING 06-25 15:47:57 [utils.py:2753] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x7fdf7dde3c50>
INFO 06-25 15:47:58 [parallel_state.py:1072] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
ERROR 06-25 15:47:59 [core.py:519] EngineCore failed to start.
ERROR 06-25 15:47:59 [core.py:519] Traceback (most recent call last):
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/inputs/registry.py", line 169, in call_hf_processor
ERROR 06-25 15:47:59 [core.py:519] output = hf_processor(**data, **merged_kwargs, return_tensors="pt")
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/usr/local/lib/python3.12/dist-packages/transformers/models/pixtral/processing_pixtral.py", line 156, in __call__
ERROR 06-25 15:47:59 [core.py:519] tokenizer_init_kwargs=self.tokenizer.init_kwargs,
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] AttributeError: 'MistralTokenizer' object has no attribute 'init_kwargs'
ERROR 06-25 15:47:59 [core.py:519]
ERROR 06-25 15:47:59 [core.py:519] The above exception was the direct cause of the following exception:
ERROR 06-25 15:47:59 [core.py:519]
ERROR 06-25 15:47:59 [core.py:519] Traceback (most recent call last):
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/engine/core.py", line 510, in run_engine_core
ERROR 06-25 15:47:59 [core.py:519] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/engine/core.py", line 394, in __init__
ERROR 06-25 15:47:59 [core.py:519] super().__init__(vllm_config, executor_class, log_stats,
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/engine/core.py", line 75, in __init__
ERROR 06-25 15:47:59 [core.py:519] self.model_executor = executor_class(vllm_config)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/executor/executor_base.py", line 53, in __init__
ERROR 06-25 15:47:59 [core.py:519] self._init_executor()
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/executor/uniproc_executor.py", line 47, in _init_executor
ERROR 06-25 15:47:59 [core.py:519] self.collective_rpc("init_device")
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 06-25 15:47:59 [core.py:519] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/utils.py", line 2687, in run_method
ERROR 06-25 15:47:59 [core.py:519] return func(*args, **kwargs)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/worker/worker_base.py", line 606, in init_device
ERROR 06-25 15:47:59 [core.py:519] self.worker.init_device() # type: ignore
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/worker/gpu_worker.py", line 165, in init_device
ERROR 06-25 15:47:59 [core.py:519] self.model_runner: GPUModelRunner = GPUModelRunner(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/worker/gpu_model_runner.py", line 142, in __init__
ERROR 06-25 15:47:59 [core.py:519] encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/core/encoder_cache_manager.py", line 199, in compute_encoder_budget
ERROR 06-25 15:47:59 [core.py:519] ) = _compute_encoder_budget_multimodal(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/v1/core/encoder_cache_manager.py", line 229, in _compute_encoder_budget_multimodal
ERROR 06-25 15:47:59 [core.py:519] .get_max_tokens_per_item_by_nonzero_modality(model_config)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/registry.py", line 158, in get_max_tokens_per_item_by_nonzero_modality
ERROR 06-25 15:47:59 [core.py:519] self.get_max_tokens_per_item_by_modality(model_config).items()
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/registry.py", line 132, in get_max_tokens_per_item_by_modality
ERROR 06-25 15:47:59 [core.py:519] return profiler.get_mm_max_tokens(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/profiling.py", line 277, in get_mm_max_tokens
ERROR 06-25 15:47:59 [core.py:519] mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/profiling.py", line 169, in _get_dummy_mm_inputs
ERROR 06-25 15:47:59 [core.py:519] return self.processor.apply(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/processing.py", line 1808, in apply
ERROR 06-25 15:47:59 [core.py:519] ) = self._cached_apply_hf_processor(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/processing.py", line 1574, in _cached_apply_hf_processor
ERROR 06-25 15:47:59 [core.py:519] ) = self._apply_hf_processor_main(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/processing.py", line 1417, in _apply_hf_processor_main
ERROR 06-25 15:47:59 [core.py:519] prompt_ids = self._apply_hf_processor_text_only(prompt)
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/processing.py", line 1342, in _apply_hf_processor_text_only
ERROR 06-25 15:47:59 [core.py:519] prompt_ids, _, _ = self._apply_hf_processor_text_mm(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/processing.py", line 1312, in _apply_hf_processor_text_mm
ERROR 06-25 15:47:59 [core.py:519] processed_data = self._call_hf_processor(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/model_executor/models/mistral3.py", line 232, in _call_hf_processor
ERROR 06-25 15:47:59 [core.py:519] processed_outputs = super()._call_hf_processor(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/multimodal/processing.py", line 1275, in _call_hf_processor
ERROR 06-25 15:47:59 [core.py:519] return self.info.ctx.call_hf_processor(
ERROR 06-25 15:47:59 [core.py:519] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 06-25 15:47:59 [core.py:519] File "/workspace/vllm/vllm/inputs/registry.py", line 187, in call_hf_processor
ERROR 06-25 15:47:59 [core.py:519] raise ValueError(msg) from exc
ERROR 06-25 15:47:59 [core.py:519] ValueError: Failed to apply PixtralProcessor on data={'text': '[IMG]'} with kwargs={}
Process EngineCore_0:
Traceback (most recent call last):
File "/workspace/vllm/vllm/inputs/registry.py", line 169, in call_hf_processor
output = hf_processor(**data, **merged_kwargs, return_tensors="pt")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/pixtral/processing_pixtral.py", line 156, in __call__
tokenizer_init_kwargs=self.tokenizer.init_kwargs,
^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MistralTokenizer' object has no attribute 'init_kwargs'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/workspace/vllm/vllm/v1/engine/core.py", line 523, in run_engine_core
raise e
File "/workspace/vllm/vllm/v1/engine/core.py", line 510, in run_engine_core
engine_core = EngineCoreProc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/engine/core.py", line 394, in __init__
super().__init__(vllm_config, executor_class, log_stats,
File "/workspace/vllm/vllm/v1/engine/core.py", line 75, in __init__
self.model_executor = executor_class(vllm_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/executor/executor_base.py", line 53, in __init__
self._init_executor()
File "/workspace/vllm/vllm/executor/uniproc_executor.py", line 47, in _init_executor
self.collective_rpc("init_device")
File "/workspace/vllm/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
answer = run_method(self.driver_worker, method, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/utils.py", line 2687, in run_method
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/worker/worker_base.py", line 606, in init_device
self.worker.init_device() # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/worker/gpu_worker.py", line 165, in init_device
self.model_runner: GPUModelRunner = GPUModelRunner(
^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/worker/gpu_model_runner.py", line 142, in __init__
encoder_compute_budget, encoder_cache_size = compute_encoder_budget(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/core/encoder_cache_manager.py", line 199, in compute_encoder_budget
) = _compute_encoder_budget_multimodal(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/core/encoder_cache_manager.py", line 229, in _compute_encoder_budget_multimodal
.get_max_tokens_per_item_by_nonzero_modality(model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/registry.py", line 158, in get_max_tokens_per_item_by_nonzero_modality
self.get_max_tokens_per_item_by_modality(model_config).items()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/registry.py", line 132, in get_max_tokens_per_item_by_modality
return profiler.get_mm_max_tokens(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/profiling.py", line 277, in get_mm_max_tokens
mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/profiling.py", line 169, in _get_dummy_mm_inputs
return self.processor.apply(
^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/processing.py", line 1808, in apply
) = self._cached_apply_hf_processor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/processing.py", line 1574, in _cached_apply_hf_processor
) = self._apply_hf_processor_main(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/processing.py", line 1417, in _apply_hf_processor_main
prompt_ids = self._apply_hf_processor_text_only(prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/processing.py", line 1342, in _apply_hf_processor_text_only
prompt_ids, _, _ = self._apply_hf_processor_text_mm(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/processing.py", line 1312, in _apply_hf_processor_text_mm
processed_data = self._call_hf_processor(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/model_executor/models/mistral3.py", line 232, in _call_hf_processor
processed_outputs = super()._call_hf_processor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/multimodal/processing.py", line 1275, in _call_hf_processor
return self.info.ctx.call_hf_processor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/inputs/registry.py", line 187, in call_hf_processor
raise ValueError(msg) from exc
ValueError: Failed to apply PixtralProcessor on data={'text': '[IMG]'} with kwargs={}
[rank0]:[W625 15:47:59.697775105 ProcessGroupNCCL.cpp:1476] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Traceback (most recent call last):
File "/usr/local/bin/vllm", line 10, in <module>
sys.exit(main())
^^^^^^
File "/workspace/vllm/vllm/entrypoints/cli/main.py", line 65, in main
args.dispatch_function(args)
File "/workspace/vllm/vllm/entrypoints/cli/serve.py", line 55, in cmd
uvloop.run(run_server(args))
File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 109, in run
return __asyncio.run(
^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1325, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1345, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 155, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 191, in build_async_engine_client_from_engine_args
async_llm = AsyncLLM.from_vllm_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/engine/async_llm.py", line 162, in from_vllm_config
return cls(
^^^^
File "/workspace/vllm/vllm/v1/engine/async_llm.py", line 124, in __init__
self.engine_core = EngineCoreClient.make_async_mp_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/engine/core_client.py", line 93, in make_async_mp_client
return AsyncMPClient(vllm_config, executor_class, log_stats,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/vllm/vllm/v1/engine/core_client.py", line 735, in __init__
super().__init__(
File "/workspace/vllm/vllm/v1/engine/core_client.py", line 433, in __init__
self._init_engines_direct(vllm_config, local_only,
File "/workspace/vllm/vllm/v1/engine/core_client.py", line 502, in _init_engines_direct
self._wait_for_engine_startup(handshake_socket, input_address,
File "/workspace/vllm/vllm/v1/engine/core_client.py", line 522, in _wait_for_engine_startup
wait_for_engine_startup(
File "/workspace/vllm/vllm/v1/utils.py", line 494, in wait_for_engine_startup
raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
I can provide my gptq quantized model if it helps but it's the same recipe as my working devstral from a month ago: https://huggingface.co/mratsim/Devstral-Small-2505.w4a16-gptq