Devstral with vLLM Container (v0.9.1 and v0.9.2)

by ReiFr1z - opened Jul 14

Jul 14

Container stops with:

ValueError: Unknown version: v13 in /root/.cache/huggingface/hub/models--mistralai--Devstral-Small-2507/snapshots/3178e9e2d8880880098af656c1fe223927ce74f8/tekken.json. Make sure to use a valid version string: ['v1', 'v2', 'v3', 'v7', 'v11']

An Issue at vLLM is opend #20902

kyrie666

Jul 14

This comment has been hidden (marked as Resolved)

kyrie666

Jul 14

Container stops with:

ValueError: Unknown version: v13 in /root/.cache/huggingface/hub/models--mistralai--Devstral-Small-2507/snapshots/3178e9e2d8880880098af656c1fe223927ce74f8/tekken.json. Make sure to use a valid version string: ['v1', 'v2', 'v3', 'v7', 'v11']

An Issue at vLLM is opend #20902
update mistral-common >=1.7.0

ReiFr1z

Jul 14

An Issue at vLLM is opend #20902
update mistral-common >=1.7.0

Thanks, but mistral_common==1.7.0 is installed

koiker

Jul 14

I got the same error intially. But after running pip install mistral-common --upgrade it fixed and the model loaded correctly in vllm 0.9.2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment