nvidia

/

Mistral-NeMo-12B-Instruct

like 158

Follow

NVIDIA 30k

Previously by this command, we could directly serve this model :

vllm serve “nvidia/Mistral-NeMo-12B-Instruct” --dtype auto --api-key token-abc123 --max-model-len 8096

But now it throws this error following error.

ValueError: Invalid repository ID or local directory specified: '“nvidia/Mistral-NeMo-12B-Instruct”'.
Please verify the following requirements:
1. Provide a valid Hugging Face repository ID.
2. Specify a local directory that contains a recognized configuration file.
   - For Hugging Face models: ensure the presence of a 'config.json'.
   - For Mistral models: ensure the presence of a 'params.json'.

And yes, there is no config file in the repo.