Modelfile doesn't enable tool calling?
Eric,
I'm attempting to use Dolphin3.0-R1-Mistral-24B:Q8_0 via ollama v0.6.8 from Zed v0.185.13. Why? Agentic Editing (watch the video 😎)
However, Zed reports that the model doesn't support tool calling ("Tools Unsupported"). It determines this by sending a POST to /api/show ollama openai endpoint. It then searches capabilities from the returned JSON, looking for 'tool'. I confirmed via curl that 'tool' is not listed under capabilities.
Ollama code is much harder to decipher. :-/. It seems I need to add tool calling to the template in the Modelfile. I found where the Hermes dataset you used defines the tool calling prompt, but it declares the tools in the system prompt. I pulled https://ollama.com/thewindmom/hermes-3-llama-3.1-8b, but its Modelfile doesn't include tool calling, and the example in its README lists the tools in the system prompt...
So, do I understand this wrong? I thought the client was supposed to pass a list of available tools to the model and the model would return which tools to call with which arguments. But the configurations I'm finding aren't lining up with that idea?
I'd much prefer to submit a PR, but I don't even think I understand the intent enough to propose a solution that's in the ball field
You lost me at ollama
Here's the thing
Ollama has their own scheme for chat templates that's different than the hugging face chat template
So anytime you use ollama you are trusting in whoever published that model to ollama, to have correctly implement the ollama chat template for the model.
You are also trusting that nothing has changed in the models chat template since the time it was published to ollama
To help you out: if you wanna effectively use ollama you simply need to become comfortable with manipulating Modelfiles and ollama's special chat template.
I could republish all the dolphin models with correct chat template - and I will - when I have 3 hours. In the mean time you will need to update the chat template yourself. (Hint: use qwen2.5-coder's template)