tool calling not working as expected?
hi @all ,
I tried to use gpt-oss-20b with llama.cpp (converted to GGUF) using tool calls.
At offering a few functions to call the model always ends up with a reasoning_content mentioning what to use in a floating text but with no execution of the function itself or at least returning a tool_call structure providing the function (and parameters if needed) as e.g. command-r does.
'choices': [{'finish_reason': 'stop',
'index': 0,
'message': {'content': '',
'reasoning_content': 'User asks: "wie ist '
'das wetter in der '
'pariser region '
'cergy". Need to use '
'get_weather function. '
'Provide city and '
'region. City: Cergy. '
'Region: Paris. '
'Probably "Pariser '
'Region Cergy" -> '
'city: Cergy, region: '
'Paris. Use '
'function.<|start|>assistant<|channel|>commentary '
'to=functions.get_weather '
'json<|message|>{"city":"Cergy","region":"Paris"}',
'role': 'assistant'}}]
This is a bit unhandy to parse and not reliable as it's not always the same structure. Did I forget anything to get a usable result?
Thanks in advance for any help! :-)
Try ollama once.
@xbruce22 : Sorry, it's not my intention to be impolite, but if sth. doesn't work on my Linux machines I'm also not switching to windows as I'm interested in solutions, learning and making things better in the future. But not in throwing things away / replacing things at any tiny mistake. Btw.: ollama is based on llama.cpp
it just got released, I see constant fixes in llama.cpp or ollama. You re-build and it starts working. Thats why I asked you to try ollama. If you are interested in solution, raise issue, start looking into llama.cpp's code base etc. are ways to get started.
Much appreciated. Really. But couldn't it just be that reasoning models have just to be handled in a different way (I maybe don't know about and/or didn't know what to search for) than non reasoning ones?
Before using the sledge hammer (opening issues and digging into the code base) I want to find out if I'm doing everything correctly from my side... :-)
The result from my initial comment above was resulting from a simple request (e.g.: role: user, content: asking about the temperature in Paris, Cergy) and a default test collection of function I'm offering the model to choose from:
[{'function': {'description': 'gibt das aktuelle datum aus',
'name': 'get_date',
'parameters': {}},
'type': 'function'},
{'function': {'description': 'gibt die aktuelle uhrzeit aus',
'name': 'get_time',
'parameters': {}},
'type': 'function'},
{'function': {'description': 'gibt das aktuelle wetter für eine stadt aus',
'name': 'get_weather',
'parameters': {'properties': {'city': {'description': 'die '
'stadt '
'für die '
'die '
'temperatur '
'wiedergegeben '
'werden '
'soll',
'type': 'string'},
'region': {'description': 'die '
'region '
'für '
'die '
'die '
'temperatur '
'wiedergegeben '
'werden '
'soll',
'type': 'string'}},
'required': ['city', 'region'],
'type': 'object'}},
'type': 'function'}]
(Btw: are there any options to format code in the comments? The left alignment is really horrible to read... ^^)
The gpt-oss models do not properly call tools for me either. I've tried both 20b and 120b with a variety of quants. Qwen3-4b-instruct-2507 is more useful with tools than the gpt-oss-120b model
ex:
<think>
We should call the curl function with appropriate URL and request method.
We'll provide arguments array: ["GET", "
https://api.coingecko.com/v3/simple_price?ids=goog
"].
We need to consider that maybe token is not recognized. But we can attempt.
Let's proceed.
<think/>
Below is the curl command:
{
args: [
"GET",
"
https://api.coingecko.com/v3/simple_price?ids=goog
"
]
}The result returned from the curl function. This should contain the current price of GOOG in USD, as a JSON response like:
{
priceUsd: 0.12345 // Example value
}
``q
Just plain text, it does not actually invoke the tool. It also thought it should check a random crypto site for the google stock price. Overall these models aren't very useful in my testing. Unusable as an agent.