Tool Calling not working - parameters not set correct

[AladinTheThird]

Hi there,

thanks for providing Smolagents. I wanted to have a first contact with Smolagents. Looks very promising and easy to use, so I gave it a try.
I was picking an given example to get first insights.

Example: tool_calling_agent_from_any_llm.py

• Picking the given example from the examples directory
• Set it up to run with gpt4o on azure
• Run and checked the output --> cool, works

Stepped into the tool call with a breakpoint to check the function parameters
Parameters seems to be wrong

1. Set a breakpoint in get_weather to see the parameters
2. Run the sample in debugger
3. Checking parameters: location = '{"location":"Paris"}', celsius = False

There seems to be no parsing of the parameters upfront the call of the function. I would expect location="Paris"

Modified the example to get a second parameter
print(agent.run("What's the weather like in Paris? Please tell me in Celsius."))

Same test. Checked the values:
location = '{"location":"Paris","celsius":true}'
celsius = False**

Observation
The LLM was able to identify the "location = Paris" and need for "celsius = True". But the tool calling does not work as it should.
There is an issue with parsing the parameters and mapped to the function call.

Am I doing wrong? Would be great if somebody could check that.

Thanks for your support

[touseefahmed96]

@AladinTheThird Can you share your code and also mention where you are setting the breakpoint.

[touseefahmed96]

The code is working as it should

Input:

What is the weather in New York?

output:

 What is the weather in New York?                                                                                                                                                                     
 HfApiModel - Qwen/Qwen2.5-Coder-32B-Instruct 

 Calling tool: 'get_weather' with arguments: {'celsius': False, 'location': 'New York'}                                                                                                               

Observations: The current weather in New York is Clear with a temperature of 12 °F.
[Step 0: Duration 1.47 seconds| Input tokens: 2,154 | Output tokens: 27]

 Calling tool: 'final_answer' with arguments: {'answer': 'The current weather in New York is Clear with a temperature of 12 °F.'}                                                                     

Final answer: The current weather in New York is Clear with a temperature of 12 °F.

2nd Input:

What is the weather in New York?Please tell me in Celsius.

output:

What is the weather in New York?Please tell me in Celsius.                                                                                                                                                                                                                                                                                                                                   
HfApiModel - Qwen/Qwen2.5-Coder-32B-Instruct 


Calling tool: 'get_weather' with arguments: {'celsius': True, 'location': 'New York'}                                                                                                                

Observations: The current weather in New York is Clear with a temperature of -11 °C.
[Step 0: Duration 2.40 seconds| Input tokens: 2,160 | Output tokens: 27]


Calling tool: 'final_answer' with arguments: {'answer': 'The current weather in New York is Clear with a temperature of -11 °C.'}                                                                    
    
Final answer: The current weather in New York is Clear with a temperature of -11 °C.

[AladinTheThird]

The output seems to be ok but only because it is doing nothing with the input values. So the problem gets not obvious.
Set a breakpoint at the return of the get_weather function and take a look at the input parameters. There is json handed over to location and the second parameter is completely ignored.

I modified the function and return a text with F and C depending on the boolean celsius.
You can set a breakpoint at the return weather

@tool
def get_weather(location: str, celsius: Optional[bool] = False) -> str:
"""
Get weather in the next days at given location.
Secretly this tool does not care about the location, it hates the weather everywhere.

Args:
    location: the location
    celsius: the temperature
"""
print("Location: ", location)
print("Celsius: ", celsius)

weather=""
if celsius:
    weather="The weather is UNGODLY with torrential rains and temperatures below -10°C"
else:
    weather="The weather is UNGODLY with torrential rains and temperatures below 14°C"

return weather

Output looks like this - Did some cleanup because format was a bit messy. LLM is also complaining that it got F instead C
╭──────────────────────────── New run ─────────────────────────────╮
│ │
│ What's the weather like in Paris? Please tell me in Celsius. │
│ │
╰─ LiteLLMModel - azure/gpt-4o ───────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ╭──────────────────────────────────────────────────────────────────╮
│ Calling tool: 'get_weather' with arguments: │
│ {"location":"Paris","celsius":true} │
╰──────────────────────────────────────────────────────────────────╯
Location: {"location":"Paris","celsius":true}
Celsius: False
Observations: The weather is UNGODLY with torrential rains and
temperatures below 14°F
[Step 0: Duration 4.83 seconds| Input tokens: 1,037 | Output tokens:
19]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ╭──────────────────────────────────────────────────────────────────╮
│ Calling tool: 'final_answer' with arguments: {"answer":"The │
│ weather in Paris is UNGODLY with torrential rains and │
│ temperatures below 14°F. (Note: This seems to be in Fahrenheit, │
│ despite the request for Celsius)"} │
╰──────────────────────────────────────────────────────────────────╯
Final answer: {"answer":"The weather in Paris is UNGODLY with
torrential rains and temperatures below 14°F. (Note: This seems to
be in Fahrenheit, despite the request for Celsius)"}

[touseefahmed96]

I dont know but this is i am getting

My Tool:

@tool
def get_weather(location: str, celsius: Optional[bool] = False) -> str:
    """
    Get the current weather at the given location using the WeatherStack API.

    Args:
        location: The location (city name).
        celsius: Whether to return the temperature in Celsius (default is False, which returns Fahrenheit).

    Returns:
        A string describing the current weather at the location.
    """
    api_key = "your_api"  # Replace with your API key from https://weatherstack.com/
    units = "m" if celsius else "f"  # 'm' for Celsius, 'f' for Fahrenheit
    print("Location: ", location)
    print("Celsius: ", celsius)
    url = f"http://api.weatherstack.com/current?access_key={api_key}&query={location}&units={units}"

    try:
        response = requests.get(url)
        response.raise_for_status()  # Raise an exception for HTTP errors

        data = response.json()

        if data.get("error"):  # Check if there's an error in the response
            return (
                f"Error: {data['error'].get('info', 'Unable to fetch weather data.')}"
            )

        weather = data["current"]["weather_descriptions"][0]
        temp = data["current"]["temperature"]
        temp_unit = "°C" if celsius else "°F"

        return f"The current weather in {location} is {weather} with a temperature of {temp} {temp_unit}."

    except requests.exceptions.RequestException as e:
        return f"Error fetching weather data: {str(e)}"

╭────────────────────────────────────────────────────────────────────────────────────────────── New run ───────────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                                                                                                                                      │
│ What is the weather in New York?ALso please tell in celsius                                                                                                                                          │
│                                                                                                                                                                                                      │
╰─ HfApiModel - Qwen/Qwen2.5-Coder-32B-Instruct ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'get_weather' with arguments: {'celsius': True, 'location': 'New York'}                                                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Location:  New York
Celsius:  True
Observations: The current weather in New York is Sunny with a temperature of -12 °C.
[Step 0: Duration 2.45 seconds| Input tokens: 2,162 | Output tokens: 28]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'final_answer' with arguments: {'answer': 'The current weather in New York is Sunny with a temperature of -12 °C.'}                                                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Final answer: The current weather in New York is Sunny with a temperature of -12 °C.

[touseefahmed96]

@AladinTheThird
I have added Multi_tool_calling example in this PR #293

Maybe try with that code

[AladinTheThird]

@touseefahmed96
Your print outs look different to mine:

your ouput
Tool call:
│ Calling tool: 'get_weather' with arguments: {'celsius': True, 'location': 'New York'}

Code:
print("Location: ", location)
print("Celsius: ", celsius)

Result:
Location: New York
Celsius: True

my output
Tool Call:
│ Calling tool: 'get_weather' with arguments: {"location":"Paris","celsius":true}
Code:
print("Location: ", location)
print("Celsius: ", celsius)

Result:
Location: {"location":"Paris","celsius":true}
Celsius: False

Observation
The Tool Call is structural the same, the result in calling the function is different. In my case the JSON is not split and placed to the corresponding variables. It is only pushed as plain json to the first variable. In case of Type int as input parameters I expect this will fail completely.

I assume this is not an llm issue because of the Calling Tool line indicates no differences. It seems to be a parsing issue in some way in the code parts.

Your pull request
Like your samples but have not API Key. I would prefer to have samples that run out of the box to provide fast understanding.

[touseefahmed96]

@AladinTheThird you can try some tools from my PR some of them require api but some dont so comment out the tools which requires api and try others.

[touseefahmed96]

One thing also in the fuction of @tool docstrings are very important as it parses the arguments from there maybe try to change the docstring