I’m attempting to use function-calling capabilities with text inference models, but none of the ways I’ve attempted to do with the inference clients work. I use OpenAI’s function calling regularly, but Hugging Face capabilities seem fragile and unpredictable, but I suspect I’m just “doing it wrong.” …

[image] Remote: Repository not found Beginners Hey @aqpeeb ! I think you should check your authentication with huggingface. Authenticate with Hugging Face: If authentication is required, make sure you’re logged in to Hugging Face via the CLI: bash CopyEdit huggingface…

@Alanturner2 I don’t think it’s a token or authentication problem for this function calling. Was that comment intended to help here as well? I have a token with the proper permissions and all that needs to happen to reproduce behavior is setting the HF_TOKEN variable.

I tried it twice with Qwen and twice with Zephyr. I wonder what the expected behavior is… ChatCompletionOutput(choices=[ChatCompletionOutputComplete(finish_reason='stop', index=0, message=ChatCompletionOutputMessage(role='assistant', content=None, tool_calls=[ChatCompletionOutputToolCall(function=C…

Glad to see someone can reproduce it! With OpenAI at least, the expected behavior is to get an “assistant” message with a text content that presents a human-readable response, like “The temperature is 4ºC.” Thanks for the GitHub link; I’ll look through it. I opened a similar bug in the inference cl…

Even within the InferenceClient class, chat_completion is quite special because it is modeled on the OpenAI API. For now, the return value needs to be plain like this. output = client.chat_completion(messages=messages, tools=tools, max_tokens=500, temperature=0.3) print(output.choices[0].message.to…

Hmm, very interesting. So one omits the “tools=tools” once manually determining the calls are satisfied… I’m quite familiar with the OpenAI API, and there are some quirks in HF that I’m obviously stumbling through. For example, in OpenAI, the tools array can always be provided, in case subsequent f…

Sorry, I was wrong. I think the Inference API is buggy. The function is defined correctly. When that happens, None is returned. Edit: I reported the symptoms on Discord.

Is there any update on this issue, I am facing the same issue and trying to find a walkaround.

So far, there’s been no progress.

I’ve had some acknowledgement on the GitHub issue I filed . I’ve tried a workaround that entails omitting the “tools” argument on subsequent instantiations of the chat completion object, and it seems to work for now, at least for simple tool situations. Vision models also fail in weird ways, when s…

Function calling not working with inference clients on (seemingly) any model

Beginners

Alanturner2 February 3, 2025, 7:17am 4

Topic		Replies	Views
Tool calling gets stuck in an infinite loop Inference Endpoints on the Hub	2	712	April 12, 2025
Dumb Question: Seeing that my inference API links not working Beginners	1	214	July 10, 2025
Function/tool calling using Transformer models 🤗Transformers	5	1901	July 17, 2025
Unable to access model with Inference Client Beginners	2	215	May 6, 2025
Cannot access the Inference Provider's service Beginners	1	237	November 21, 2025

Function calling not working with inference clients on (seemingly) any model

Related topics