klbr/khoj
1
0
Fork 0
mirror of https://github.com/khoaliber/khoj.git synced 2026-03-02 21:19:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Disable parallel tool call by anthropic models as unsupported

Browse Source 
Khoj doesn't handle parallel tool calling right now. Models were told
to call tools in serial but it wasn't enforced via the Anthropic API.
So if model did try make parallel tool call, next response would fail
as it expects a tool result for the other tool calls. But khoj just
returned the first tool calls results. This mostly affected haiku due
to its lower fine-grained instruction following capabilities.

This changes enforces serial tool calls at the API layer to avoid this
issue altogether for claude models.
This commit is contained in:
Debanjum
2025-12-05 11:52:52 -08:00
parent 4654ac4962
commit 9c03af2735
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/khoj/processor/conversation/anthropic/utils.py
View File

@@ -85,6 +85,8 @@ def anthropic_completion_with_backoff(
# Cache tool definitions # Cache tool definitions
last_tool = model_kwargs["tools"][-1] last_tool = model_kwargs["tools"][-1]
last_tool["cache_control"] = {"type": "ephemeral"} last_tool["cache_control"] = {"type": "ephemeral"}
# Disable parallel tool call until we add support for it
model_kwargs["tool_choice"] = {"type": "auto", "disable_parallel_tool_use": True}
elif response_schema: elif response_schema:
tool = create_tool_definition(response_schema) tool = create_tool_definition(response_schema)
model_kwargs["tools"] = [ model_kwargs["tools"] = [