Explicitly drop stream_options if not streaming openai chat response

Not sure why but it some cases when interacting with o3 (which needs
non-streaming) the stream_options seems to be set.

Cannot reproduce but hopefully dropping the stream_options explicitly
should resolve this issue.

Related 985a98214
This commit is contained in:
Debanjum
2025-06-11 12:39:39 -07:00
parent 65644f78b0
commit 71763684a9

View File

@@ -201,6 +201,8 @@ async def chat_completion_with_backoff(
stream_processor = adefault_stream_processor
if stream:
model_kwargs["stream_options"] = {"include_usage": True}
else:
model_kwargs.pop("stream_options", None)
formatted_messages = format_message_for_api(messages, api_base_url)