It seems to me that it would be useful to be able to be explicit about
where the embedded database should live - as well as where it _does_
live (via the info log), when not specifying.
Extract questions has chat history in prompt and in actual chat history.
Only pass in prompt for now. Later update prompts to pass chat history
in chat messages list for better truncation flexibility.
1. Due to the interaction of two changes:
- dedupe by corpus_id, where corpus_id tracks logical content blocks
like files, org/md headings.
- return compiled, not logical blocks, where compiled track smaller
content chunks that fit within search model, llm context windows.
When combined they showed only 1 hit compiled chunk per logical
block. Even if multiple chunks match within a logical content block.
Fix is to either dedupe by compiled text or to return deduped
logical content blocks (by corpus_id) corresponding to matched
compiled chunks. This commit fixes it by the first method.
2. Due to inferred query, search results zip which resulted in a
single search result being returned per query!
This silently cut down matching search results and went undetected.
The tailwing theme spacing of the scroll area surrounding chat history
on large screens was what was causing the large gap between chat input
box and chat history on some screen layouts.
This change reduces the spacing to a more acceptable level.
Previously summarizedResult would be unset when a tool call failed.
This caused research to fail due to ChatMessageModel failures when
constructing tool chat histories and would have caused similar errors
in other constructed chat histories.
Putting a failed iteration message in the summary prevents that while
letting the research agent continue its research.
All web search providers, like Jina/Searxng?, do not return a text
snippet. Making snippet optional allows processing search results by
these web search providers, without hitting validation errors.
This bug was introduced in 05d4e19cb, version 1.42.2, during migration
to save deeply typed ChatMessageModel. As the ChatMessageModel did
not use the right field name for organic results (since the start).
Previously it did not matter as it was storing to DB irrespective but
now the mapping of dictionary to ChatMessageModel drops that field
before save to conversation in DB.
This was resulting in organic context being lost on page reload and
only being shown on first response.
Not sure why but it some cases when interacting with o3 (which needs
non-streaming) the stream_options seems to be set.
Cannot reproduce but hopefully dropping the stream_options explicitly
should resolve this issue.
Related 985a98214
Older package (like 1.84.0) seem to always pass reasoning_effort
argument to openai api, which now seems to be throwing unexpected
request argument error when used with non-reasoning models (like
4o-mini).
There had been a regression that made all agents display the default
chat model instead of the actual chat model associated with the agent.
This change resolves that issue by prioritizing agent specific chat
model from DB (over user or server chat model).
- Fix code context data type for validation on server. This would
prevent the chat message from being written to history
- Handle null code results on web app
We now pass deeply typed chat messages throughout the application to
construct tool specific chat history views since 05d4e19cb.
This ChatMessageModel didn't allow intent.query to be unset. But
interrupted research iteration history can have unset query. This
changes allows makes intent.query optional.
It also uses message by user entry to populate user message in tool
chat history views. Using query from khoj intent was an earlier
shortcut used to not have to deal with message by user. But that
doesn't scale to current scenario where turns are not always required
to have a single user, assistant message pair.
Specifically a chat history can now contain multiple user messages
followed by a single khoj message. The new change constructs a chat
history that handles this scenario naturally and makes the code more
readable.
Also now only previous research iterations that completed are
populated. Else they do not serve much purpose.
