Instead of implicitly defaulting to assuming it is available as:
- For pip install searxng has to be explicitly setup to work
- For docker install we explicitly do set it up and set the
KHOJ_SEARXNG_URL env var already
Also check if Searxng URL is also unset before disable web search
tools now that it is required explicit enablement.
Using prompt cache key enables sticky routing to openai servers.
This increases probability of a chat actor hitting same server and
reusing cached prompts.
We use stable hash of first N characters to uniquely identify a chat
actor prompt
Webpage read is gated behind having a web search engine configured for
now. It can later be decoupled from web search and depend on whether
any web scrapers is configured.
New truncation logic return a new message list.
It does not update message list by reference/in place since 8a16f5a2a.
So truncation tests should run verification on the truncated chat
history returned by the truncation func instead of the original chat
history passed into the truncation func.
- It does not support strict mode for json schema, tool use
- It likes text content to be plain string, not nested in a dictionary
- Verified to work with gpt oss models on cerebras
Responses API is starting to get supported by other ai apis as well.
This change does preparatory improvements to ease moving to use
responses api with other ai apis.
Use the new, better named `supports_responses_api' method.
The method currently just maps to `is_openai_api'. It will add other
ai apis once support for using responses api with them is added.
- Fix identifying gpt-oss as openai reasoning model
- Drop unsupported stop param for openai reasoning models
- Drop the Formatting re-enabled logic for openai reasoing only models
We use responses api for openai models and latest openai models are
hybrid models, they don't seem to need this convoluted system
message to format response as markdown
is_automated_task check isn't required as automation cannot be created
via chat anymore.
conversation specific file_filters are extracted directly in document
search, so doesn't need to be passed down from chat api endpoint
The context building logic was nearly identical across all model
types.
This change extracts that logic into a shared function and calls it
once in the `agenerate_chat_response', the entrypoint to the converse
methods for all 3 model types.
Main differences handled are
- Gemini system prompt had additional verbosity instructions. Keep it
- Pass system messsage via chatml messages list to anthropic, gemini
models as well (like openai models) instead of passing it as
separate arg to chat_completion_* funcs.
The model specific message formatters for both already extract
system instruction from the messages list. So system messages wil be
automatically extracted from the chat_completion_* funcs to pass as
separate arg required by anthropic, gemini api libraries.
Overview
Enable improving speed and cost of chat by setting fast, deep think
models for intermediate steps and non user facing operations.
Details
- Allow decoupling default chat models from models used for
intermediate steps by setting server chat settings on admin panel
- Use deep think models for most intermediate steps like tool
selection, subquery construction etc. in default and research mode
- Use fast think models for webpage read, chat title setting etc.
Faster webpage read should improve conversation latency
What
Explicit selection of notes tool/conversation command by agent is
required now.
Why
- Newer models are good at deciding when to look up notes
- Modern khoj is less of a notes only chat to search notes by default
generated_files wasn't being set (anymore?). But it was being passed
around through for chat context and being saved to db.
Also reduce variables used to set mermaid diagram description
- Process chat history in default order instead of processing it in
reverse. Improve legibility of context construction for minor
performance hit in dropping message from front of list.
- Handle multiple system messages by collating them into list
- Remove logic to drop system role for gemma-2, o1 models. Better to
make code more readable than support old models.
Use seed to stabilize image change consistency across turns when
- KHOJ_LLM_SEED env var is set
- Using Image models via Replicate
OpenAI, Google do not support image seed
Inferred queries is stored with underscore in db but aliased with - in memory.
This conversation.messages logic was broken, so inferred queries field
of chat message history was getting ignored.
This change fixes that issue and improve previous image generation
description for better context for subsequent image generation attempts.
Overview
- Khoj references files it used in its response as markdown links.
For example [1](file://path/to/file.txt#line=121)
- Previously these file links were just shown as raw text
- This change renders khoj's inline file references as a proper links
and shows file content preview (around specified line if deeplink)
on hover or click in the web app
Details
- Render inline file references as links in chat message on web app.
Previously references like [1](file://path/to/file.txt#line=120)
would be shown as plain text. Now they are rendered as links
- Preview file content of referenced files on click or hover.
If reference uses a deeplink with line number, the file content
around that line is shown on hover, click. Click allows viewing file
preview on mobile, unlike hover. Hover is easier with mouse.
Fixes
- Fix to allow khoj to delete content in obsidian write mode
- Do not throw error when no edit blocks in write mode on obsidian
- Limit retries to fix invalid edit blocks in obsidian write mode
Improvements
- Only show 3 recent files as context in obsidian file read, write mode
- Persist open file access mode setting across restarts in obsidian
- Make khoj obsidian keyboard shortcuts toggle voice chat, chat history
- Do not show <SYSTEM> instructions in chat session title on obsidian
Closes#1209
In obsidian we have a hacky system instruction being passed in read,
write file access modes. This shouldn't be shown in chat sessions list
during view or edit. It is an internal implementation detail.
Previously hitting voice chat keybinding would just start voice chat,
not end it and just open chat history and not close it.
This is unintuitive and different from the equivalent button click
behaviors.
Fix toggles voice chat on/off and shows/hides chat history when hit
Ctrl+Alt+V, Ctrl+Alt+O keybindings in khoj obsidian chat view
