Intelligently initialize a decent default set of chat model options

Given the LLM landscape is rapidly changing, providing a good default
set of options should help reduce decision fatigue to get started

Improve initialization flow during first run
- Set Google, Anthropic Chat models too
  Previously only Offline, Openai chat models could be set during init

- Add multiple chat models for each LLM provider
  Interactively set a comma separated list of models for each provider

- Auto add default chat models for each provider in non-interactive
  model if the {OPENAI,GEMINI,ANTHROPIC}_API_KEY env var is set

- Do not ask for max_tokens, tokenizer for offline models during
  initialization. Use better defaults inferred in code instead

- Explicitly set default chat model to use
  If unset, it implicitly defaults to using the first chat model.
  Make it explicit to reduce this confusion

Resolves #882
This commit is contained in:
Debanjum Singh Solanky
2024-09-19 12:58:22 -07:00
parent f177723711
commit 91c76d4152
5 changed files with 185 additions and 73 deletions

View File

@@ -19,12 +19,12 @@ from khoj.processor.conversation.offline.chat_model import (
from khoj.processor.conversation.offline.utils import download_model
from khoj.processor.conversation.utils import message_to_log
from khoj.routers.helpers import aget_relevant_output_modes
from khoj.utils.constants import default_offline_chat_model
from khoj.utils.constants import default_offline_chat_models
@pytest.fixture(scope="session")
def loaded_model():
return download_model(default_offline_chat_model, max_tokens=5000)
return download_model(default_offline_chat_models[0], max_tokens=5000)
freezegun.configure(extend_ignore_list=["transformers"])