mirror of
https://github.com/khoaliber/khoj.git
synced 2026-03-02 13:18:18 +00:00
Add Advanced Self Hosting Section, Improve Self Hosting, OpenAI Proxy Docs
- Add instructions for self-hosted users with info, warning boxes to
avoid, fix common issues when setting up Khoj server
- Create new Advanced Self Hosting section
- Extract Advanced Self-Hosting Sections from the Advanced Page and
move them to separate Pages under Advanced Self Hosting section
- Improve OpenAI Proxy Docs
- Put Ollama setup as a section under OpenAI API Proxy page instead
of a separate page
- Add Section to use Khoj with chat model from LM Studio
- Update LiteLLM docs to use chat model from LM Studio
This commit is contained in:
Debanjum Singh Solanky
8
documentation/docs/advanced/_category_.json
Normal file
8
documentation/docs/advanced/_category_.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"label": "Advanced Self Hosting",
|
||||
"position": 6,
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Advanced setup for Self Hosting Khoj server"
|
||||
}
|
||||
}
|
||||
52
documentation/docs/advanced/authentication.md
Normal file
52
documentation/docs/advanced/authentication.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Authenticate
|
||||
|
||||
:::info
|
||||
This is only helpful for self-hosted users or teams. If you're using [Khoj Cloud](https://app.khoj.dev), both Magic Links and Google OAuth work.
|
||||
:::
|
||||
|
||||
By default, most of the instructions for self-hosting Khoj assume a single user, and so the default configuration is to run in anonymous mode. However, if you want to enable authentication, you can do so either with with [Magic Links](#using-magic-links) or [Google OAuth](#using-google-oauth) as shown below. This can be helpful to make Khoj securely accessible to you and your team.
|
||||
|
||||
:::tip[Note]
|
||||
Remove the `--anonymous-mode` flag in your start up command to enable authentication.
|
||||
:::
|
||||
|
||||
## Using Magic Links
|
||||
The most secure way to do this is to integrate with [Resend](https://resend.com) by setting up an account and adding an environment variable for `RESEND_API_KEY`. You can get your API key [here](https://resend.com/api-keys). This will allow you to automatically send sign-in links to users who want to log in.
|
||||
|
||||
It's still possible to use the magic links feature without Resend, but you'll need to manually send the magic links to users who want to log in.
|
||||
|
||||
## Manually sending magic links
|
||||
|
||||
1. The user will have to enter their email address in the login form.
|
||||
They'll click `Send Magic Link`. Without the Resend API key, this will just create an unverified account for them in the backend
|
||||
<img src="/img/magic_link.png" alt="Magic link login form" width="400"/>
|
||||
|
||||
2. You can get their magic link using the admin panel
|
||||
Go to the [admin panel](http://localhost:42110/server/admin/database/khojuser/). You'll see a list of users. Search for the user you want to send a magic link to. Tick the checkbox next to their row, and use the action drop down at the top to 'Get email login URL'. This will generate a magic link that you can send to the user, which will appear at the top of the admin interface.
|
||||
|
||||
| Get email login URL | Retrieved login URL |
|
||||
|---------------------|---------------------|
|
||||
| <img src="/img/admin_get_emali_login.png" alt="Get user magic sign in link" width="400" />| <img src="/img/admin_successful_login_url.png" alt="Successfully retrieved a login URL" width="400" />|
|
||||
|
||||
3. Send the magic link to the user. They can click on it to log in.
|
||||
|
||||
Once they click on the link, they'll automatically be logged in. They'll have to repeat this process for every new device they want to log in from, but they shouldn't have to repeat it on the same device.
|
||||
|
||||
A given magic link can only be used once. If the user tries to use it again, they'll be redirected to the login page to get a new magic link.
|
||||
|
||||
## Using Google OAuth
|
||||
|
||||
To set up your self-hosted Khoj with Google Auth, you need to create a project in the Google Cloud Console and enable the Google Auth API.
|
||||
|
||||
To implement this, you'll need to:
|
||||
1. You must use the `python` package or build from source, because you'll need to install additional packages for the google auth libraries (`prod`). The syntax to install the right packages is
|
||||
```
|
||||
pip install khoj-assistant[prod]
|
||||
```
|
||||
2. [Create authorization credentials](https://developers.google.com/identity/sign-in/web/sign-in) for your application.
|
||||
3. Open your [Google cloud console](https://console.developers.google.com/apis/credentials) and create a configuration like below for the relevant `OAuth 2.0 Client IDs` project:
|
||||
|
||||
|
||||
4. Configure these environment variables: `GOOGLE_CLIENT_SECRET`, and `GOOGLE_CLIENT_ID`. You can find these values in the Google cloud console, in the same place where you configured the authorized origins and redirect URIs.
|
||||
|
||||
That's it! That should be all you have to do. Now, when you reload Khoj without `--anonymous-mode`, you should be able to use your Google account to sign in.
|
||||
17
documentation/docs/advanced/support-multilingual-docs.md
Normal file
17
documentation/docs/advanced/support-multilingual-docs.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Support Multilingual Docs
|
||||
Khoj uses an embedding model to understand documents. Multilingual embedding models improve the search quality for documents not in English. This affects both search and chat with docs experiences across Khoj.
|
||||
|
||||
To improve search and chat quality for non-english documents you can use a [multilingual model](https://www.sbert.net/docs/pretrained_models.html#multi-lingual-models).<br />
|
||||
For example, the [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) supports [50+ languages](https://www.sbert.net/docs/pretrained_models.html#:~:text=we%20used%20the%20following%2050%2B%20languages), has decent search quality and speed for a consumer machine.
|
||||
To use it:
|
||||
1. Open [the search config](http://localhost:42110/server/admin/database/searchmodelconfig/) on your server's admin settings page. Either create a new search model, if none exists, or update the existing one. For example,
|
||||
- Set the `bi_encoder` field to `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`
|
||||
- Set the `cross_encoder` field to `mixedbread-ai/mxbai-rerank-xsmall-v1`
|
||||
2. Regenerate your content index from all the relevant clients. This step is very important, as you'll need to re-encode all your content with the new model.
|
||||
|
||||
:::info[Note]
|
||||
Modern search/embedding model like [mixedbread-ai/mxbai-embed-large-v1](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1) expect a prefix to the query (or docs) string to improve encoding. Update the `bi_encoder_query_encode_config` field of your [embedding model](http://localhost:42110/server/admin/database/searchmodelconfig/) with `{prompt: <prefix-prompt>}` to improve the search quality of these models.
|
||||
|
||||
E.g. `{prompt: "Represent this query for searching documents"}`. You can pass any valid JSON object that the SentenceTransformer `encode` function accepts
|
||||
|
||||
:::
|
||||
119
documentation/docs/advanced/use-openai-proxy.md
Normal file
119
documentation/docs/advanced/use-openai-proxy.md
Normal file
@@ -0,0 +1,119 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
# Use OpenAI Proxy
|
||||
:::info
|
||||
This is only helpful for self-hosted users. If you're using [Khoj Cloud](https://app.khoj.dev), you're limited to our first-party models.
|
||||
:::
|
||||
|
||||
:::info
|
||||
Khoj natively supports local LLMs [available on HuggingFace in GGUF format](https://huggingface.co/models?library=gguf). Using an OpenAI API proxy with Khoj maybe useful for ease of setup, trying new models or using commercial LLMs via API.
|
||||
:::
|
||||
|
||||
Khoj can use any OpenAI API compatible server including [Ollama](#ollama), [LMStudio](#lm-studio) and [LiteLLM](#litellm).
|
||||
Configuring this allows you to use non-standard, open or commercial, local or hosted LLM models for Khoj
|
||||
|
||||
Combine them with Khoj can turn your favorite LLM into an AI agent. Allowing you to chat with your docs, find answers from the internet, build custom agents and run automations.
|
||||
|
||||
## Ollama
|
||||
Ollama allows you to run [many popular open-source LLMs](https://ollama.com/library) locally from your terminal.
|
||||
For folks comfortable with the terminal, Ollama's terminal based flows can ease setup and management of chat models.
|
||||
|
||||
Ollama exposes a local [OpenAI API compatible server](https://github.com/ollama/ollama/blob/main/docs/openai.md#models). This makes it possible to use chat models from Ollama to create your personal AI agents with Khoj.
|
||||
|
||||
### Setup
|
||||
|
||||
1. Setup Ollama: https://ollama.com/
|
||||
2. Start your preferred model with Ollama. For example,
|
||||
```bash
|
||||
ollama run llama3
|
||||
```
|
||||
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
|
||||
- Name: `ollama`
|
||||
- Api Key: `any string`
|
||||
- Api Base Url: `http://localhost:11434/v1/` (default for Ollama)
|
||||
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
|
||||
- Name: `llama3` (replace with the name of your local model)
|
||||
- Model Type: `Openai`
|
||||
- Openai Config: `<the ollama config you created in step 3>`
|
||||
- Max prompt size: `1000` (replace with the max prompt size of your model)
|
||||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
|
||||
That's it! You should now be able to chat with your Ollama model from Khoj. If you want to add additional models running on Ollama, repeat step 6 for each model.
|
||||
|
||||
|
||||
## LM Studio
|
||||
[LM Studio](https://lmstudio.ai/) is a desktop app to chat with open-source LLMs on your local machine. LM Studio provides a neat interface for folks comfortable with a GUI.
|
||||
|
||||
LM Studio can also expose an [OpenAI API compatible server](https://lmstudio.ai/docs/local-server). This makes it possible to turn chat models from LM Studio into your personal AI agents with Khoj.
|
||||
|
||||
### Setup
|
||||
1. Install [LM Studio](https://lmstudio.ai/) and download your preferred Chat Model
|
||||
2. Go to the Server Tab on LM Studio, Select your preferred Chat Model and Click the green Start Server button
|
||||
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
|
||||
- Name: `proxy-name`
|
||||
- Api Key: `any string`
|
||||
- Api Base Url: `http://localhost:1234/v1/` (default for LMStudio)
|
||||
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
|
||||
- Name: `llama3` (replace with the name of your local model)
|
||||
- Model Type: `Openai`
|
||||
- Openai Config: `<the proxy config you created in step 3>`
|
||||
- Max prompt size: `2000` (replace with the max prompt size of your model)
|
||||
- Tokenizer: *Do not set for OpenAI, mistral, llama3 based models*
|
||||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
|
||||
## LiteLLM
|
||||
[LiteLLM](https://docs.litellm.ai/docs/proxy/quick_start) exposes an OpenAI compatible API that proxies requests to other LLM API services. This provides a standardized API to interact with both open-source and commercial LLMs.
|
||||
|
||||
Using LiteLLM with Khoj makes it possible to turn any LLM behind an API into your personal AI agent.
|
||||
|
||||
### Setup
|
||||
1. Install LiteLLM
|
||||
```bash
|
||||
pip install litellm[proxy]
|
||||
```
|
||||
2. Start LiteLLM and use Mistral tiny via Mistral API
|
||||
```
|
||||
export MISTRAL_API_KEY=<MISTRAL_API_KEY>
|
||||
litellm --model mistral/mistral-tiny --drop_params
|
||||
```
|
||||
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
|
||||
- Name: `proxy-name`
|
||||
- Api Key: `any string`
|
||||
- Api Base Url: **URL of your Openai Proxy API**
|
||||
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
|
||||
- Name: `llama3` (replace with the name of your local model)
|
||||
- Model Type: `Openai`
|
||||
- Openai Config: `<the proxy config you created in step 3>`
|
||||
- Max prompt size: `2000` (replace with the max prompt size of your model)
|
||||
- Tokenizer: *Do not set for OpenAI, mistral, llama3 based models*
|
||||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
|
||||
|
||||
## General
|
||||
|
||||
1. Start your preferred OpenAI API compatible app
|
||||
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
|
||||
- Name: `proxy-name`
|
||||
- Api Key: `any string`
|
||||
- Api Base Url: **URL of your Openai Proxy API**
|
||||
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
|
||||
- Name: `llama3` (replace with the name of your local model)
|
||||
- Model Type: `Openai`
|
||||
- Openai Config: `<the proxy config you created in step 3>`
|
||||
- Max prompt size: `2000` (replace with the max prompt size of your model)
|
||||
- Tokenizer: *Do not set for OpenAI, mistral, llama3 based models*
|
||||
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
|
||||
- Default model: `<name of chat model option you created in step 4>`
|
||||
- Summarizer model: `<name of chat model option you created in step 4>`
|
||||
6. Go to [your config](http://localhost:42110/config) and select the model you just created in the chat model dropdown.
|
||||
Reference in New Issue
Block a user