diff --git a/docs/README.md b/docs/README.md index 06d026a4..a91cd72f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -24,12 +24,13 @@ ## Introduction -Welcome to the Khoj Docs! This is the best place to [get started](./setup.md) with Khoj. +Welcome to the Khoj Docs! This is the best place to [get started](./setup.md) with Khoj. Unless otherwise mentioned, the docs only pertain to self-hosted Khoj instances. -- Khoj is a desktop application to [search](./search.md) and [chat](./chat.md) with your notes, documents and images -- It is an offline-first, open source AI personal assistant accessible from your [Emacs](./emacs.md), [Obsidian](./obsidian.md) or [Web browser](./web.md) -- It works with jpeg, markdown, [notion](./notion_integration.md) org-mode, pdf files and [github repositories](./github_integration.md) -- If you have more questions, check out the [FAQ](https://faq.khoj.dev/) - it's a live Khoj instance indexing our Github repository! +- Khoj is an application to dynamically engage with your notes, documents and images. We support APIs for [semantic search](./search.md) and [chat](./chat.md). +- It can be easily self-hosted and run on your consumer hardware or private cloud. +- It provides an open source, AI personal assistant accessible from your [Emacs](./emacs.md), [Obsidian](./obsidian.md) or [Web browser](./web.md), or our [desktop app](https://khoj.dev/downloads). +- It works with plaintext, markdown, [notion](./notion_integration.md) org-mode, pdf files and [github repositories](./github_integration.md) +- It can support use with multiple users, so you and your family, friends, or team can have a shared assistance server. As the admin, you can configure the server settings at `/server/admin`. ## Quickstart [Click here](./setup.md) for full setup instructions diff --git a/docs/github_integration.md b/docs/github_integration.md index 413dd41e..588919b0 100644 --- a/docs/github_integration.md +++ b/docs/github_integration.md @@ -1,4 +1,4 @@ -# Setup the Github integration +# 🧑🏾‍💻 Setup the Github integration The Github integration allows you to index as many repositories as you want. It's currently default configured to index Issues, Commits, and all Markdown/Org files in each repository. For large repositories, this takes a fairly long time, but it works well for smaller projects. diff --git a/docs/index.html b/docs/index.html index 33ba0735..34df6e9a 100644 --- a/docs/index.html +++ b/docs/index.html @@ -22,6 +22,7 @@ + diff --git a/docs/obsidian.md b/docs/obsidian.md index 10e65ba1..b7a37b85 100644 --- a/docs/obsidian.md +++ b/docs/obsidian.md @@ -18,24 +18,15 @@ ## Setup -- *Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine* - *Ensure you follow the ordering of the setup steps. Install the plugin after starting the khoj backend. This allows the plugin to configure the khoj backend* ### 1. Setup Backend -Open terminal/cmd and run below command to install and start the khoj backend -- On Linux/MacOS - ```shell - python -m pip install khoj-assistant && khoj - ``` - -- On Windows - ```shell - py -m pip install khoj-assistant && khoj - ``` +Follow the instructions in [the setup page](./setup.md?id=_1-install) ### 2. Setup Plugin 1. Open [Khoj](https://obsidian.md/plugins?id=khoj) from the *Community plugins* tab in Obsidian settings panel 2. Click *Install*, then *Enable* on the Khoj plugin page in Obsidian + 3. Get an API key from the Khoj web settings page. Use this key to configure the Khoj plugin in Obsidian. 3. [Optional] To enable Khoj Chat, set your [OpenAI API key](https://platform.openai.com/account/api-keys) in the Khoj plugin settings See [official Obsidian plugin docs](https://help.obsidian.md/Extending+Obsidian/Community+plugins) for details diff --git a/docs/setup.md b/docs/setup.md index 2e02d271..9debb2ec 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -3,41 +3,15 @@ These are the general setup instructions for Khoj. - Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine - Check the [Khoj Emacs docs](/emacs?id=setup) to setup Khoj with Emacs
- Its simpler as it can skip the server *install*, *run* and *configure* step below. + It's simpler as it can skip the server *install*, *run* and *configure* step below. - Check the [Khoj Obsidian docs](/obsidian?id=_2-setup-plugin) to setup Khoj with Obsidian
Its simpler as it can skip the *configure* step below. -### 1. Install +For Installation, you can either use Docker or install Khoj locally. -#### 1.1 Local Server Setup -Run the following command in your terminal to install the Khoj backend. +### 1. Installation (Docker) -- On Linux/MacOS - ```shell - python -m pip install khoj-assistant - ``` - -- On Windows - ```shell - py -m pip install khoj-assistant - ``` -For more detailed Windows installation and troubleshooting, see [Windows Install](./windows_install.md). - - -##### 1.1.1 Local Server Start - -Run the following command from your terminal to start the Khoj backend and open Khoj in your browser. - -```shell -khoj -``` - -Khoj should now be running at http://localhost:42110. You can see the web UI in your browser. - -Note: To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g with `@reboot khoj`) - -#### 1.2 Local Docker Setup -Use the sample docker-compose [in Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml) to run Khoj in Docker. To start the container, run the following command in the same directory as the docker-compose.yml file. You'll have to configure the mounted directories to match your local knowledge base. +Use the sample docker-compose [in Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml) to run Khoj in Docker. Start by configuring all the environment variables to your choosing. Your admin account will automatically be created based on the admin credentials in that file, so pay attention to those. To start the container, run the following command in the same directory as the docker-compose.yml file. This will automatically setup the database and run the Khoj server. ```shell docker-compose up @@ -45,27 +19,131 @@ docker-compose up Khoj should now be running at http://localhost:42110. You can see the web UI in your browser. -#### 1.3 Download the desktop client [Optional] +### 1. Installation (Local) -You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time. This part is currently optional, but may make setup and configuration slightly easier. It removes the need for setting up custom file paths for your Khoj data configurations. +#### Prerequisites -**To download the desktop client, go to https://download.khoj.dev** and the correct executable for your OS will automatically start downloading. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step. +##### Install Postgres (with PgVector) -### 1.4 Use (deprecated) desktop builds +Khoj uses the `pgvector` package to store embeddings of your index in a Postgres database. In order to use this, you need to have Postgres installed. -Before `v0.12.0``, we had self-contained desktop builds that included both the server and the client. These were difficult to maintain, but are still available as part of earlier releases. To find setup instructions, see here: + -- [Desktop Installation](desktop_installation.md) -- [Windows Installation](windows_install.md) +#### **MacOS** -### 2. Configure -1. Set `File`, `Folder` and hit `Save` in each Plugins you want to enable for Search on the Khoj config page -2. Add your OpenAI API key to Chat Feature settings if you want to use Chat -3. Click `Configure` and wait. The app will download ML models and index the content for search and (optionally) chat +Install [Postgres.app](https://postgresapp.com/). This comes pre-installed with `pgvector` and relevant dependencies. -![configure demo](https://user-images.githubusercontent.com/6413477/255307879-61247d3f-c69a-46ef-b058-9bc533cb5c72.mp4 ':include :type=mp4') +#### **Windows** -### 3. Install Interface Plugins (Optional) +Use the [recommended installer](https://www.postgresql.org/download/windows/) + +#### **Linux** +From [official instructions](https://wiki.postgresql.org/wiki/Apt) + +```bash +sudo apt install -y postgresql-common +sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh +sudo apt install postgres-16 postgresql-16-pgvector +``` + +##### **From Source** +1. Follow instructions to [Install Postgres](https://www.postgresql.org/download/) +2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#installation) in case you need to manually install it. Reproduced instructions below for convenience. + +```bash +cd /tmp +git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git +cd pgvector +make +make install # may need sudo +``` + + + + +##### Create the Khoj database + +Make sure to update your environment variables to match your Postgres configuration if you're using a different name. The default values should work for most people. + + + +#### **MacOS** +```bash +createdb khoj -U postgres +``` + +#### **Windows** +```bash +createdb khoj -U postgres +``` + +#### **Linux** +```bash +sudo -u postgres createdb khoj +``` + + + +#### Install package + +##### Local Server Setup +- *Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine* + +Run the following command in your terminal to install the Khoj backend. + + + +#### **MacOS** + +```shell +python -m pip install khoj-assistant +``` + +#### **Windows** + +```shell +py -m pip install khoj-assistant +``` +For more detailed Windows installation and troubleshooting, see [Windows Install](./windows_install.md). + +#### **Linux** + +```shell +python -m pip install khoj-assistant +``` + + + +##### Local Server Start + +Run the following command from your terminal to start the Khoj backend and open Khoj in your browser. + +```shell +khoj --anonymous-mode +``` +`--anonymous-mode` allows you to run the server without setting up Google credentials for login. This allows you to use any of the clients without a login wall. If you want to use Google login, you can skip this flag, but you will have to add your Google developer credentials. + +On the first run, you will be prompted to input credentials for your admin account and do some basic configuration for your chat model settings. Once created, you can go to http://localhost:42110/server/admin and login with the credentials you just created. + +Khoj should now be running at http://localhost:42110. You can see the web UI in your browser. + +Note: To start Khoj automatically in the background use [Task scheduler](https://www.windowscentral.com/how-create-automated-task-using-task-scheduler-windows-10) on Windows or [Cron](https://en.wikipedia.org/wiki/Cron) on Mac, Linux (e.g with `@reboot khoj`) + + +### 2. Download the desktop client + +You can use our desktop executables to select file paths and folders to index. You can simply select the folders or files, and they'll be automatically uploaded to the server. Once you specify a file or file path, you don't need to update the configuration again; it will grab any data diffs dynamically over time. + +**To download the latest desktop client, go to https://download.khoj.dev** and the correct executable for your OS will automatically start downloading. Once downloaded, you can configure your folders for indexing using the settings tab. To set your chat configuration, you'll have to use the web interface for the Khoj server you setup in the previous step. + +To use the desktop client, you need to go to your Khoj server's settings page (http://localhost:42110/config) and copy the API key. Then, paste it into the desktop client's settings page. Once you've done that, you can select files and folders to index. + +### 3. Configure +1. Go to http://localhost:42110/server/admin and login with your admin credentials. Go to the ChatModelOptions if you want to add additional models for chat. +1. Select files and folders to index [using the desktop client](./setup.md?id=_2-download-the-desktop-client). When you click 'Save', the files will be sent to your server for indexing. + - Select Notion workspaces and Github repositories to index using the web interface. + +### 4. Install Client Plugins (Optional) Khoj exposes a web interface to search, chat and configure by default.
The optional steps below allow using Khoj from within an existing application like Obsidian or Emacs. @@ -75,9 +153,17 @@ The optional steps below allow using Khoj from within an existing application li - **Khoj Emacs**:
[Install](/emacs?id=setup) khoj.el +### 5. Use Khoj 🚀 + +You can head to http://localhost:42110 to use the web interface. You can also use the desktop client to search and chat. ## Upgrade ### Upgrade Khoj Server + + + +#### **Local Setup** + ```shell pip install --upgrade khoj-assistant ``` @@ -88,6 +174,16 @@ pip install --upgrade khoj-assistant pip install --upgrade --pre khoj-assistant ``` +#### **Docker** +From the same directory where you have your `docker-compose` file, this will fetch the latest build and upgrade your server. + +```shell +docker-compose up --build +``` + + + + ### Upgrade Khoj on Emacs - Use your Emacs Package Manager to Upgrade - See [khoj.el package setup](/emacs?id=setup) for details