Commit Graph

4292 Commits

Author SHA1 Message Date
sabaimran
95ad1f936e upgrade postgres server to 17 2025-01-20 00:10:20 -08:00
sabaimran
a214bd4100 upgrade pg server dev version to 15 2025-01-20 00:05:35 -08:00
sabaimran
82ff74cfa9 Run on container with ubuntu latest for pytest gh action workflow 2025-01-19 23:57:57 -08:00
Debanjum
fde71ded16 Upgrade web app dependencies 2025-01-19 13:44:59 +07:00
Debanjum
2d4633d298 Use encoded email, otp in login URL in email & web app sign-in flow
Previously emails with url special characters would not get
successfully identified for login. Account creation was fine due to
email being in POST request body. But login with such emails did not
work due to query params not being escaped before being sent to server

This change escapes both the code and email in login URL sent to
server. So login with emails containing special characters like
`email+khoj@gmail.com' works. It fixes both the URL web app sent by
web app directly and the magic link sent to users to their email

This change also fixes accessibility issue of having a DialogTitle in
DialogContent for screen readers.

Resolves #1090
2025-01-19 13:11:23 +07:00
Debanjum
51f3af11b5 Fix Qwen 2.5 14B model source to use Q4_K_M quantized model
The official Qwen2.5 14B model doesn't mention standard quantization
suffixes like Q4_K_M, so doesn't work with Khoj
2025-01-19 12:27:35 +07:00
sabaimran
af9e906cb5 Use python3 instead of python when running pip install commands in gh actions 2025-01-17 17:48:42 -08:00
sabaimran
b8c866014d Improve instruction description for the agent command description for notes 2025-01-17 17:19:59 -08:00
Tuğhan Belbek
849348e638 Handle additional HTTP redirect status code 308 in scheduled chat requests (#1088)
Closes #1067
2025-01-16 07:03:43 -08:00
Debanjum
00843f4f24 Release Khoj version 1.34.0 2025-01-16 12:11:28 +07:00
Debanjum
ad27f34c96 Support online search using Google Search API (#1087)
Add official Google Search API as an online search provider.
We currently support Serper.dev and Searxng as online search providers.
2025-01-16 11:41:59 +07:00
Debanjum
a649b03658 Support online search using Google Search API 2025-01-16 11:39:03 +07:00
sabaimran
42d4d15346 Merge pull request #1054 from khoj-ai/features/add-support-for-mermaidjs
We've been having issues generating diagrams with Excalidraw that are any degree of complexity. By contrast, LLMs are able to handle Mermaid.js syntax a lot better, as it's much more forgiving and has a simpler declarative style. Refer to https://mermaid.js.org/.

Update so that new diagrams are generated with Mermaid.js, while old diagrams generated with Excalidraw can still be viewed.
2025-01-15 11:55:12 -08:00
Debanjum
e2b2b3415e Fix handling of inline base64 images by Obsidian, Desktop clients
Fix for #1082 pushed down adding the `data:image/webp;base64' prefix
of the base64 images to the server image gen API. But the code on the
Obsidian and Desktop client still add these prefixes.

This change stops the Desktop, Obsidian clients from adding the prefix
as it is being handled by the API now. It should resolve showing
images inline in those clients as well
2025-01-15 23:34:23 +07:00
Debanjum
2e585efd2f Fix end with newline styling issue in style.css to pass lint checks 2025-01-15 19:43:02 +07:00
Debanjum
ed18c04576 Fix wrapping base64 generated image for inline display
Resolves #1082
2025-01-15 19:19:31 +07:00
Debanjum
f8b887cabd Allow using OpenAI (compatible) API for Speech to Text transcription 2025-01-15 19:19:31 +07:00
Debanjum
182c49b41c Prefer explicitly configured OpenAI API url, key for image gen model
Previously we'd use the general openai client, even if the image
generation model has a different api key and base url set.

This change uses the openai config of the image generation models when
set. Otherwise it fallbacks to use the first openai api provider set
2025-01-15 19:19:31 +07:00
Debanjum
24204873c8 Use same openai base url env var name as the official openai client
This eases re-use of the OpenAI API across all openai clients,
including chat, image generation, speech to text.

Resolves #1085
2025-01-15 19:19:30 +07:00
Debanjum
63dd3985b5 Support using Embeddings Model exposed via OpenAI (compatible) API (#1051)
This change adds the ability to use OpenAI, Azure OpenAI or any embedding model exposed behind an OpenAI compatible API (like Ollama, LiteLLM, vLLM etc.).

Khoj previously only supported HuggingFace embedding models running locally on device or via HuggingFaceW inference API endpoint. This allows using commercial embedding models to index your content with Khoj.
2025-01-15 17:39:54 +07:00
Debanjum
a6bf6803b6 Add docs on how to add, edit search model configs when self-hosting 2025-01-15 17:30:18 +07:00
Debanjum
92a1ec7afc Do not auto restart khoj docker services by default
Let folks who want to add that, add it manually if they want to. It
creates too much noise for folks having trouble with self-host setup
2025-01-15 13:09:50 +07:00
Debanjum
85c537a1de Set default PORT arg in Dockerfile to default Khoj port, 42110 2025-01-15 13:09:50 +07:00
Debanjum
9355381fac Catch error in call to data sources, output format selection tool AI
Previously if the call to this tool AI failed, the API call would
non-gracefully fail on server. This would leave the client hanging in
a wierd state (e.g with spinner running on web app with no indication
of issue).

Also do not show filters in query debug log lines when no filters in query
2025-01-15 13:09:50 +07:00
Debanjum
24ab8450ba Handle scenario where read chat stream error is not json on web app 2025-01-15 13:09:50 +07:00
sabaimran
7f329e7e9d Fix configuration of name field for chatmodel options during initalization 2025-01-12 22:37:08 -08:00
sabaimran
1a00540ee9 Improve error handling in mermaid chart rendering 2025-01-12 22:36:31 -08:00
Osama Ata
96e3d0a7b9 Fix stale lmstudio documentation to set ai model api via admin panel (#1075)
Use new name `Ai Model API` instead of  `OpenAI Processor Conversation Config`
2025-01-12 03:06:01 -08:00
Debanjum
6bd9f6bb61 Give a shorter, simpler name to github workflow to deploy docs 2025-01-12 10:54:56 +07:00
Debanjum
51a774c993 Add contrast to setting card inputs in dark mode on web app 2025-01-11 14:50:47 +07:00
Debanjum
9e8b8dc5a2 Toggle showing api key on web settings page via a visibility toggle
- Background
  Access to the clipboard API is disabled by certain browsers in non
  localhost http scenarios for security reasons.

  So the copy API key button doesn't work when khoj is self-hosted
  with authentication enabled at a non localhost domain

- Change
  This change enables copying API keys by manual text highlight + copy
  if copy button is disabled

Resolves #1070
2025-01-11 14:50:47 +07:00
Debanjum
25c39bd7da Extract api keys setting card into separate component on web app 2025-01-11 14:50:46 +07:00
sabaimran
be4b091a21 Add new line to styles.css 2025-01-10 21:52:52 -08:00
sabaimran
f398e1eb0c Add codeblock rendering for the mermaidjs diagram in obsidian 2025-01-10 21:46:39 -08:00
Debanjum
6e955e158b Use normalized email address for new users
Not check email deliverability for now to allow air-gapped usage or
authenticated/multi-user setups with admin managed otp

Closes #1069
2025-01-11 12:28:40 +07:00
sabaimran
c441663394 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-support-for-mermaidjs 2025-01-10 21:25:33 -08:00
sabaimran
85c34a5f0f Merge pull request #1018 from hjamet/master
This PR delivers comprehensive improvements to the Khoj plugin across multiple key areas:

🔍 Search Enhancements:
- Added visual loading indicators during search operations
- Implemented color-coded results to distinguish between vault and external files
- Added abort logic for previous requests to improve performance
- Enhanced search feedback with clear status indicators
- Improved empty state handling

🔄 Synchronization Improvements:
- Added configurable sync interval setting in minutes
- Implemented manual "Sync new changes" command
- Enhanced sync timer management with automatic restart
- Improved notification system for sync operations

📁 Folder Management:
- Added granular folder selection for sync
- Implemented intuitive folder suggestion modal
- Enhanced folder list visualization

💅 UI/UX Improvements:
- Added loading animations and spinners
- Enhanced search results visualization with color coding
- Refined chat interface styling
- Improved overall settings panel organization

🔧 Technical Improvements:
- Refactored search and synchronization logic
- Implemented proper request cancellation
- Enhanced error handling and user feedback
- Improved code organization and maintainability
2025-01-10 21:24:12 -08:00
Patrick Jackson
6e0c767ff0 Use the configured OpenAI Base URL for Automations (#1065)
This change makes Automations (and possibly other entrypoints) use the configured OpenAI-compatible server if that has been set. Without this change it tries to use the hardcoded OpenAI provider.
2025-01-10 17:17:51 -08:00
Debanjum
1b5826d8b6 Support using Embeddings Model exposed via OpenAI (compatible) API 2025-01-10 23:48:04 +07:00
Debanjum
65f1c27963 Remove old, big warning about Khoj not configured on server init
- Just say using default config. This old khoj.yml settings mechamism
  isn't standard, so not having a configured khoj.yml isn't a concern
- Deep link to desktop download instead of the whole download page as
  android etc. are also on it, which don't help with syncing docs
2025-01-10 23:46:27 +07:00
Debanjum
3cc6597b49 Support Azure OpenAI API endpoint (#1048)
OpenAI chat models deployed on Azure are (ironically) not OpenAI API compatible endpoints.
This change enables using OpenAI chat models deployed on Azure with Khoj.
2025-01-10 08:35:03 -08:00
sabaimran
bac90ad69d Upgrade deploy-pages action to vv4 2025-01-09 19:04:31 -08:00
Debanjum
2069f571c8 Upgrade upload-artifact gh action to v4 as <=v3 deprecated
This started failing github workflow jobs
2025-01-10 00:41:24 +07:00
Debanjum
dd63bd8bcf Fix dark mode dropdown colors of phone no. country code on web settings page
Resolves #1046
2025-01-10 00:10:51 +07:00
Debanjum
bb6a6cbe19 Restart Khoj docker services unless stopped. Remove default network
- Seems less aggressive to use unless-stopped versus always.
- Default network is used anyway, so doesn't seem necessary to specify
2025-01-09 22:08:48 +07:00
Debanjum
01d27f5220 Do not show user logout button on web app side pane in anoymous mode
Refer
https://github.com/khoj-ai/khoj/issues/1050#issuecomment-2579119234
2025-01-09 21:18:50 +07:00
Debanjum
a739936563 Mark Github integration as unmaintained in documentation
Also mention what the Reflective Questions table is about

Resolves #1059
2025-01-09 20:42:38 +07:00
Debanjum
1eaa54b0ae Make PAT token requirement optional for Github indexing for now
The github integration has not been tested and may still be broken.
This change at least makes it easier to add repositories without
needing a PAT token if/when it does work.
2025-01-09 20:42:38 +07:00
sabaimran
ec02757fd1 Add an export feature along with the mermaid diagram. Add sidebar to loading page. 2025-01-08 23:53:58 -08:00
sabaimran
889f34c7bf Adjust typing and error handling for incorrect diagrams 2025-01-08 23:22:16 -08:00