Commit Graph

4467 Commits

Author SHA1 Message Date
sabaimran
da2b89e46a Merge branch 'master' of github.com:khoj-ai/khoj into features/add-a-knowledge-base-page 2025-01-10 22:18:14 -08:00
sabaimran
f170487338 Fix apostrophe in the add documents modal 2025-01-10 21:58:17 -08:00
sabaimran
be4b091a21 Add new line to styles.css 2025-01-10 21:52:52 -08:00
sabaimran
f398e1eb0c Add codeblock rendering for the mermaidjs diagram in obsidian 2025-01-10 21:46:39 -08:00
Debanjum
6e955e158b Use normalized email address for new users
Not check email deliverability for now to allow air-gapped usage or
authenticated/multi-user setups with admin managed otp

Closes #1069
2025-01-11 12:28:40 +07:00
sabaimran
c441663394 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-support-for-mermaidjs 2025-01-10 21:25:33 -08:00
sabaimran
85c34a5f0f Merge pull request #1018 from hjamet/master
This PR delivers comprehensive improvements to the Khoj plugin across multiple key areas:

🔍 Search Enhancements:
- Added visual loading indicators during search operations
- Implemented color-coded results to distinguish between vault and external files
- Added abort logic for previous requests to improve performance
- Enhanced search feedback with clear status indicators
- Improved empty state handling

🔄 Synchronization Improvements:
- Added configurable sync interval setting in minutes
- Implemented manual "Sync new changes" command
- Enhanced sync timer management with automatic restart
- Improved notification system for sync operations

📁 Folder Management:
- Added granular folder selection for sync
- Implemented intuitive folder suggestion modal
- Enhanced folder list visualization

💅 UI/UX Improvements:
- Added loading animations and spinners
- Enhanced search results visualization with color coding
- Refined chat interface styling
- Improved overall settings panel organization

🔧 Technical Improvements:
- Refactored search and synchronization logic
- Implemented proper request cancellation
- Enhanced error handling and user feedback
- Improved code organization and maintainability
2025-01-10 21:24:12 -08:00
sabaimran
57545c1485 Fix the migration script to delete orphaned fileobjects
- Remove knowledge page from the sidebar
- Improve speed and rendering of the documents in the search page
2025-01-10 21:06:48 -08:00
sabaimran
d77984f9d1 Remove separate knowledge base file - consolidated in the search page 2025-01-10 18:57:38 -08:00
sabaimran
f2c6ce2435 Improve rendering of the file objects and sort files by updated_date 2025-01-10 18:18:15 -08:00
Patrick Jackson
6e0c767ff0 Use the configured OpenAI Base URL for Automations (#1065)
This change makes Automations (and possibly other entrypoints) use the configured OpenAI-compatible server if that has been set. Without this change it tries to use the hardcoded OpenAI provider.
2025-01-10 17:17:51 -08:00
sabaimran
454a752071 Initial commit: add a dedicated page for managing the knowledge base
- One current issue in the Khoj application is that managing the files being referenced as the user's knowledge base is slightly opaque and difficult to access
- Add a migration for associating the fileobjects directly with the Entry objects, making it easier to get data via foreign key
- Add the new page that shows all indexed files in the search view, also allowing you to upload new docs directly from that page
- Support new APIs for getting / deleting files
2025-01-10 16:24:50 -08:00
Debanjum
1b5826d8b6 Support using Embeddings Model exposed via OpenAI (compatible) API 2025-01-10 23:48:04 +07:00
Debanjum
65f1c27963 Remove old, big warning about Khoj not configured on server init
- Just say using default config. This old khoj.yml settings mechamism
  isn't standard, so not having a configured khoj.yml isn't a concern
- Deep link to desktop download instead of the whole download page as
  android etc. are also on it, which don't help with syncing docs
2025-01-10 23:46:27 +07:00
Debanjum
3cc6597b49 Support Azure OpenAI API endpoint (#1048)
OpenAI chat models deployed on Azure are (ironically) not OpenAI API compatible endpoints.
This change enables using OpenAI chat models deployed on Azure with Khoj.
2025-01-10 08:35:03 -08:00
sabaimran
bac90ad69d Upgrade deploy-pages action to vv4 2025-01-09 19:04:31 -08:00
Debanjum
2069f571c8 Upgrade upload-artifact gh action to v4 as <=v3 deprecated
This started failing github workflow jobs
2025-01-10 00:41:24 +07:00
Debanjum
dd63bd8bcf Fix dark mode dropdown colors of phone no. country code on web settings page
Resolves #1046
2025-01-10 00:10:51 +07:00
Debanjum
bb6a6cbe19 Restart Khoj docker services unless stopped. Remove default network
- Seems less aggressive to use unless-stopped versus always.
- Default network is used anyway, so doesn't seem necessary to specify
2025-01-09 22:08:48 +07:00
Debanjum
01d27f5220 Do not show user logout button on web app side pane in anoymous mode
Refer
https://github.com/khoj-ai/khoj/issues/1050#issuecomment-2579119234
2025-01-09 21:18:50 +07:00
Debanjum
a739936563 Mark Github integration as unmaintained in documentation
Also mention what the Reflective Questions table is about

Resolves #1059
2025-01-09 20:42:38 +07:00
Debanjum
1eaa54b0ae Make PAT token requirement optional for Github indexing for now
The github integration has not been tested and may still be broken.
This change at least makes it easier to add repositories without
needing a PAT token if/when it does work.
2025-01-09 20:42:38 +07:00
sabaimran
ec02757fd1 Add an export feature along with the mermaid diagram. Add sidebar to loading page. 2025-01-08 23:53:58 -08:00
sabaimran
889f34c7bf Adjust typing and error handling for incorrect diagrams 2025-01-08 23:22:16 -08:00
sabaimran
11fcf2f299 Remove dangling response type 2025-01-08 22:10:59 -08:00
sabaimran
6b0a49b12d Add the mermaid package and apply front-end parsing
- Add the mermaid package and apply front-end parsing for interpreting the diagrams. Retain processing of the excalidraw type for backwards compatibility
2025-01-08 22:09:35 -08:00
sabaimran
539ce99343 Add backend support for parsing and processing and storing mermaidjs diagrams
- Replace default diagram output from excalidraw to mermaid
- Retain typing of the excalidraw type for backwards compatibility in the chatlog
2025-01-08 22:08:40 -08:00
sabaimran
c448c49811 Clean-up some code on homepage and disable initial card animations because of jitter 2025-01-08 22:07:23 -08:00
Ikko Eltociear Ashimine
9ce9f02886 Fix typo in Admin Doc (#1034)
appropiate -> appropriate
2025-01-08 21:27:34 -08:00
sabaimran
ad5f0c7a02 Merge pull request #1029 from DPS0340/master
Improve docker-compose.yml

- Do not expose dependencies on host internet
- Put all services on the same network
2025-01-08 10:42:47 -08:00
Henri Jamet
f42b0cb08c Refactor comments and CSS for improved clarity in Khoj plugin
- Translated comments from French to English for better accessibility and understanding.
- Updated CSS comment for loading animation to reflect the change in language.
- Enhanced code readability by ensuring consistent language usage across multiple files.
- Improved user experience by clarifying the purpose of various functions and settings in the codebase.
2025-01-08 09:31:43 +01:00
sabaimran
875cdde9b9 Release Khoj version 1.33.2 2025-01-07 15:32:18 -08:00
sabaimran
5c5c4a6bbc Add help text for Enterprises in the README 2025-01-07 14:55:57 -08:00
sabaimran
8d028e10c6 Fix populating login url in sign in email 2025-01-07 14:53:00 -08:00
omahs
36bdaedd2d Fix typos in Khoj Docs (#1033) 2025-01-07 15:55:57 +07:00
sabaimran
25c1c1c591 Release Khoj version 1.33.1 2025-01-06 09:08:01 -08:00
sabaimran
689d9d8b3a Update formatting in admin.py and utils.py 2025-01-06 09:07:28 -08:00
thinker007
aa442c28eb Handle reporting chat estimated cost when some fields unavailable (#1026)
Fix AttributeError: 'NoneType' object has no attribute 'model_extra'

* cost = chunk.usage.model_extra.get("estimated_cost", 0) if hasattr(chunk, "usage") and chunk.usage else 0  # Estimated costs returned by DeepInfra API
2025-01-06 09:03:49 -08:00
sabaimran
4aed6f7e08 Add a link around the header khojlogotype to go home 2025-01-06 08:55:00 -08:00
Debanjum
266d274e21 Make automation should_notify check robust to non json mode chat models
Use clean_json to handle automation should_notify check For gemini and
other chat models where enforcing json mode is problematic, not supported
2025-01-06 20:16:32 +07:00
Debanjum
9a5e3583cf Remove bullet styling only from sidebar items on web app
Previous fix had removed bullet styling from all components in web
app. This made chat messages on the web app lose bullet styling too.
2025-01-06 20:15:42 +07:00
Debanjum
dc0bc5bcca Evaluate information retrieval quality using eval script
- Encode article urls in filename indexed in Khoj KB
  Makes it easier for humans to compare, trace retrieval performance
  by looking at logs than using content hash (which was previously
  explored)
2025-01-06 13:19:52 +07:00
Debanjum
daeba66c0d Optionally pass references used by agent for response to eval scorers
This will allow the eval framework to evaluate retrieval quality too
2025-01-06 13:19:52 +07:00
Debanjum
8231f4bb6e Return accuracy as decision to generalize across IR & standard scorers 2025-01-06 13:19:52 +07:00
Jiho Lee
c1c086e431 fix: Use localhost on SEARXNG_BASE_URL
Co-authored-by: sabaimran <65192171+sabaimran@users.noreply.github.com>
2025-01-06 13:32:06 +09:00
sabaimran
eb9aadf72a Add an Obsidian README documentation for development 2025-01-05 19:06:27 -08:00
sabaimran
e89e49818b Merge pull request #1028 from ReallyVirtual/patch-1
Update image_generation.md
2025-01-05 13:53:00 -08:00
sabaimran
616cc189d1 Remove bullet points from li styling explicitly 2025-01-05 13:52:08 -08:00
sabaimran
a5705a5aa6 After agent prompt safe check is parsed as json, load it into a json object 2025-01-05 13:40:20 -08:00
Yash-1511
f306159a5a feat: add autocomplete suggestions feature in search page 2025-01-05 17:30:00 +05:30