klbr/khoj
1
0
Fork 0
mirror of https://github.com/khoaliber/khoj.git synced 2026-03-02 21:19:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
203 Commits 1 Branch 0 Tags
b0067fc32efb6bb3559d1affc50e1bbaed7b6f72
Commit Graph

90 Commits

Author SHA1 Message Date
Saba
33bc62dc19 Fix type of use_xmp_metadata to be bool, rather than str 2022-01-24 21:53:26 -05:00
Debanjum Singh Solanky
179153dc5a Rename RawConfig Types for Consistency 
- Naming convention - [ContentType][ConfigType]Config
  - Where [ConfigType] ~ Content, Search, Processor
  - Where [ContentType] ~ Text, Image, Asymmetric, Symmetric, Conversation

- Current Configs:
  - Content:
    - Org Notes
    - Org Music
    - Image
    - Ledger/Beancount

  - Search:
     - Asymmetric
     - Symmetric
     - Image

  - Processor:
    - Conversation
 2022-01-14 20:54:38 -05:00
Debanjum Singh Solanky
c64e0c2965 Load model from HuggingFace if model_directory unset in config YAML 
- Do not save/load the model to/from disk when model_directory unset
in config.yml
- Add symmetric search default config to cli.py
 2022-01-14 17:36:59 -05:00
Debanjum Singh Solanky
510faa1904 Save Image Search Model to Disk 2022-01-14 17:36:59 -05:00
Debanjum Singh Solanky
934ec233b0 Add Search Config for Symmetric Model. Save Model to Disk 2022-01-14 17:36:59 -05:00
Debanjum Singh Solanky
b63026d97c Save Asymmetric Search Model to Disk 
- Improve application load time
- Remove dependence on internet to startup application and perform semantic search
 2022-01-14 17:36:27 -05:00
Debanjum Singh Solanky
2e53fbc844 Fix the user intent extraction prompt for GPT. Clean up chatbot test 2022-01-12 10:36:01 -05:00
Debanjum Singh Solanky
ea28897cdd Remove deprecated conversation_history field from config 2022-01-12 10:35:52 -05:00
Debanjum Singh Solanky
5a686b7be9 Add logs for chat bot in verbose mode 2022-01-12 10:35:52 -05:00
Debanjum Singh Solanky
6dc2a99d35 Merge branch 'master' of github.com:debanjum/semantic-search into add-summarize-capability-to-chat-bot 
- Fix openai_api_key being set in ConfigProcessorConfig
- Merge addition of config UI and config instantiation updates
 2021-12-20 13:30:42 +05:30
Debanjum Singh Solanky
65da7daf1f Load, Save Conversation Session Summaries to Log. s/chat_log/chat_session 
Conversation logs structure now has session info too instead of just chat info
Session info will allow loading past conversation summaries as context for AI in new conversations

{
    "session": [
    {
        "summary": <chat_session_summary>,
        "session-start": <session_start_index_in_chat_log>,
        "session-end": <session_end_index_in_chat_log>
    }],
    "chat": [
    {
        "intent": <intent-object>
        "trigger-emotion": <emotion-triggered-by-message>
        "by": <AI|Human>
        "message": <chat_message>
        "created": <message_created_date>
    }]
}
 2021-12-15 10:17:07 +05:30
Saba
97a6dfaa1e Use default value False for verbose parameter, and small changes 
Pass config as parameter to initialize_search, change name of API methods to handle config CRUD operations, and initalize config to FullConfig
 2021-12-11 14:13:14 -05:00
Saba
9536358d34 Fix key error model_name issue by upgrade sentence-transformers version 
Refer to https://github.com/UKPLab/sentence-transformers/issues/1241
Also user verbose flag passed through function parameters in image_search
 2021-12-11 11:58:19 -05:00
Saba
ce7a751e6b Fix passing verbose flag down in symmetric_ledger.py 2021-12-11 11:36:32 -05:00
Saba
d65190c3ee Update unit tests, files with removing model suffix to config types 2021-12-09 08:50:38 -05:00
Debanjum Singh Solanky
0ac1e5f372 Summarize chat logs and notes returned by semantic search via /chat API 2021-12-08 02:34:07 +05:30
Saba
76e9e9da2f Update unit tests to use the new BaseModel types 2021-12-05 09:31:39 -05:00
Saba
9b16cdbb41 Use past tense for verbose log 2021-12-04 11:45:44 -05:00
Saba
10e4065e05 Consolidate the search config models and pass verbose as a top level flag 2021-12-04 11:43:48 -05:00
Saba
43e647835b Append Model Suffixed to config models 2021-12-04 10:51:21 -05:00
Saba
e068968b35 Update imports for raw config models in config.py 2021-12-04 10:44:55 -05:00
Saba
4d6284b0af Remove Test suffix from Config models 2021-12-04 10:44:13 -05:00
Saba
7fcc8d2cef Add null check for processor config 2021-12-04 10:11:00 -05:00
Saba
7ca4fc3453 Resolve mrege conflicts with updated processor conversation data model 2021-11-28 16:22:52 -05:00
Saba
87a6c2d716 Use parse_obj instead of parse_raw as incoming data is in dict 2021-11-28 14:34:32 -05:00
Saba
5d50487d83 Linting 
New line at end of config.html
Remove debug print statement
 2021-11-28 13:32:56 -05:00
Saba
6f466c8d99 Use global config and add a regenerate button to the config ui' && git push 2021-11-28 13:28:22 -05:00
Saba
34d1e4199c Use alias generator when deserializing the config file 2021-11-28 13:05:48 -05:00
Saba
19b81e82f0 Write back to the raw config.yml file on update 2021-11-28 12:34:40 -05:00
Saba
8837b02de6 dump updated config to a yaml file 2021-11-28 12:26:07 -05:00
Saba
5b80b87379 Streamline None checking in initialize_search 2021-11-28 12:05:04 -05:00
Saba
bf8ae31e6a Streamline None checking in initialize_search 2021-11-28 11:59:45 -05:00
Saba
da52433d89 Update to re-use the raw config base models in config.py as well 2021-11-28 11:57:33 -05:00
Saba
6292fe4481 Update to re-use the raw config base models in config.py as well 2021-11-28 11:57:13 -05:00
Saba
311c4b7e7b Working API request body parsing to /post config! 2021-11-28 11:16:33 -05:00
Saba
66183cc298 Working API request body parsing to /post config! 2021-11-28 11:12:26 -05:00
Debanjum Singh Solanky
5cd920544d Add GPT method to summarize notes and chat logs 2021-11-28 13:08:05 +05:30
Debanjum Singh Solanky
1785047ea6 Improve understand primer and load understand response as dict 2021-11-28 13:04:16 +05:30
Saba
64645c3ac1 Begin type checking/input validation effort 2021-11-27 21:47:56 -05:00
Saba
9a0264b7fc Add a dummy POST config endpoint, integrate with editable UI 2021-11-27 20:36:03 -05:00
Saba
f3b03ea5b7 Make raw data reactive to changes 2021-11-27 19:17:15 -05:00
Debanjum Singh Solanky
67c3cd7372 Wire up GPT understand method to /chat API. Log conversation metadata too 2021-11-28 00:04:39 +05:30
Saba
3db06eee3f Basic example of serving conifg as JSON and retriving on button click 2021-11-27 10:49:33 -05:00
Saba
3d4471e107 Merge branch 'master' of github.com:debanjum/semantic-search into saba/configui 2021-11-27 08:52:48 -05:00
Debanjum Singh Solanky
ccfb97e1a7 Wire up minimal conversation processor. Expose it over /chat API endpoint 
Ensure conversation history persists across application restart
 2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
a99b4b3434 Make conversation processor configurable 2021-11-27 18:12:01 +05:30
Debanjum Singh Solanky
d4e1120b22 Add GPT based conversation processor to understand intent and converse with user 
- Allow conversing with user using GPT's contextually aware, generative capability
- Extract metadata, user intent from user's messages using GPT's general understanding
 2021-11-27 18:12:01 +05:30
Saba
baee52648d Set up basic ui page with no functionality 2021-11-26 14:51:11 -05:00
debanjum
46661b3057 Ensure top_k never more than total entries to run symmetric search on 2021-11-16 11:32:21 -08:00
debanjum
8c858d1a94 Reduce symmetric search results for cross-encoder to re-rank to improve search speed 2021-11-16 11:31:19 -08:00