mirror of
https://github.com/khoaliber/khoj.git
synced 2026-03-02 21:19:12 +00:00
65da7daf1f
Conversation logs structure now has session info too instead of just chat info
Session info will allow loading past conversation summaries as context for AI in new conversations
{
"session": [
{
"summary": <chat_session_summary>,
"session-start": <session_start_index_in_chat_log>,
"session-end": <session_end_index_in_chat_log>
}],
"chat": [
{
"intent": <intent-object>
"trigger-emotion": <emotion-triggered-by-message>
"by": <AI|Human>
"message": <chat_message>
"created": <message_created_date>
}]
}
Semantic Search
Allow natural language search on user content like notes, images, transactions using transformer based models
All data is processed locally. User can interface with semantic-search app via Emacs, API or Commandline
Dependencies
- Python3
- Miniconda
Install
git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f environment.yml
conda activate semantic-searchConfigure
Configure application search types and their underlying data source/files in sample_config.yml
Use the sample_config.yml as reference
Run
Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML
python3 -m src.main -c=sample_config.yml -vvUse
-
Semantic Search via Emacs
- Install semantic-search.el
- Run
M-x semantic-search <user-query>
-
Semantic Search via API
- Query:
GEThttp://localhost:8000/search?q="What is the meaning of life&t=notes" - Regenerate Embeddings:
GEThttp://localhost:8000/regenerate?t=image - Semantic Search API Docs
- Query:
Upgrade
cd semantic-search
git pull origin master
conda env update -f environment.yml
conda activate semantic-searchAcknowledgments
- MiniLM Model for Asymmetric Text Search. See SBert Documentation
- OpenAI CLIP Model for Image Search. See SBert Documentation
- Charles Cave for OrgNode Parser
- Sven Marnach for PyExifTool