Commit Graph

  • 0d49398954 Reuse code to query api, render results. Formalize method, arg names Debanjum Singh Solanky 2022-07-27 04:14:14 +04:00
  • fd1963d781 Implement Basic Incremental Search Interface in Emacs for Org Mode Notes Debanjum Singh Solanky 2022-07-27 03:05:00 +04:00
  • 3fa7d8f03a Skeleton to allow incremental search on Khoj via Emacs Debanjum Singh Solanky 2022-07-27 02:48:27 +04:00
  • 1168244c92 Make cross-encoder re-rank results if query param set on /search API Debanjum Singh Solanky 2022-07-26 22:56:36 +04:00
  • b1e64fd4a8 Improve search speed. Only apply filter if filter keywords in query Debanjum Singh Solanky 2022-07-26 22:47:26 +04:00
  • f094c86204 Trace query response performance and display timings in verbose mode Debanjum Singh Solanky 2022-07-26 21:03:53 +04:00
  • d8efcd559f Add Feature Section to Readme Debanjum 2022-07-25 15:43:27 -07:00
  • f953b20415 Add Khoj Architecture Diagram in Docs. Show it in the Project Readme Debanjum Singh Solanky 2022-07-26 02:07:33 +04:00
  • 674d933282 Improve Khoj Intro text. Move Run Unit Test Section under Developement Heading Debanjum Singh Solanky 2022-07-26 02:03:41 +04:00
  • 3728583e08 Update Readme. Add section for using Khoj via Web interface Debanjum Singh Solanky 2022-07-22 04:02:03 +04:00
  • 65fea7681a Rename notes search type to org search, now that markdown notes supported Debanjum Singh Solanky 2022-07-21 21:57:57 +04:00
  • 1f4b5ac112 Create test markdown files. Use them in sample config, docker-compose Debanjum Singh Solanky 2022-07-21 21:54:30 +04:00
  • 4c24202e42 Update documentation. Simplify, reflect current capabilities Debanjum Singh Solanky 2022-07-21 21:18:25 +04:00
  • d4d7dbaca6 Support Natural Search on Markdown Files Debanjum Singh Solanky 2022-07-21 20:22:24 +04:00
  • 0602d018c0 Merge Symmetric, Asymmetric Search Types into a single Text Search Type Debanjum Singh Solanky 2022-07-21 18:05:43 +04:00
  • 0917f1574d Consolidate jsonl helper methods in a single file under utils module Debanjum Singh Solanky 2022-07-21 03:28:37 +04:00
  • de726c4b6c Minor fixes to unused installer utility script Debanjum Singh Solanky 2022-07-21 03:29:24 +04:00
  • 5aad297286 Reuse logic to extract entries across symmetric, asymmetric search Debanjum Singh Solanky 2022-07-21 02:53:18 +04:00
  • e220ecc00b Generate compiled form of each transaction directly in the beancount processor Debanjum Singh Solanky 2022-07-21 02:43:28 +04:00
  • 06cf425314 Generate compiled form of each entry directly in the org-mode processor Debanjum Singh Solanky 2022-07-21 01:47:51 +04:00
  • 4ead79d272 Make Notes Search Natural Language Date Aware Debanjum Singh Solanky 2022-07-21 01:00:15 +04:00
  • d50bfb5188 Parse Logbook Entries in the OrgNode parser for Org-Mode. Update tests Debanjum Singh Solanky 2022-07-21 00:15:30 +04:00
  • 70e70d4b15 Rename 'embed' key to more generic 'compiled' for jsonl extracted results Debanjum Singh Solanky 2022-07-20 20:35:50 +04:00
  • c1369233db Consistently use "entry", "score" in json response for all search types Debanjum Singh Solanky 2022-07-20 20:33:27 +04:00
  • d68a9dc445 Sort extracted images before computing their embeddings Debanjum Singh Solanky 2022-07-20 03:51:27 +04:00
  • c4c7f38b15 Fix extracting image names from multiple image directories Debanjum Singh Solanky 2022-07-20 03:40:49 +04:00
  • c9ff97451b Fix tests to handle updated response types by API Debanjum Singh Solanky 2022-07-20 03:01:56 +04:00
  • bdc1b9f2bb Resolve edge case errors in encoding image metadata Debanjum Singh Solanky 2022-07-20 02:58:43 +04:00
  • 2a5445216c Image input directory not required by collate result as image_name already absolute path Debanjum Singh Solanky 2022-07-20 02:56:23 +04:00
  • 6c9ffdba57 Allow indexing multiple image directories for image search Debanjum Singh Solanky 2022-07-20 02:54:03 +04:00
  • 68ee88cebc Fix image search tests after update to API response for image search types Debanjum Singh Solanky 2022-07-20 01:37:01 +04:00
  • 70221bb038 Allow filtering transactions by date in symmetric ledger Debanjum Singh Solanky 2022-07-19 20:58:24 +04:00
  • b673d26a12 Extract Entries in a standardized format across text search types Debanjum Singh Solanky 2022-07-19 20:52:25 +04:00
  • e66cd5bf59 Only extract transactions from Beancount Debanjum Singh Solanky 2022-07-19 19:50:58 +04:00
  • 732b2d287f Give the project a short, less generic name. Rename it to Khoj Debanjum Singh Solanky 2022-07-19 18:26:16 +04:00
  • 989526ae54 Use a more accurate model for symmetric semantic search Debanjum Singh Solanky 2022-07-18 20:16:40 +04:00
  • 4a90972e38 Use a better model for asymmetric semantic search Debanjum Singh Solanky 2022-07-18 20:00:19 +04:00
  • 5e302dbcda Fix using 1 column layout on small screens Debanjum Singh Solanky 2022-07-18 02:31:30 +04:00
  • 7d16b673b1 Use Single Column Layout for Small Screens on Web Interface Debanjum Singh Solanky 2022-07-18 02:08:52 +04:00
  • 31a221a76b Auto focus cursor on query input box to simplify, speed interactions Debanjum Singh Solanky 2022-07-16 19:39:15 +04:00
  • 06b0c720d6 Improve Rendering of Image Search Results in Emacs Debanjum Singh Solanky 2022-07-16 19:31:49 +04:00
  • 28ec9af589 Extract image URL location from response in elisp after API update Debanjum Singh Solanky 2022-07-16 18:43:55 +04:00
  • 47613cba1f Improve Landing Page Look in General and Layout for Mobile Debanjum Singh Solanky 2022-07-16 16:40:53 +04:00
  • cf207d6ebe Add title, heading to the semantic search web interface Debanjum Singh Solanky 2022-07-16 03:44:29 +04:00
  • e0d8398b27 Normalize metadata match score to work better with image match score Debanjum Singh Solanky 2022-07-16 03:39:33 +04:00
  • a3fc82817d Log and continue on image metadata encoding error due to Tensor size mismatch Debanjum Singh Solanky 2022-07-16 03:39:19 +04:00
  • f26d0ddbbd Minor fix to asymmetric search when no entries returned Debanjum Singh Solanky 2022-07-15 23:45:47 +04:00
  • ca3f93e641 Add button on web interface to regenerate embeddings of specified type Debanjum Singh Solanky 2022-07-15 23:35:42 +04:00
  • 231cc91e14 Force reload of images every time user clicks search button Debanjum Singh Solanky 2022-07-15 23:14:41 +04:00
  • a6aef62a99 Create Basic Landing Page to Query Semantic Search and Render Results Debanjum Singh Solanky 2022-07-15 22:07:39 +04:00
  • 4e27ae0577 Ease access to image result for given query by image_search Debanjum Singh Solanky 2022-07-15 23:20:19 +04:00
  • 801e59a20d Allow explicit filters when querying Ledger transactions Debanjum Singh Solanky 2022-07-14 23:41:39 +04:00
  • 0e979587e0 Add configurable filter support to Symmetric Ledger Search Debanjum Singh Solanky 2022-07-14 23:40:41 +04:00
  • 50658453cd Add separate conda environment.yml for osx-arm64 Debanjum Singh Solanky 2022-07-14 23:16:49 +04:00
  • 152e11ac8a Add Date Range Filter to Search Debanjum 2022-07-14 12:15:55 -07:00
  • 85077bc1d1 Handle unparseable date range passed via date filter in query Debanjum Singh Solanky 2022-07-14 22:47:23 +04:00
  • a60de2c02b Include date filter in asymmetic search on music as well Debanjum Singh Solanky 2022-07-14 22:37:17 +04:00
  • e96253a7c1 Add dateparser library to conda environment YAML Debanjum Singh Solanky 2022-07-14 22:29:07 +04:00
  • c3b3e8959d Put entry splitting regex in explicit filter into a variable for code readability Debanjum Singh Solanky 2022-07-14 22:00:10 +04:00
  • 3aac3c7d52 Run explicit filter on raw entry, add more terms to split entries by Debanjum Singh Solanky 2022-07-14 21:54:04 +04:00
  • 7640e2ab0c Wrap attempt to extract dates from entry in try/catch Debanjum Singh Solanky 2022-07-14 21:38:00 +04:00
  • 9de2097182 Fix date filter usage with multi word queries. Simplify date regex Debanjum Singh Solanky 2022-07-14 21:32:58 +04:00
  • 67e9366c0f Minor style fix. Use consistent/standard dates for date_filter tests Debanjum Singh Solanky 2022-07-14 20:06:39 +04:00
  • dcb6fe479e Fix date_filter query, entry in query range check. Add tests for it Debanjum Singh Solanky 2022-07-14 20:01:35 +04:00
  • 011f81fac5 Fix date_filter to handle non overlapping date ranges Debanjum Singh Solanky 2022-07-14 18:53:38 +04:00
  • 70ac35b2a5 Compute Date Range to filter entries to, from Comparators, Dates in Query Debanjum Singh Solanky 2022-07-14 18:20:09 +04:00
  • e6db3e3d00 Prefer Dates From Future only when specific words in date string Debanjum Singh Solanky 2022-07-14 18:13:12 +04:00
  • 4a201d52af Add, test date filter regex and date parsing to get natural date range Debanjum Singh Solanky 2022-07-14 16:47:32 +04:00
  • b54588717f Filter for entries with dates specified by user in query Debanjum Singh Solanky 2022-07-13 18:46:34 +04:00
  • 6356feb637 Make filters applied before semantic search configurable Debanjum 2022-07-13 05:53:02 -07:00
  • b82aef26bf Make filters to apply before semantic search configurable Debanjum Singh Solanky 2022-07-13 16:29:23 +04:00
  • c92789d20a Extract explicit pre-search filter function into a separate module Debanjum Singh Solanky 2022-07-13 16:07:45 +04:00
  • 589bfa9424 Run Explicit Filter on Entries, Embeddings before Semantic Search for Query Debanjum 2022-07-12 13:12:22 -07:00
  • 741fca0e6b Fix asymmetric search test to pass entries returned by query to collate_results Debanjum Singh Solanky 2022-07-12 18:48:49 +04:00
  • 6d7ab50113 Run Explicit Filter on Entries, Embeddings before Semantic Search for Query Debanjum Singh Solanky 2022-07-12 13:58:32 +04:00
  • 36ef37e940 Fix formatting for pytest command sabaimran 2022-07-08 10:18:26 -04:00
  • d6945f4f6b Merge pull request #29 from debanjum/saba/fix-docker-build sabaimran 2022-07-06 21:32:37 -04:00
  • 2eb44c7a64 Correct syntax of memory limit in docker-compose.yml Saba 2022-07-06 20:07:11 -04:00
  • 8bb9a49994 Cleanup Test Asymmetric Search to Fix Test Debanjum Singh Solanky 2022-07-07 01:25:31 +04:00
  • 7bb35ccc7e Run build on PR Saba 2022-07-04 18:09:47 -04:00
  • 07a56c4ab6 Add specific version for Python packages and downgrade miniconda Docker image to potentially fix build issues Saba 2022-07-04 18:01:55 -04:00
  • 0f88abd219 Allocate 8GB of memory to docker container. Adjust path to Dockerfile in Github action Saba 2022-07-04 14:01:59 -04:00
  • 092d0f2f21 Move Dockerfile to project root to avoid permissions issues. Allocate more memory to docker-compose to avoid OOM Saba 2022-07-04 12:33:55 -04:00
  • 7677465f23 Fix passing of device to setup method in /reload, /regenerate API Debanjum Singh Solanky 2022-06-30 01:32:56 +04:00
  • eda4b65ddb Improve Query Speed. Normalize Embeddings, Moving them to Cuda GPU Debanjum Singh Solanky 2022-06-30 00:59:57 +04:00
  • 2f7ef08b11 Add Unit Tests to verify the Reload API functions as desired Debanjum Singh Solanky 2022-06-29 23:09:48 +04:00
  • b89fc2f4ac Add /reload API to reload model embeddings and entries from file Debanjum Singh Solanky 2022-06-29 22:46:17 +04:00
  • f5d6d1e752 Tiny style fix to separate functions by 2 newlines Debanjum Singh Solanky 2022-06-29 23:45:59 +04:00
  • d64bed26f3 Make Docker ignore unnecessary files Debanjum Singh Solanky 2022-06-29 22:29:34 +04:00
  • 85fbe1c42b Normalize org notes path to be relative to home directory Debanjum Singh Solanky 2022-06-28 19:16:11 +04:00
  • 094eaf3fcc Fix minor bugs in OrgNode parser Debanjum Singh Solanky 2022-06-17 19:14:54 +03:00
  • f66192f2a7 Test OrgNode Parsing and Rendering Debanjum Singh Solanky 2022-06-17 19:13:11 +03:00
  • 36495038dd Fix storing parsed CLOSED date in OrgNode Debanjum Singh Solanky 2022-06-17 16:28:56 +03:00
  • 1c5754bf95 Simplify storing Tags in OrgNode object Debanjum Singh Solanky 2022-06-17 16:24:56 +03:00
  • 51a43245d3 Escape square brackets in file+heading based org-mode links Debanjum Singh Solanky 2022-06-17 16:16:42 +03:00
  • 04610f453a Include scheduled date, deadline date and close date in repr of org node Debanjum Singh Solanky 2022-06-17 05:02:02 +03:00
  • 367d7377df Ignore scheduled, closed, deadline time and logbook start, end in org node body Debanjum Singh Solanky 2022-06-17 04:27:05 +03:00
  • b77ccadcba Make property key regex more strict. Property key has to be alphanumeric Debanjum Singh Solanky 2022-06-17 05:03:57 +03:00
  • ac9d746444 Fix Tags extraction in Org Node parser Debanjum Singh Solanky 2022-06-17 04:21:22 +03:00
  • fb86be8cd9 Add ID, File+Heading based Links to Org-Mode Entries Debanjum Singh Solanky 2022-06-17 03:11:11 +03:00