9536358d34
Fix key error model_name issue by upgrade sentence-transformers version
...
Refer to https://github.com/UKPLab/sentence-transformers/issues/1241
Also user verbose flag passed through function parameters in image_search
2021-12-11 11:58:19 -05:00
ce7a751e6b
Fix passing verbose flag down in symmetric_ledger.py
2021-12-11 11:36:32 -05:00
d65190c3ee
Update unit tests, files with removing model suffix to config types
2021-12-09 08:50:38 -05:00
0ac1e5f372
Summarize chat logs and notes returned by semantic search via /chat API
2021-12-08 02:34:07 +05:30
76e9e9da2f
Update unit tests to use the new BaseModel types
2021-12-05 09:31:39 -05:00
9b16cdbb41
Use past tense for verbose log
2021-12-04 11:45:44 -05:00
10e4065e05
Consolidate the search config models and pass verbose as a top level flag
2021-12-04 11:43:48 -05:00
43e647835b
Append Model Suffixed to config models
2021-12-04 10:51:21 -05:00
e068968b35
Update imports for raw config models in config.py
2021-12-04 10:44:55 -05:00
4d6284b0af
Remove Test suffix from Config models
2021-12-04 10:44:13 -05:00
7fcc8d2cef
Add null check for processor config
2021-12-04 10:11:00 -05:00
7ca4fc3453
Resolve mrege conflicts with updated processor conversation data model
2021-11-28 16:22:52 -05:00
87a6c2d716
Use parse_obj instead of parse_raw as incoming data is in dict
2021-11-28 14:34:32 -05:00
5d50487d83
Linting
...
New line at end of config.html
Remove debug print statement
2021-11-28 13:32:56 -05:00
6f466c8d99
Use global config and add a regenerate button to the config ui' && git push
2021-11-28 13:28:22 -05:00
34d1e4199c
Use alias generator when deserializing the config file
2021-11-28 13:05:48 -05:00
19b81e82f0
Write back to the raw config.yml file on update
2021-11-28 12:34:40 -05:00
8837b02de6
dump updated config to a yaml file
2021-11-28 12:26:07 -05:00
5b80b87379
Streamline None checking in initialize_search
2021-11-28 12:05:04 -05:00
bf8ae31e6a
Streamline None checking in initialize_search
2021-11-28 11:59:45 -05:00
da52433d89
Update to re-use the raw config base models in config.py as well
2021-11-28 11:57:33 -05:00
6292fe4481
Update to re-use the raw config base models in config.py as well
2021-11-28 11:57:13 -05:00
311c4b7e7b
Working API request body parsing to /post config!
2021-11-28 11:16:33 -05:00
66183cc298
Working API request body parsing to /post config!
2021-11-28 11:12:26 -05:00
5cd920544d
Add GPT method to summarize notes and chat logs
2021-11-28 13:08:05 +05:30
1785047ea6
Improve understand primer and load understand response as dict
2021-11-28 13:04:16 +05:30
64645c3ac1
Begin type checking/input validation effort
2021-11-27 21:47:56 -05:00
9a0264b7fc
Add a dummy POST config endpoint, integrate with editable UI
2021-11-27 20:36:03 -05:00
f3b03ea5b7
Make raw data reactive to changes
2021-11-27 19:17:15 -05:00
67c3cd7372
Wire up GPT understand method to /chat API. Log conversation metadata too
2021-11-28 00:04:39 +05:30
3db06eee3f
Basic example of serving conifg as JSON and retriving on button click
2021-11-27 10:49:33 -05:00
3d4471e107
Merge branch 'master' of github.com:debanjum/semantic-search into saba/configui
2021-11-27 08:52:48 -05:00
ccfb97e1a7
Wire up minimal conversation processor. Expose it over /chat API endpoint
...
Ensure conversation history persists across application restart
2021-11-27 18:12:01 +05:30
a99b4b3434
Make conversation processor configurable
2021-11-27 18:12:01 +05:30
d4e1120b22
Add GPT based conversation processor to understand intent and converse with user
...
- Allow conversing with user using GPT's contextually aware, generative capability
- Extract metadata, user intent from user's messages using GPT's general understanding
2021-11-27 18:12:01 +05:30
baee52648d
Set up basic ui page with no functionality
2021-11-26 14:51:11 -05:00
46661b3057
Ensure top_k never more than total entries to run symmetric search on
2021-11-16 11:32:21 -08:00
8c858d1a94
Reduce symmetric search results for cross-encoder to re-rank to improve search speed
2021-11-16 11:31:19 -08:00
f3fd5ae978
Improve code comments. Do not import unused modules in asymmetric search
2021-11-17 00:58:31 +05:30
8cf2465e8e
Ensure top_k never more than total entries to search from
2021-11-17 00:56:31 +05:30
4d37ace3d6
Reduce search results for cross-encoder to re-rank to improve search speed
...
Search time on my notes reduced from 14s to 4s. Cross-encoder
re-ranking step takes majority time, not the cosine similarity search
2021-11-17 00:50:28 +05:30
1832e418e5
Use raw string for regex in orgnode to fix deprecation warning
2021-10-02 17:38:31 -07:00
f59e321419
Update CLIP model load path
2021-10-02 16:50:06 -07:00
c47a8cdf16
Allow configuring host, port or unix socket of server via CLI
2021-10-02 16:16:33 -07:00
516f28b082
Merge branch 'master' of github.com:debanjum/semantic-search
2021-09-30 04:17:32 -07:00
d2905c4be6
Move tests out to project root. Use absolute import in project
...
tests/ directory in project root is more standard.
Just had to use absolute path for internal module imports to get it to
work
2021-09-30 04:12:14 -07:00
58bb420f69
Fix image_metadata argument ordering bug. Add E2E image search test
...
- Image search test seems a little flaky
- Interchanged argument was causing inaccurate results earlier
2021-09-30 03:30:47 -07:00
d5597442f4
Modularize Code. Wrap Search, Model Config in Classes. Add Tests
...
Details
- Rename method query_* to query in search_types for standardization
- Wrapping Config code in classes simplified mocking test config
- Reduce args beings passed to a function by passing it as single
argument wrapped in a class
- Minimize setup in main.py:__main__. Put most of it into functions
These functions can be mocked if required in tests later too
Setup Flow:
CLI_Args|Config_YAML -> (Text|Image)SearchConfig -> (Text|Image)SearchModel
2021-09-30 02:04:04 -07:00
f4dd9cd117
Use type specific model for other search types too. Expose them via SearchModels
...
- Wrap Image, Music, Ledger search into the type of SearchModel they use
Similar to what was done for notes model by wrapping it's config
into an AsymmetricSearchModel.
- Use the uber wrapper class to expose all type specific search models
2021-09-29 21:09:42 -07:00
352d2930ee
Use multiple threads to generate model embeddings. Other minor formating
2021-09-29 20:47:58 -07:00
Saba