Debanjum Singh Solanky 826f9dc054 Drop long words from compiled entries to be within max token limit of models ...

Long words (>500 characters) provide less useful context to models.

Dropping very long words allow models to create better embeddings by
passing more of the useful context from the entry to the model

2023-01-07 23:13:56 -03:00

..

data

Do not version API. Premature given current state of the codebase

2022-10-08 16:32:46 +03:00

__init__.py

Move tests out to project root. Use absolute import in project

2021-09-30 04:12:14 -07:00

conftest.py

Regenerate initial model in asymmetric reload test to reduce flakyness

2022-12-25 21:36:15 -03:00

test_beancount_to_jsonl.py

Use Base TextToJsonl class to standardize <text>_to_jsonl processors

2022-09-16 00:53:11 +03:00

test_chatbot.py

Fix the user intent extraction prompt for GPT. Clean up chatbot test

2022-01-12 10:36:01 -05:00

test_cli.py

Clean-up generated file after image search test run

2022-09-10 21:43:31 +03:00

test_client.py

Do not version API. Premature given current state of the codebase

2022-10-08 16:32:46 +03:00

test_date_filter.py

Remove unused imports, `embeddings' variable from text search tests

2022-10-08 12:06:05 +03:00

test_file_filter.py

Remove unused imports, `embeddings' variable from text search tests

2022-10-08 12:06:05 +03:00

test_helpers.py

Create LRU helper class for caching

2022-09-04 16:31:46 +03:00

test_image_search.py

Remove unused imports, `embeddings' variable from text search tests

2022-10-08 12:06:05 +03:00

test_markdown_to_jsonl.py

Use Base TextToJsonl class to standardize <text>_to_jsonl processors

2022-09-16 00:53:11 +03:00

test_org_to_jsonl.py

Drop long words from compiled entries to be within max token limit of models

2023-01-07 23:13:56 -03:00

test_orgnode.py

Fix OrgNode render of entries with property drawers and empty body

2022-09-11 16:09:19 +03:00

test_text_search.py

Fix comments, use minimal test case, regenerate test index, merge debug logs

2022-12-25 22:33:04 -03:00

test_word_filter.py

Remove unused imports, `embeddings' variable from text search tests

2022-10-08 12:06:05 +03:00