klbr/khoj
1
0
Fork 0
mirror of https://github.com/khoaliber/khoj.git synced 2026-03-09 21:29:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Process text content files in sorted order for stable indexing

Browse Source 
- Image search already uses a sorted list of images to process
- Prevents index of entries to desync when entries, embeddings
  generated by a separate server/app instance
This commit is contained in:
Debanjum Singh Solanky
2022-09-12 11:02:05 +03:00
parent a701ad08b9
commit 536f03af8f
6 changed files with 12 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tests/test_markdown_to_jsonl.py
View File

@@ -86,7 +86,7 @@ def test_get_markdown_files(tmp_path):
create_file(tmp_path, filename="not-included-markdown.md")
create_file(tmp_path, filename="not-included-text.txt")
expected_files = set(map(str, [group1_file1, group1_file2, group2_file1, group2_file2, file1]))
expected_files = sorted(map(str, [group1_file1, group1_file2, group2_file1, group2_file2, file1]))
# Setup input-files, input-filters
input_files = [tmp_path / 'notes.md']