Debanjum Singh Solanky 2f7a6af56a Support incremental update of org-mode entries and embeddings ...

- What
  - Hash the entries and compare to find new/updated entries
  - Reuse embeddings encoded for existing entries
  - Only encode embeddings for updated or new entries
  - Merge the existing and new entries and embeddings to get the updated
    entries, embeddings

- Why
  - Given most note text entries are expected to be unchanged
    across time. Reusing their earlier encoded embeddings should
    significantly speed up embeddings updates
  - Previously we were regenerating embeddings for all entries,
    even if they had existed in previous runs

2022-09-10 20:58:33 +03:00

1.9 KiB

Raw Blame History

View Raw