Use app LRU, not functools LRU decorator, to cache search results in router

- Provides more control to invalidate cache on update to entries, embeddings
- Allows logging when results are being returned from cache etc
- FastAPI, Swagger API docs look better as the `search' controller not
  wrapped in generically named function when using functools LRU decorator
This commit is contained in:
Debanjum Singh Solanky
2022-09-12 09:28:49 +03:00
parent c6fa09d8fc
commit 940c8fac8c
3 changed files with 16 additions and 3 deletions

View File

@@ -6,6 +6,7 @@ from pathlib import Path
# Internal Packages
from src.utils.config import SearchModels, ProcessorConfigModel
from src.utils.helpers import LRU
from src.utils.rawconfig import FullConfig
# Application Global State
@@ -17,6 +18,7 @@ verbose: int = 0
host: str = None
port: int = None
cli_args = None
query_cache = LRU()
if torch.cuda.is_available():
# Use CUDA GPU