Update README and default folders in docker_sample_config.yml

- Add instruction to using Docker with README
- Use the ./tests/data folder in docker_sample_conifg.yml so it can work right away for users
This commit is contained in:
Saba
2022-01-28 23:20:50 -05:00
parent 52e701b3c2
commit 1ba7fa66e5
2 changed files with 66 additions and 36 deletions

View File

@@ -5,27 +5,47 @@
All data is processed locally. User can interface with semantic-search app via [[./src/interface/emacs/semantic-search.el][Emacs]], API or Commandline All data is processed locally. User can interface with semantic-search app via [[./src/interface/emacs/semantic-search.el][Emacs]], API or Commandline
** Dependencies ** Setup
- Python3
- [[https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links][Miniconda]]
** Install *** Setup using Docker
**** 1. Clone Repository
#+begin_src shell
git clone https://github.com/debanjum/semantic-search && cd semantic-search
#+end_src
**** 2. Configure
Add Content Directories for Semantic Search to Docker-Compose
Update [[./docker-compose.yml][docker-compose.yml]] to mount your images, org-mode notes, ledger/beancount directories
If required, edit config settings in [[./docker_sample_config.yml][docker_sample_config.yml]].
**** 3. Run
#+begin_src shell
docker-compose up -d
#+end_src
*** Setup on Local Machine
**** 1. Install Dependencies
1. Install Python3 [Required[
2. [[https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html][Install Conda]] [Required]
3. Install Exiftool [Optional]
#+begin_src shell
sudo apt-get -y install libimage-exiftool-perl
#+end_src
**** 2. Install Semantic Search
#+begin_src shell #+begin_src shell
git clone https://github.com/debanjum/semantic-search && cd semantic-search git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f environment.yml conda env create -f environment.yml
conda activate semantic-search conda activate semantic-search
#+end_src #+end_src
*** Install Environmental Dependencies **** 3. Configure
#+begin_src shell
sudo apt-get -y install libimage-exiftool-perl
#+end_src
** Configure
Configure application search types and their underlying data source/files in ~sample_config.yml~ Configure application search types and their underlying data source/files in ~sample_config.yml~
Use the ~sample_config.yml~ as reference Use the ~sample_config.yml~ as reference
** Run **** 4. Run
Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML
#+begin_src shell #+begin_src shell
@@ -39,10 +59,20 @@
- *Semantic Search via API* - *Semantic Search via API*
- Query: ~GET~ [[http://localhost:8000/search?q=%22what%20is%20the%20meaning%20of%20life%22][http://localhost:8000/search?q="What is the meaning of life"&t=notes]] - Query: ~GET~ [[http://localhost:8000/search?q=%22what%20is%20the%20meaning%20of%20life%22][http://localhost:8000/search?q="What is the meaning of life"&t=notes]]
- Regenerate Embeddings: ~GET~ [[http://localhost:8000/regenerate][http://localhost:8000/regenerate?t=image]] - Regenerate Embeddings: ~GET~ [[http://localhost:8000/regenerate][http://localhost:8000/regenerate]]
- [[http://localhost:8000/docs][Semantic Search API Docs]] - [[http://localhost:8000/docs][Semantic Search API Docs]]
- *UI to Edit Config*
- [[https://localhost:8000/ui][Config UI]]
** Upgrade ** Upgrade
*** Using Docker
#+begin_src shell
docker-compose up
#+end_src
*** On Local Machine
#+begin_src shell #+begin_src shell
cd semantic-search cd semantic-search
git pull origin master git pull origin master

View File

@@ -20,13 +20,13 @@ services:
# points to the files you want to index. # points to the files you want to index.
# The path of the mounted directory (right hand side), # The path of the mounted directory (right hand side),
# must match the path prefix in your config file. # must match the path prefix in your config file.
- /path/to/notes/:/data/notes/ - ./tests/data/:/data/notes/
- /path/to/photos/:/data/images/ - ./tests/data/:/data/images/
- /path/to/ledger/:/data/ledger/ - ./tests/data/:/data/ledger/
- /path/to/music/:/data/music/ - ./tests/data/:/data/music/
# It's ok if you don't have existing embeddings. # It's ok if you don't have existing embeddings.
# Leave the line as is - an empty volume will be created if it doesn't exist. # You can set this volume to point to an empty folder.
- /path/to/embeddings/:/data/generated/ - ./tests/data/:/data/generated/
# Use 0.0.0.0 to explicitly set the host ip for the service on the container. https://pythonspeed.com/articles/docker-connection-refused/ # Use 0.0.0.0 to explicitly set the host ip for the service on the container. https://pythonspeed.com/articles/docker-connection-refused/
command: --host="0.0.0.0" --port=8000 -c=docker_sample_config.yml command: --host="0.0.0.0" --port=8000 -c=docker_sample_config.yml -vv