klbr/khoj
1
0
Fork 0
mirror of https://github.com/khoaliber/khoj.git synced 2026-03-07 05:40:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Set >=6Gb RAM required for offline chat

Browse Source 
Llama v2 7B with 4bit quantization technically needs ~3.5Gb RAM (7B * 0.5byte), practically a system with 6Gb of RAM should suffice
This commit is contained in:
Debanjum
2023-10-18 12:05:54 -07:00
committed by GitHub
parent 8346e1193c
commit d93395ae48
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/chat.md
View File

@@ -10,7 +10,7 @@
Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive. Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive.
> **System Requirements**: > **System Requirements**:
> - You need at least **16 GB of RAM** and **4 GB of Disk** > - Machine with at least **6 GB of RAM** and **4 GB of Disk** available
> - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required > - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required
> - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times > - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times