From d93395ae48d668ff372ac8fc4dd4b46f950fa8bc Mon Sep 17 00:00:00 2001 From: Debanjum Date: Wed, 18 Oct 2023 12:05:54 -0700 Subject: [PATCH] Set >=6Gb RAM required for offline chat Llama v2 7B with 4bit quantization technically needs ~3.5Gb RAM (7B * 0.5byte), practically a system with 6Gb of RAM should suffice --- docs/chat.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/chat.md b/docs/chat.md index eeca3132..b900d052 100644 --- a/docs/chat.md +++ b/docs/chat.md @@ -10,7 +10,7 @@ Offline chat stays completely private and works without internet. But it is slower, lower quality and more compute intensive. > **System Requirements**: -> - You need at least **16 GB of RAM** and **4 GB of Disk** +> - Machine with at least **6 GB of RAM** and **4 GB of Disk** available > - A CPU supporting [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) is required > - A Mac M1+ or [Vulcan supported GPU](https://vulkan.gpuinfo.org/) should significantly speed up chat response times