Spaces:
Paused
Paused
Vaibhav Srivastav
commited on
Add steps to install from source for llama.cpp (#1396)
Browse files* Add steps to install from source for llama.cpp
* Formatting.
README.md
CHANGED
|
@@ -37,9 +37,22 @@ You can quickly start a locally running chat-ui & LLM text-generation server tha
|
|
| 37 |
|
| 38 |
**Step 1 (Start llama.cpp server):**
|
| 39 |
|
|
|
|
|
|
|
| 40 |
```bash
|
| 41 |
# install llama.cpp
|
| 42 |
brew install llama.cpp
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
# start llama.cpp server (using hf.co/microsoft/Phi-3-mini-4k-instruct-gguf as an example)
|
| 44 |
llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf -c 4096
|
| 45 |
```
|
|
|
|
| 37 |
|
| 38 |
**Step 1 (Start llama.cpp server):**
|
| 39 |
|
| 40 |
+
Install llama.cpp w/ brew (for Mac):
|
| 41 |
+
|
| 42 |
```bash
|
| 43 |
# install llama.cpp
|
| 44 |
brew install llama.cpp
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
or [build directly from the source](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md) for your target device:
|
| 48 |
+
|
| 49 |
+
```
|
| 50 |
+
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
Next, start the server with the [LLM of your choice](https://huggingface.co/models?library=gguf):
|
| 54 |
+
|
| 55 |
+
```bash
|
| 56 |
# start llama.cpp server (using hf.co/microsoft/Phi-3-mini-4k-instruct-gguf as an example)
|
| 57 |
llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf -c 4096
|
| 58 |
```
|