You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Local apps: Add RyzenAI models to Lemonade (#1729)
# Description
This PR adds models labeled `ryzenai-hybrid` and `ryzenai-npu` to the
Lemonade app instructions. This makes Lemonade the first Hugging Face
app to offer instructions on how to run AMD NPU-accelerated models.
Once this PR is merged, we intend add the proposed tags to all AMD
compatible models.
Currently, two models are labeled to test this integration:
`amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-fp16-onnx-hybrid` and
`amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-bf16-onnx-ryzen-strix`.
## Note on implementation
@krampstudio during our last PR you made a change called "multi line
snippet and format" as shown
[here](cafd218).
Breaking the lines with `\` as you proposed caused the instructions to
not work for Windows users (they had to manually remove those slashes).
I reverted this change here. Please let me know if that is ok or if
those are absolutely needed for the content to be properly displayed.
## How it looks
### Hybrid model example
```
1. Pull the model
-----------------
Setup:
# Download Lemonade from https://lemonade-server.ai/
Content:
lemonade-server pull user.Phi-3-mini-4k-instruct-Hybrid --checkpoint amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-fp16-onnx-hybrid --recipe oga-hybrid
# Note: If you installed from source, use the lemonade-server-dev command instead.
2. Run and chat with the model (requires RyzenAI 300 series)
------------------------------------------------------------
Content:
lemonade-server run user.Phi-3-mini-4k-instruct-Hybrid
3. List all available models
----------------------------
Content:
lemonade-server list
```
### NPU model example
```
1. Pull the model
-----------------
Setup:
# Download Lemonade from https://lemonade-server.ai/
Content:
lemonade-server pull user.Phi-3-mini-4k-instruct-NPU --checkpoint amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-bf16-onnx-ryzen-strix --recipe oga-npu
# Note: If you installed from source, use the lemonade-server-dev command instead.
2. Run and chat with the model (requires RyzenAI 300 series)
------------------------------------------------------------
Content:
lemonade-server run user.Phi-3-mini-4k-instruct-NPU
3. List all available models
----------------------------
Content:
lemonade-server list
```
### GGUF model example
```
1. Pull the model
-----------------
Setup:
# Download Lemonade from https://lemonade-server.ai/
Content:
lemonade-server pull user.gpt-oss-20b-GGUF --checkpoint unsloth/gpt-oss-20b-GGUF:{{QUANT_TAG}} --recipe llamacpp
# Note: If you installed from source, use the lemonade-server-dev command instead.
2. Run and chat with the model
------------------------------
Content:
lemonade-server run user.gpt-oss-20b-GGUF
3. List all available models
----------------------------
Content:
lemonade-server list
```
@Vaibhavs10 Please review :)
---------
Co-authored-by: Bertrand CHEVRIER <bertrand@huggingface.co>
0 commit comments