Skip to content

Update README and fix deps #185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+22 −23
Open

Update README and fix deps #185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 21 additions & 22 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,32 +20,27 @@ Optimum ExecuTorch enables efficient deployment of transformer models using Meta

## ⚡ Quick Installation

### 1. Create a virtual environment
Install [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) on your machine. Then, create a virtual environment to manage our dependencies.
```
conda create -n optimum-executorch python=3.11
conda activate optimum-executorch
```

### 2. Install optimum-executorch from source
```
git clone https://github.com/huggingface/optimum-executorch.git
cd optimum-executorch
pip install '.[dev]'
To install the latest stable version:
```bash
pip install optimum-executorch
```

- 🔜 Install from pypi coming soon...
<details>
<summary>Other installation options</summary>

### 3. Install dependencies in dev mode

To access every available optimization and experiment with the newest features, run:
```
python install_dev.py
```
```
# Install from source for development in editable mode
pip install -e '.[dev]'

This script will install `executorch`, `torch`, `torchao`, `transformers`, etc. from nightly builds or from source to access the latest models and optimizations.
# Install from source, using the most recent nightly Torch and Transformers dependencies.
# When a new model is released and enabled on Optimum ExecuTorch, it will usually first be
# available here as it will require installing recent Transformers from source.
python install_dev.py

To leave an existing ExecuTorch installation untouched, run `install_dev.py` with `--skip_override_torch` to prevent it from being overwritten.
# Leave existing ExecuTorch installation and other torch dependencies untouched.
python install_dev.py --skip_override_torch
```
</details>

## 🎯 Quick Start

Expand Down Expand Up @@ -127,7 +122,11 @@ Optimum transformer models utilize:
- A **custom KV cache** that uses a custom op for efficient in-place cache update on CPU, boosting performance by **2.5x** compared to default static KV cache.

### Backends Delegation
Currently, **Optimum-ExecuTorch** supports the [XNNPACK Backend](https://pytorch.org/executorch/main/backends-xnnpack.html) for CPU and [CoreML Backend](https://docs.pytorch.org/executorch/stable/backends-coreml.html) for GPU on Apple devices.
Currently, **Optimum-ExecuTorch** supports the following backends:
- [XNNPACK](https://pytorch.org/executorch/main/backends-xnnpack.html) - this is the most supported backend and will work with all models.
- [CoreML](https://docs.pytorch.org/executorch/stable/backends-coreml.html)
- Cuda
- [Metal](https://docs.pytorch.org/executorch/stable/backends/mps/mps-overview.html) - current only available with `executorch >= 1.1.0.dev20251017`. Please separately install this nightly to use the Metal backend.
Copy link
Contributor

@manuelcandales manuelcandales Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is linking to the MPS backend instead. Also, our instructions are to install executorch from source, not the nightly.

Copy link
Collaborator Author

@jackzhxng jackzhxng Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ET needs to be installed from source for Metal?

Copy link
Contributor

@manuelcandales manuelcandales Nov 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not particular to Metal. I would say the same for CUDA. Given that we are actively developing these backends, it makes sense to instruct users to install from source (i.e. using the install_executorch.sh script) which is also how things are being tested in CI.

Copy link
Collaborator Author

@jackzhxng jackzhxng Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, could you suggest some installation instructions? Do we install ET from source before or after Optimum ET, and do we install from tip of main or a specific commit?


For a comprehensive overview of all backends supported by ExecuTorch, please refer to the [ExecuTorch Backend Overview](https://pytorch.org/executorch/main/backends-overview.html).

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ dependencies = [
"executorch>=1.0.0",
"transformers==4.56.1",
"pytorch-tokenizers>=1.0.1",
"accelerate>=0.26.0",
]

[project.optional-dependencies]
Expand All @@ -46,6 +45,7 @@ dev = [
"tiktoken",
"black~=23.1",
"ruff==0.4.4",
"mistral-common",
]

[project.urls]
Expand Down
Loading