|
| 1 | +# Homogeneous Model Group Deployment |
| 2 | + |
| 3 | +Deploy multiple models together using OCI Data Science Model Groups for efficient multi-model serving from a single endpoint. |
| 4 | + |
| 5 | +## What This Notebook Covers |
| 6 | + |
| 7 | +- Building scikit-learn pipelines and registering models to the Model Catalog |
| 8 | +- Creating homogeneous model groups with multiple models |
| 9 | +- Deploying model groups to a single deployment endpoint |
| 10 | +- Performing inference using model keys |
| 11 | +- Live updates without downtime |
| 12 | + |
| 13 | +## Prerequisites |
| 14 | + |
| 15 | +- OCI Data Science project with Model Catalog access |
| 16 | +- Python packages: `ads`, `oci` (≥2.163.0), `scikit-learn`, `pandas`, `requests` |
| 17 | +- Environment variables: `PROJECT_OCID`, `NB_SESSION_COMPARTMENT_OCID` |
| 18 | +- OCI config file or resource principal authentication |
| 19 | +- Log group ID and log ID for deployment logging |
| 20 | + |
| 21 | +## Quick Start |
| 22 | + |
| 23 | +1. Open `homogeneous_model_group_deployment_example.ipynb` in your OCI Data Science notebook session |
| 24 | +2. Set `project_id`, `compartment_id`, `access_log_group_id`, and `log_id` |
| 25 | +3. Run cells sequentially |
| 26 | + |
| 27 | +## Inference Example |
| 28 | + |
| 29 | +Specify the model key in request headers to route to a specific model: |
| 30 | + |
| 31 | +```python |
| 32 | +headers = { |
| 33 | + 'Content-Type': 'application/json', |
| 34 | + 'model-key': 'svm1' # Routes to specific model |
| 35 | +} |
| 36 | +response = requests.post(endpoint, json=data, auth=auth, headers=headers).json() |
| 37 | +``` |
| 38 | + |
| 39 | +## Key Points |
| 40 | + |
| 41 | +- **Homogeneous groups**: Models share the same framework, scoring script, and runtime |
| 42 | +- **Model keys**: Each model has a unique inference key (e.g., "svm1", "svm2") |
| 43 | +- **Common artifact**: Requires a shared `score.py` and `runtime.yaml` for dynamic model loading |
| 44 | +- **Live updates**: Update deployments without downtime |
| 45 | +- **OCI SDK**: Some Model Group features require OCI SDK directly (not fully supported in ADS SDK yet) |
0 commit comments