Skip to content

Commit 32f5501

Browse files
authored
Merge pull request #667 from YashPandit4u/homogeneous-model-group-deployment
Adding sample homogeneous model group code.
2 parents 2c1b901 + 35f8903 commit 32f5501

File tree

2 files changed

+1290
-0
lines changed

2 files changed

+1290
-0
lines changed
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Homogeneous Model Group Deployment
2+
3+
Deploy multiple models together using OCI Data Science Model Groups for efficient multi-model serving from a single endpoint.
4+
5+
## What This Notebook Covers
6+
7+
- Building scikit-learn pipelines and registering models to the Model Catalog
8+
- Creating homogeneous model groups with multiple models
9+
- Deploying model groups to a single deployment endpoint
10+
- Performing inference using model keys
11+
- Live updates without downtime
12+
13+
## Prerequisites
14+
15+
- OCI Data Science project with Model Catalog access
16+
- Python packages: `ads`, `oci` (≥2.163.0), `scikit-learn`, `pandas`, `requests`
17+
- Environment variables: `PROJECT_OCID`, `NB_SESSION_COMPARTMENT_OCID`
18+
- OCI config file or resource principal authentication
19+
- Log group ID and log ID for deployment logging
20+
21+
## Quick Start
22+
23+
1. Open `homogeneous_model_group_deployment_example.ipynb` in your OCI Data Science notebook session
24+
2. Set `project_id`, `compartment_id`, `access_log_group_id`, and `log_id`
25+
3. Run cells sequentially
26+
27+
## Inference Example
28+
29+
Specify the model key in request headers to route to a specific model:
30+
31+
```python
32+
headers = {
33+
'Content-Type': 'application/json',
34+
'model-key': 'svm1' # Routes to specific model
35+
}
36+
response = requests.post(endpoint, json=data, auth=auth, headers=headers).json()
37+
```
38+
39+
## Key Points
40+
41+
- **Homogeneous groups**: Models share the same framework, scoring script, and runtime
42+
- **Model keys**: Each model has a unique inference key (e.g., "svm1", "svm2")
43+
- **Common artifact**: Requires a shared `score.py` and `runtime.yaml` for dynamic model loading
44+
- **Live updates**: Update deployments without downtime
45+
- **OCI SDK**: Some Model Group features require OCI SDK directly (not fully supported in ADS SDK yet)

0 commit comments

Comments
 (0)