Skip to content

Commit fc21ce3

Browse files
authored
Update deploy-openai-llm-byoc.md
1 parent 72ee1a2 commit fc21ce3

File tree

1 file changed

+7
-9
lines changed

1 file changed

+7
-9
lines changed

LLM/deploy-openai-llm-byoc.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,24 @@
11
# Deploy OpenAI open-source models
22

3-
This guide demonstrates how to deploy and perform inference using OCI Data Science Service. In this example, we will use a model downloaded from Hugging Face specifically, [openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) from OpenAI.
4-
5-
3+
This guide demonstrates how to deploy and perform inference using OCI Data Science Service. In this example, we will use a model downloaded from Hugging Face, specifically [openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) from OpenAI.
64

75
## Required IAM Policies
86

97
Add these [policies](https://github.com/oracle-samples/oci-data-science-ai-samples/tree/main/model-deployment/containers/llama2#required-iam-policies) to grant access to OCI services.
108

119
## Setup
1210

13-
Create a data science notebook session with at least 400GB of space. We will use notebook session to -
11+
Create a data science notebook session with at least 400GB of space. We will use a notebook session to:
1412
1. Download model weights
1513
2. Create Model Catalog entry
1614
3. Deploy the model
1715

18-
To prepare inference container, we will use local laptop since this step requires docker commmands. The notebook session does not come with the docker tooling.
16+
To prepare the inference container, we will use local laptop since this step requires Docker commands. The notebook session does not come with the docker tooling.
1917

2018

2119
# Prepare Inference container
2220

23-
vLLM is an easy-to-use library for LLM inference and server. You can get the container image from [DockerHub](https://hub.docker.com/r/vllm/vllm-openai/tags).
21+
vLLM is an easy-to-use library for LLM inference and serving. You can get the container image from [DockerHub](https://hub.docker.com/r/vllm/vllm-openai/tags).
2422

2523
Following commands are to run on your laptop -
2624

@@ -41,7 +39,7 @@ You may need to docker login to the Oracle Cloud Container Registry (OCIR) first
4139
docker login -u '<tenant-namespace>/<username>' <region>.ocir.io
4240
```
4341

44-
If your tenancy is federated with Oracle Identity Cloud Service, use the format <tenancy-namespace>/oracleidentitycloudservice/<username>. You can then push the container image to the OCI Registry
42+
If your tenancy is federated with Oracle Identity Cloud Service, use the format <tenancy-namespace>/oracleidentitycloudservice/<username>. You can then push the container image to the OCI Registry:
4543

4644
```shell
4745
docker tag vllm/vllm-openai:gptoss -t <region>.ocir.io/<tenancy>/vllm-odsc/vllm-openai:gptoss
@@ -78,7 +76,7 @@ wget -P models/gpt-oss-120b https://openaipublic.blob.core.windows.net/encodings
7876
```
7977
## Upload Model to OCI Object Storage
8078

81-
**Note**: **The bucket has to be versioned bucket**
79+
**Note**: **The bucket has to be a versioned bucket**
8280

8381
```shell
8482
oci os object bulk-upload --src-dir models/gpt-oss-120b --prefix gpt-oss-120b/ -bn <bucket_name> -ns <bucket_namespace> --auth "resource_principal"
@@ -154,7 +152,7 @@ from ads.model.deployment import (
154152
## Setup Model Deployment Infrastructure
155153

156154
```python
157-
container_image = "<region>.ocir.io/<tenancy>/vllm-odsc/vllm-openai:gptoss" # name given to vllm image pushed to oracle container registry
155+
container_image = "<region>.ocir.io/<tenancy>/vllm-odsc/vllm-openai:gptoss" # name given to vllm image pushed to Oracle container registry
158156
```
159157

160158
```python

0 commit comments

Comments
 (0)