Skip to content

Commit fbad796

Browse files
viniciusdsmellogustavocidornelas
authored andcommitted
feat(tracing): add OCI Generative AI tracing examples and documentation
- Introduced a comprehensive Jupyter notebook `oci_genai_tracing.ipynb` demonstrating the integration of Oracle OCI Generative AI with Openlayer tracing, covering non-streaming and streaming chat completions, advanced parameter configurations, and error handling. - Added a simple Python script `simple_oci_example.py` for quick testing of the OCI Generative AI tracer with Openlayer integration. - Created a README file to provide an overview, prerequisites, usage instructions, and supported models for the OCI tracing examples. - Enhanced the `__init__.py` file to include the new `trace_oci_genai` function for easier access to the OCI tracing functionality. - Ensured all new files adhere to coding standards with comprehensive type annotations and Google-style docstrings for clarity and maintainability.
1 parent 7e0621f commit fbad796

File tree

4 files changed

+724
-0
lines changed

4 files changed

+724
-0
lines changed

examples/tracing/oci/README.md

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Oracle OCI Generative AI Tracing with Openlayer
2+
3+
This directory contains examples for integrating Oracle Cloud Infrastructure (OCI) Generative AI with Openlayer tracing.
4+
5+
## Overview
6+
7+
Oracle OCI Generative AI is a fully managed service that provides state-of-the-art, customizable large language models (LLMs) through a single API. The Openlayer integration allows you to automatically trace and monitor all interactions with OCI Generative AI models.
8+
9+
## Prerequisites
10+
11+
1. **OCI Account**: Access to Oracle Cloud Infrastructure with Generative AI service enabled
12+
2. **OCI Configuration**: Properly configured OCI CLI or config file
13+
3. **Python Packages**:
14+
```bash
15+
pip install oci openlayer
16+
```
17+
18+
## Files
19+
20+
### `oci_genai_tracing.ipynb`
21+
Comprehensive Jupyter notebook demonstrating:
22+
- Basic non-streaming chat completions
23+
- Streaming chat completions
24+
- Advanced parameter configuration
25+
- Error handling
26+
- Multi-turn conversations
27+
28+
### `simple_oci_example.py`
29+
Simple Python script for quick testing:
30+
```bash
31+
export OCI_COMPARTMENT_ID="ocid1.compartment.oc1..your-actual-ocid"
32+
python simple_oci_example.py
33+
```
34+
35+
## Quick Start
36+
37+
### 1. Configure OCI
38+
39+
Set up your OCI configuration using one of these methods:
40+
41+
**Option A: OCI CLI Setup**
42+
```bash
43+
oci setup config
44+
```
45+
46+
**Option B: Environment Variables**
47+
```bash
48+
export OCI_CONFIG_FILE="~/.oci/config"
49+
export OCI_CONFIG_PROFILE="DEFAULT"
50+
```
51+
52+
**Option C: Instance Principal** (when running on OCI compute)
53+
```python
54+
from oci.auth.signers import InstancePrincipalsSecurityTokenSigner
55+
config = {}
56+
signer = InstancePrincipalsSecurityTokenSigner()
57+
```
58+
59+
### 2. Basic Usage
60+
61+
```python
62+
import oci
63+
from oci.generative_ai_inference import GenerativeAiInferenceClient
64+
from oci.generative_ai_inference.models import ChatDetails, GenericChatRequest, Message
65+
from openlayer.lib.integrations import trace_oci_genai
66+
67+
# Configure OCI client
68+
config = oci.config.from_file()
69+
client = GenerativeAiInferenceClient(
70+
config=config,
71+
service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com"
72+
)
73+
74+
# Apply Openlayer tracing
75+
traced_client = trace_oci_genai(client)
76+
77+
# Make a request
78+
chat_request = GenericChatRequest(
79+
messages=[Message(role="user", content="Hello, AI!")],
80+
model_id="cohere.command-r-plus",
81+
max_tokens=100,
82+
temperature=0.7
83+
)
84+
85+
chat_details = ChatDetails(
86+
compartment_id="your-compartment-ocid",
87+
chat_request=chat_request
88+
)
89+
90+
response = traced_client.chat(chat_details, inference_id="my-custom-id")
91+
```
92+
93+
## Supported Models
94+
95+
The integration supports all OCI Generative AI models including:
96+
97+
### Cohere Models
98+
- `cohere.command-r-16k` - 16K context window
99+
- `cohere.command-r-plus` - Enhanced capabilities
100+
101+
### Meta Llama Models
102+
- `meta.llama-3.1-70b-instruct` - 70B parameters, 128K context
103+
- `meta.llama-3.1-405b-instruct` - 405B parameters, largest available
104+
105+
## Features Traced
106+
107+
The Openlayer integration automatically captures:
108+
109+
-**Request Details**: Model ID, parameters, messages
110+
-**Response Data**: Generated content, token usage
111+
-**Performance Metrics**: Latency, time to first token (streaming)
112+
-**Error Information**: When requests fail
113+
-**Custom Inference IDs**: For request tracking
114+
-**Model Parameters**: Temperature, top_p, max_tokens, etc.
115+
116+
## Streaming Support
117+
118+
Both streaming and non-streaming requests are fully supported:
119+
120+
```python
121+
# Non-streaming
122+
chat_request = GenericChatRequest(..., is_stream=False)
123+
response = traced_client.chat(chat_details)
124+
125+
# Streaming
126+
chat_request = GenericChatRequest(..., is_stream=True)
127+
for chunk in traced_client.chat(chat_details):
128+
print(chunk.data.choices[0].delta.content, end='')
129+
```
130+
131+
## Configuration Options
132+
133+
### OCI Endpoints by Region
134+
- **US East (Ashburn)**: `https://inference.generativeai.us-ashburn-1.oci.oraclecloud.com`
135+
- **US West (Phoenix)**: `https://inference.generativeai.us-phoenix-1.oci.oraclecloud.com`
136+
- **UK South (London)**: `https://inference.generativeai.uk-london-1.oci.oraclecloud.com`
137+
- **Germany Central (Frankfurt)**: `https://inference.generativeai.eu-frankfurt-1.oci.oraclecloud.com`
138+
139+
### Model Parameters
140+
```python
141+
GenericChatRequest(
142+
messages=[...],
143+
model_id="cohere.command-r-plus",
144+
max_tokens=500, # Maximum tokens to generate
145+
temperature=0.7, # Creativity (0.0-1.0)
146+
top_p=0.8, # Nucleus sampling
147+
top_k=40, # Top-k sampling
148+
frequency_penalty=0.2, # Reduce repetition
149+
presence_penalty=0.1, # Encourage new topics
150+
stop=["\n\n"], # Stop sequences
151+
is_stream=True # Enable streaming
152+
)
153+
```
154+
155+
## Error Handling
156+
157+
The integration gracefully handles errors and traces them:
158+
159+
```python
160+
try:
161+
response = traced_client.chat(chat_details)
162+
except oci.exceptions.ServiceError as e:
163+
print(f"OCI Service Error: {e}")
164+
except Exception as e:
165+
print(f"Unexpected error: {e}")
166+
# All errors are automatically traced by Openlayer
167+
```
168+
169+
## Best Practices
170+
171+
1. **Use Custom Inference IDs**: For better tracking and debugging
172+
2. **Set Appropriate Timeouts**: For long-running requests
173+
3. **Monitor Token Usage**: To manage costs
174+
4. **Handle Rate Limits**: Implement retry logic
175+
5. **Secure Credentials**: Use IAM roles and policies
176+
177+
## Troubleshooting
178+
179+
### Common Issues
180+
181+
**Config File Not Found**
182+
```bash
183+
oci setup config
184+
```
185+
186+
**Authentication Errors**
187+
```bash
188+
oci iam user get --user-id $(oci iam user list --query 'data[0].id' --raw-output)
189+
```
190+
191+
**Service Unavailable**
192+
- Check if Generative AI is available in your region
193+
- Verify compartment OCID is correct
194+
- Ensure proper IAM permissions
195+
196+
**Import Errors**
197+
```bash
198+
pip install --upgrade oci openlayer
199+
```
200+
201+
## Support
202+
203+
- **OCI Generative AI Documentation**: [docs.oracle.com](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm)
204+
- **Openlayer Documentation**: [openlayer.com/docs](https://openlayer.com/docs)
205+
- **OCI Python SDK**: [github.com/oracle/oci-python-sdk](https://github.com/oracle/oci-python-sdk)
206+
207+
## License
208+
209+
This integration follows the same license as the main Openlayer project.

0 commit comments

Comments
 (0)