Skip to content

Commit 4ea5535

Browse files
author
Fede Kamelhar
committed
feat: Adding portkey.ai gateway as a custom model
feat: Adding portkey.ai gateway as a custom model
1 parent bb34e7c commit 4ea5535

File tree

2 files changed

+65
-0
lines changed

2 files changed

+65
-0
lines changed
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Portkey
2+
3+
[Portkey](https://docs.portkey.ai) is a gateway platform that enables seamless integration with multiple language model providers such as OpenAI, Anthropic, and Amazon Bedrock. The Strands Agents SDK includes a Portkey-based provider, allowing you to run agents across OpenAI-compatible models (i.e., those using OpenAI’s API schema) and other supported models via a unified interface.
4+
5+
## Installation
6+
7+
Portkey is configured as an optional dependency in Strands Agents. To install, run:
8+
9+
```bash
10+
pip install 'strands-agents[portkey]'
11+
```
12+
13+
## Usage
14+
15+
After installing `portkey`, you can import and initialize the Strands Agents' Portkey provider as follows:
16+
17+
```python
18+
from strands import Agent
19+
from strands.models.portkey import PortkeyModel
20+
from strands_tools import calculator
21+
22+
# Portkey for all models
23+
model = PortkeyModel(
24+
api_key="<PORTKEY_API_KEY>",
25+
model_id="anthropic.claude-3-5-sonnet-20241022-v2:0", #Example
26+
virtual_key="<BEDROCK_VIRTUAL_KEY>", # Required for providers like Bedrock. See https://docs.portkey.ai for other provider-specific notes.
27+
provider="bedrock", # You can set the provider to 'anthropic', 'bedrock', 'openai' depending on your model and API setup.
28+
base_url="http://portkey-service-gateway.service.prod.example.com/v1",
29+
)
30+
31+
agent = Agent(model=model, tools=[calculator])
32+
# You can also add tools like web_search, code_interpreter, etc.
33+
response = agent("What is 2+2")
34+
print(response)
35+
```
36+
37+
38+
## Configuration
39+
40+
### Client Configuration
41+
42+
The `client_args` configure the underlying OpenAI client. For a complete list of available arguments, please refer to the OpenAI [source](https://github.com/openai/openai-python).
43+
44+
### Model Configuration
45+
46+
The `model_config` configures the underlying model selected for inference. The supported configurations are:
47+
48+
| Parameter | Description | Example | Options |
49+
|-------------------|--------------------------------------|--------------------------------------------|------------------------------------------------------------------------|
50+
| `model_id` | ID of a model to use | `anthropic.claude-3-5-sonnet-20241022-v2:0` | [reference](https://docs.portkey.ai/docs/llm-routing/overview) |
51+
| `base_url` | Base URL for Portkey service | `http://portkey-service-gateway.service.prod.example.com/v1` | [reference](https://portkey-ai.com/docs) |
52+
| `provider` | Model provider | `bedrock` | `openai`, `bedrock`, `anthropic`, etc. |
53+
| `virtual_key` | Virtual key for authentication | `<BEDROCK_VIRTUAL_KEY>` | [reference](https://portkey-ai.com/docs/authentication) |
54+
55+
## Troubleshooting
56+
57+
### Module Not Found
58+
59+
If you encounter the error `ModuleNotFoundError: No module named 'portkey'`, this means you haven't installed the `portkey` dependency in your environment. To fix, run `pip install 'strands-agents[portkey]'`.
60+
61+
## References
62+
63+
- [API](../../../api-reference/models.md)
64+
- [PortkeyAI](https://docs.portkey.ai)

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ nav:
8787
- LlamaAPI: user-guide/concepts/model-providers/llamaapi.md
8888
- Ollama: user-guide/concepts/model-providers/ollama.md
8989
- OpenAI: user-guide/concepts/model-providers/openai.md
90+
- PortKey: user-guide/concepts/model-providers/portkey.md
9091
- Custom Providers: user-guide/concepts/model-providers/custom_model_provider.md
9192
- Streaming:
9293
- Async Iterators: user-guide/concepts/streaming/async-iterators.md

0 commit comments

Comments
 (0)