Skip to content

Commit 09ba5aa

Browse files
eliastomarcusschiesser
eliasto
and
marcusschiesser
authored
Add OVHcloud AI Endpoints provider (#2238)
Co-authored-by: Marcus Schiesser <mail@marcusschiesser.de>
1 parent 5583d92 commit 09ba5aa

File tree

15 files changed

+776
-0
lines changed

15 files changed

+776
-0
lines changed

‎.changeset/famous-ducks-bathe.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"@llamaindex/ovhcloud": major
3+
"@llamaindex/examples": minor
4+
---
5+
6+
Add OVHcloud AI Endpoints provider

‎docs/src/content/docs/framework/modules/models/embeddings/ovhcloud.mdx‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
---
2+
title: OVHcloud AI Endpoints
3+
---
4+
5+
OVHcloud AI Endpoints provide OpenAI-compatible embedding models. The service can be used for free with rate limits, or with an API key for higher limits.
6+
7+
OVHcloud is a global player and the leading European cloud provider operating over 450,000 servers within 40 data centers across 4 continents to reach 1.6 million customers in over 140 countries. Our product AI Endpoints offers access to various models with sovereignty, data privacy and GDPR compliance.
8+
9+
You can find the full list of models in the [OVHcloud AI Endpoints catalog](https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/).
10+
11+
## Installation
12+
13+
```package-install
14+
npm i llamaindex @llamaindex/ovhcloud
15+
```
16+
17+
## Authentication
18+
19+
OVHcloud AI Endpoints can be used in two ways:
20+
21+
1. **Free tier (with rate limits)**: No API key required. You can omit the `apiKey` parameter or set it to an empty string.
22+
2. **With API key**: For higher rate limits, generate an API key from the [OVHcloud Manager](https://ovh.com/manager) → Public Cloud → AI & Machine Learning → AI Endpoints → API keys.
23+
24+
## Basic Usage
25+
26+
```ts
27+
import { Document, Settings, VectorStoreIndex } from "llamaindex";
28+
import { OVHcloudEmbedding } from "@llamaindex/ovhcloud";
29+
30+
// Update Embed Model (using free tier)
31+
Settings.embedModel = new OVHcloudEmbedding();
32+
33+
// Or with API key from environment variable
34+
import { config } from "dotenv";
35+
config();
36+
Settings.embedModel = new OVHcloudEmbedding({
37+
apiKey: process.env.OVHCLOUD_API_KEY || "",
38+
});
39+
40+
const document = new Document({ text: essay, id_: "essay" });
41+
42+
const index = await VectorStoreIndex.fromDocuments([document]);
43+
44+
const queryEngine = index.asQueryEngine();
45+
46+
const query = "What is the meaning of life?";
47+
48+
const results = await queryEngine.query({
49+
query,
50+
});
51+
```
52+
53+
By default, `OVHcloudEmbedding` uses the `BGE-M3` model. You can change the model by passing the model parameter to the constructor:
54+
55+
```ts
56+
import { OVHcloudEmbedding } from "@llamaindex/ovhcloud";
57+
58+
const model = "text-embedding-3-small";
59+
Settings.embedModel = new OVHcloudEmbedding({
60+
model,
61+
});
62+
```
63+
64+
You can also set the `maxRetries` and `timeout` parameters when initializing `OVHcloudEmbedding` for better control over the request behavior:
65+
66+
```ts
67+
import { Settings } from "llamaindex";
68+
import { OVHcloudEmbedding } from "@llamaindex/ovhcloud";
69+
70+
const model = "text-embedding-3-small";
71+
const maxRetries = 5;
72+
const timeout = 5000; // 5 seconds
73+
74+
Settings.embedModel = new OVHcloudEmbedding({
75+
model,
76+
maxRetries,
77+
timeout,
78+
});
79+
```
80+
81+
## Standalone Usage
82+
83+
```ts
84+
import { OVHcloudEmbedding } from "@llamaindex/ovhcloud";
85+
import { config } from "dotenv";
86+
// For standalone usage, you can optionally configure OVHCLOUD_API_KEY in .env file
87+
config();
88+
89+
const main = async () => {
90+
const model = "BGE-M3";
91+
// Using without API key (free tier)
92+
const embeddings = new OVHcloudEmbedding({ model });
93+
const text = "What is the meaning of life?";
94+
const response = await embeddings.embed([text]);
95+
console.log(response);
96+
};
97+
98+
main();
99+
```
100+
101+
## Base URL
102+
103+
The default base URL is `https://oai.endpoints.kepler.ai.cloud.ovh.net/v1`. You can override it if needed:
104+
105+
```ts
106+
const embedding = new OVHcloudEmbedding({
107+
model: "BGE-M3",
108+
additionalSessionOptions: {
109+
baseURL: "https://custom.endpoint.com/v1",
110+
},
111+
});
112+
```
113+
114+
## Resources
115+
116+
- [OVHcloud AI Endpoints Catalog](https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/)
117+
- [OVHcloud Manager](https://ovh.com/manager)
118+
- [OVHcloud AI Endpoints Documentation](https://www.ovhcloud.com/en/public-cloud/ai-endpoints/)
119+
120+
## API Reference
121+
122+
- [OVHcloudEmbedding](/typescript/framework-api-reference/classes/ovhcloudembedding/)
123+

‎docs/src/content/docs/framework/modules/models/llms/ovhcloud.mdx‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
---
2+
title: OVHcloud AI Endpoints
3+
---
4+
5+
OVHcloud AI Endpoints provide serverless access to a variety of pre-trained AI models. The service is OpenAI-compatible and can be used for free with rate limits, or with an API key for higher limits.
6+
7+
OVHcloud is a global player and the leading European cloud provider operating over 450,000 servers within 40 data centers across 4 continents to reach 1.6 million customers in over 140 countries. Our product AI Endpoints offers access to various models with sovereignty, data privacy and GDPR compliance.
8+
9+
You can find the full list of models in the [OVHcloud AI Endpoints catalog](https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/).
10+
11+
## Installation
12+
13+
```package-install
14+
npm i llamaindex @llamaindex/ovhcloud
15+
```
16+
17+
## Authentication
18+
19+
OVHcloud AI Endpoints can be used in two ways:
20+
21+
1. **Free tier (with rate limits)**: No API key required. You can omit the `apiKey` parameter or set it to an empty string.
22+
2. **With API key**: For higher rate limits, generate an API key from the [OVHcloud Manager](https://ovh.com/manager) → Public Cloud → AI & Machine Learning → AI Endpoints → API keys.
23+
24+
## Basic Usage
25+
26+
```ts
27+
import { OVHcloudLLM } from "@llamaindex/ovhcloud";
28+
import { Settings } from "llamaindex";
29+
30+
// Using without API key (free tier with rate limits)
31+
Settings.llm = new OVHcloudLLM({
32+
model: "gpt-oss-120b",
33+
});
34+
35+
// Or with API key from environment variable
36+
import { config } from "dotenv";
37+
config();
38+
Settings.llm = new OVHcloudLLM({
39+
model: "gpt-oss-120b",
40+
apiKey: process.env.OVHCLOUD_API_KEY || "",
41+
});
42+
43+
// Or with explicit API key
44+
Settings.llm = new OVHcloudLLM({
45+
model: "gpt-oss-120b",
46+
apiKey: "YOUR_API_KEY",
47+
});
48+
```
49+
50+
You can set the API key via environment variable:
51+
52+
```bash
53+
export OVHCLOUD_API_KEY="<YOUR_API_KEY>"
54+
```
55+
56+
## Load and index documents
57+
58+
For this example, we will use a single document. In a real-world scenario, you would have multiple documents to index.
59+
60+
```ts
61+
import { Document, VectorStoreIndex } from "llamaindex";
62+
63+
const document = new Document({ text: essay, id_: "essay" });
64+
65+
const index = await VectorStoreIndex.fromDocuments([document]);
66+
```
67+
68+
## Query
69+
70+
```ts
71+
const queryEngine = index.asQueryEngine();
72+
73+
const query = "What is the meaning of life?";
74+
75+
const results = await queryEngine.query({
76+
query,
77+
});
78+
```
79+
80+
## Full Example
81+
82+
```ts
83+
import { OVHcloudLLM } from "@llamaindex/ovhcloud";
84+
import { Document, VectorStoreIndex, Settings } from "llamaindex";
85+
86+
// Use custom LLM
87+
const model = "gpt-oss-120b";
88+
Settings.llm = new OVHcloudLLM({ model, temperature: 0 });
89+
90+
async function main() {
91+
const document = new Document({ text: essay, id_: "essay" });
92+
93+
// Load and index documents
94+
const index = await VectorStoreIndex.fromDocuments([document]);
95+
96+
// get retriever
97+
const retriever = index.asRetriever();
98+
99+
// Create a query engine
100+
const queryEngine = index.asQueryEngine({
101+
retriever,
102+
});
103+
104+
const query = "What is the meaning of life?";
105+
106+
// Query
107+
const response = await queryEngine.query({
108+
query,
109+
});
110+
111+
// Log the response
112+
console.log(response.response);
113+
}
114+
```
115+
116+
## Streaming
117+
118+
OVHcloud AI Endpoints supports streaming responses:
119+
120+
```ts
121+
import { OVHcloudLLM } from "@llamaindex/ovhcloud";
122+
123+
const llm = new OVHcloudLLM({
124+
model: "gpt-oss-120b",
125+
});
126+
127+
const generator = await llm.chat({
128+
messages: [
129+
{
130+
role: "user",
131+
content: "Tell me about OVHcloud AI Endpoints",
132+
},
133+
],
134+
stream: true,
135+
});
136+
137+
for await (const message of generator) {
138+
process.stdout.write(message.delta);
139+
}
140+
```
141+
142+
## Base URL
143+
144+
The default base URL is `https://oai.endpoints.kepler.ai.cloud.ovh.net/v1`. You can override it if needed:
145+
146+
```ts
147+
const llm = new OVHcloudLLM({
148+
model: "gpt-oss-120b",
149+
additionalSessionOptions: {
150+
baseURL: "https://custom.endpoint.com/v1",
151+
},
152+
});
153+
```
154+
155+
## Resources
156+
157+
- [OVHcloud AI Endpoints Catalog](https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/)
158+
- [OVHcloud Manager](https://ovh.com/manager)
159+
- [OVHcloud AI Endpoints Documentation](https://www.ovhcloud.com/en/public-cloud/ai-endpoints/)
160+
161+
## API Reference
162+
163+
- [OVHcloudLLM](/typescript/framework-api-reference/classes/ovhcloudllm/)
164+

‎examples/models/ovhcloud.ts‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
import { OVHcloudEmbedding, OVHcloudLLM } from "@llamaindex/ovhcloud";
2+
3+
// OVHcloud AI Endpoints can be used for free with rate limits without an API key
4+
// To use with an API key, set OVHCLOUD_API_KEY environment variable
5+
// or pass it directly in the constructor
6+
// To generate an API key, go to https://ovh.com/manager > Public Cloud > AI & Machine Learning > AI Endpoints > API keys
7+
// Visit our catalog for the list of all available models: https://www.ovhcloud.com/en/public-cloud/ai-endpoints/catalog/
8+
9+
// Example 1: Using without API key (free tier with rate limits)
10+
const ovhcloudFree = new OVHcloudLLM({
11+
model: "gpt-oss-120b",
12+
// apiKey is optional - can be omitted or set to empty string for free tier
13+
});
14+
15+
// Example 2: Using with API key
16+
const ovhcloud = new OVHcloudLLM({
17+
model: "gpt-oss-120b",
18+
apiKey: process.env.OVHCLOUD_API_KEY || "",
19+
});
20+
21+
(async () => {
22+
console.log("Chatting with OVHcloud AI Endpoints...");
23+
const generator = await ovhcloud.chat({
24+
messages: [
25+
{
26+
role: "system",
27+
content: "You are a helpful AI assistant.",
28+
},
29+
{
30+
role: "user",
31+
content: "Tell me about OVHcloud AI Endpoints",
32+
},
33+
],
34+
stream: true,
35+
});
36+
37+
for await (const message of generator) {
38+
process.stdout.write(message.delta);
39+
}
40+
console.log("\n");
41+
42+
// Example with embeddings
43+
console.log("Getting embeddings...");
44+
const embedding = new OVHcloudEmbedding({
45+
model: "BGE-M3",
46+
});
47+
const vector = await embedding.getTextEmbedding("Hello world!");
48+
console.log("Vector dimensions:", vector.length);
49+
console.log("First 5 values:", vector.slice(0, 5));
50+
})();

‎examples/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
"@llamaindex/notion": "^0.1.21",
4141
"@llamaindex/ollama": "^0.1.23",
4242
"@llamaindex/openai": "^0.4.21",
43+
"@llamaindex/ovhcloud": "^0.0.1",
4344
"@llamaindex/perplexity": "^0.0.34",
4445
"@llamaindex/pinecone": "^0.1.22",
4546
"@llamaindex/portkey-ai": "^0.0.64",

‎packages/providers/ovhcloud/CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# @llamaindex/ovhcloud
2+
3+
## 0.0.1
4+
5+
### Patch Changes
6+
7+
- Initial release of OVHcloud AI Endpoints provider

0 commit comments

Comments
 (0)