Skip to content

Commit 3ca8b4b

Browse files
peterbandapeterbanda
committed
More examples added (structured output, responses api)
1 parent e654cc0 commit 3ca8b4b

File tree

1 file changed

+244
-21
lines changed

1 file changed

+244
-21
lines changed

‎README.md‎

Lines changed: 244 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -40,22 +40,22 @@ In addition to OpenAI, this library supports many other LLM providers. For provi
4040
|----------|------------------------|---------------|-------------|
4141
| [OpenAI](https://platform.openai.com) | Full | Standard + Responses API | Full API support |
4242
| [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) | Full | Standard + Responses API | OpenAI on Azure|
43-
| [Azure AI](https://azure.microsoft.com/en-us/products/ai-studio) | Varies | | Open-source models |
4443
| [Anthropic](https://www.anthropic.com/api) | Implied | | Claude models |
45-
| [Google Vertex AI](https://cloud.google.com/vertex-ai) | Full | Yes | Gemini models |
46-
| [Google Gemini](https://ai.google.dev/) (🔥 **New**) | Full | Yes | Google's models |
47-
| [Groq](https://wow.groq.com/) | Only JSON object mode | Yes | Fast inference |
48-
| [Grok](https://x.ai/) | Full | | x.AI models |
49-
| [Fireworks AI](https://fireworks.ai/) | Only JSON object mode | | Cloud provider |
50-
| [Octo AI](https://octo.ai/) | Only JSON object mode | | Cloud provider (obsolete) |
51-
| [TogetherAI](https://www.together.ai/) | Only JSON object mode | | Cloud provider |
44+
| [Azure AI](https://azure.microsoft.com/en-us/products/ai-studio) | Varies | | Open-source models |
5245
| [Cerebras](https://cerebras.ai/) | Only JSON object mode | | Fast inference |
53-
| [Mistral](https://mistral.ai/) | Only JSON object mode | | Open-source leader |
5446
| [Deepseek](https://deepseek.com/) | Only JSON object mode | | Chinese provider |
55-
| [Ollama](https://ollama.com/) | Varies | | Local LLMs |
5647
| [FastChat](https://github.com/lm-sys/FastChat) | Varies | | Local LLMs |
48+
| [Fireworks AI](https://fireworks.ai/) | Only JSON object mode | | Cloud provider |
49+
| [Google Gemini](https://ai.google.dev/) (🔥 **New**) | Full | Yes | Google's models |
50+
| [Google Vertex AI](https://cloud.google.com/vertex-ai) | Full | Yes | Gemini models |
51+
| [Grok](https://x.ai/) | Full | | x.AI models |
52+
| [Groq](https://wow.groq.com/) | Only JSON object mode | | Fast inference |
53+
| [Mistral](https://mistral.ai/) | Only JSON object mode | | Open-source leader |
5754
| [Novita](https://novita.ai/) (🔥 **New**) | Only JSON object mode | | Cloud provider |
55+
| [Octo AI](https://octo.ai/) | Only JSON object mode | | Cloud provider (obsolete) |
56+
| [Ollama](https://ollama.com/) | Varies | | Local LLMs |
5857
| [Perplexity Sonar](https://www.perplexity.ai/) (🔥 **New**) | Only implied | | Search-based AI |
58+
| [TogetherAI](https://www.together.ai/) | Only JSON object mode | | Cloud provider |
5959

6060
---
6161

@@ -378,7 +378,7 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
378378
}
379379
```
380380

381-
- Create chat completion with json/structured output
381+
- Create chat completion with **JSON/structured output**
382382

383383
```scala
384384
val messages = Seq(
@@ -408,7 +408,7 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
408408
val jsonSchemaDef = JsonSchemaDef(
409409
name = "capitals_response",
410410
strict = true,
411-
structure = schema
411+
structure = capitalsSchema
412412
)
413413

414414
service
@@ -427,7 +427,7 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
427427
}
428428
```
429429

430-
- Create chat completion with json/structured output using a handly implicit function (`createChatCompletionWithJSON[T]`) that handles JSON extraction with a potential repair, as well as deserialization to an object T.
430+
- Create chat completion with **JSON/structured output** using a handly implicit function (`createChatCompletionWithJSON[T]`) that handles JSON extraction with a potential repair, as well as deserialization to an object T.
431431

432432
```scala
433433
import io.cequence.openaiscala.service.OpenAIChatCompletionExtra._
@@ -449,10 +449,12 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
449449
}
450450
```
451451

452-
- Failover
452+
- **Failover** to alternative models if the primary one fails
453453

454-
```
455-
private val messages = Seq(
454+
```scala
455+
import io.cequence.openaiscala.service.OpenAIChatCompletionExtra._
456+
457+
val messages = Seq(
456458
SystemMessage("You are a helpful weather assistant."),
457459
UserMessage("What is the weather like in Norway?")
458460
)
@@ -471,14 +473,235 @@ There is a new project [openai-scala-client-examples](./openai-examples/src/main
471473
print(response.contentHead)
472474
}
473475
```
474-
- Failover with JSON/structred outuput
475476

476-
- Responses API (basic)
477+
- **Failover** with JSON/structured output
477478

478-
- Responses API (tool use)
479+
```scala
480+
import io.cequence.openaiscala.service.OpenAIChatCompletionExtra._
479481

482+
val capitalsSchema = JsonSchema.Object(
483+
properties = Map(
484+
"countries" -> JsonSchema.Array(
485+
items = JsonSchema.Object(
486+
properties = Map(
487+
"country" -> JsonSchema.String(
488+
description = Some("The name of the country")
489+
),
490+
"capital" -> JsonSchema.String(
491+
description = Some("The capital city of the country")
492+
)
493+
),
494+
required = Seq("country", "capital")
495+
)
496+
)
497+
),
498+
required = Seq("countries")
499+
)
500+
501+
val jsonSchemaDef = JsonSchemaDef(
502+
name = "capitals_response",
503+
strict = true,
504+
structure = capitalsSchema
505+
)
506+
507+
// Define the chat messages
508+
val messages = Seq(
509+
SystemMessage("Give me the most populous capital cities in JSON format."),
510+
UserMessage("List only african countries")
511+
)
480512

513+
// Call the service with failover support
514+
service
515+
.createChatCompletionWithJSON[JsObject](
516+
messages = messages,
517+
settings = CreateChatCompletionSettings(
518+
model = ModelId.o3_mini, // Primary model
519+
max_tokens = Some(1000),
520+
response_format_type = Some(ChatCompletionResponseFormatType.json_schema),
521+
jsonSchema = Some(jsonSchemaDef)
522+
),
523+
failoverModels = Seq(
524+
ModelId.gpt_4_5_preview, // First fallback model
525+
ModelId.gpt_4o // Second fallback model
526+
),
527+
maxRetries = Some(3), // Maximum number of retries per model
528+
retryOnAnyError = true, // Retry on any error, not just retryable ones
529+
taskNameForLogging = Some("capitals-query") // For better logging
530+
)
531+
.map { json =>
532+
println(Json.prettyPrint(json))
533+
}
534+
```
481535

536+
- **Responses API** - basic usage with textual inputs / messages
537+
538+
```scala
539+
import io.cequence.openaiscala.domain.responsesapi.Inputs
540+
541+
service
542+
.createModelResponse(
543+
Inputs.Text("What is the capital of France?")
544+
)
545+
.map { response =>
546+
println(response.outputText.getOrElse("N/A"))
547+
}
548+
```
549+
550+
```scala
551+
import io.cequence.openaiscala.domain.responsesapi.Input
552+
553+
service
554+
.createModelResponse(
555+
Inputs.Items(
556+
Input.ofInputSystemTextMessage(
557+
"You are a helpful assistant. Be verbose and detailed and don't be afraid to use emojis."
558+
),
559+
Input.ofInputUserTextMessage("What is the capital of France?")
560+
)
561+
)
562+
.map { response =>
563+
println(response.outputText.getOrElse("N/A"))
564+
}
565+
```
566+
567+
- **Responses API** - image input
568+
569+
```scala
570+
571+
import io.cequence.openaiscala.domain.responsesapi.{Inputs, Input}
572+
import io.cequence.openaiscala.domain.responsesapi.InputMessageContent
573+
import io.cequence.openaiscala.domain.ChatRole
574+
575+
service
576+
.createModelResponse(
577+
Inputs.Items(
578+
Input.ofInputMessage(
579+
Seq(
580+
InputMessageContent.Text("what is in this image?"),
581+
InputMessageContent.Image(
582+
imageUrl = Some(
583+
"https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
584+
)
585+
)
586+
),
587+
role = ChatRole.User
588+
)
589+
)
590+
)
591+
.map { response =>
592+
println(response.outputText.getOrElse("N/A"))
593+
}
594+
```
595+
596+
- **Responses API** - tool use (file search)
597+
598+
```scala
599+
600+
service
601+
.createModelResponse(
602+
Inputs.Text("What are the attributes of an ancient brown dragon?"),
603+
settings = CreateModelResponseSettings(
604+
model = ModelId.gpt_4o_2024_08_06,
605+
tools = Seq(
606+
FileSearchTool(
607+
vectorStoreIds = Seq("vs_1234567890"),
608+
maxNumResults = Some(20),
609+
filters = None,
610+
rankingOptions = None
611+
)
612+
)
613+
)
614+
)
615+
.map { response =>
616+
println(response.outputText.getOrElse("N/A"))
617+
618+
// citations
619+
val citations: Seq[Annotation.FileCitation] = response.outputMessageContents.collect {
620+
case e: OutputText =>
621+
e.annotations.collect { case citation: Annotation.FileCitation => citation }
622+
}.flatten
623+
624+
println("Citations:")
625+
citations.foreach { citation =>
626+
println(s"${citation.fileId} - ${citation.filename}")
627+
}
628+
}
629+
```
630+
631+
- **Responses API** - tool use (web search)
632+
633+
```scala
634+
service
635+
.createModelResponse(
636+
Inputs.Text("What was a positive news story from today?"),
637+
settings = CreateModelResponseSettings(
638+
model = ModelId.gpt_4o_2024_08_06,
639+
tools = Seq(WebSearchTool())
640+
)
641+
)
642+
.map { response =>
643+
println(response.outputText.getOrElse("N/A"))
644+
645+
// citations
646+
val citations: Seq[Annotation.UrlCitation] = response.outputMessageContents.collect {
647+
case e: OutputText =>
648+
e.annotations.collect { case citation: Annotation.UrlCitation => citation }
649+
}.flatten
650+
651+
println("Citations:")
652+
citations.foreach { citation =>
653+
println(s"${citation.title} - ${citation.url}")
654+
}
655+
}
656+
```
657+
658+
- **Responses API** - tool use (function call)
659+
660+
```scala
661+
service
662+
.createModelResponse(
663+
Inputs.Text("What is the weather like in Boston today?"),
664+
settings = CreateModelResponseSettings(
665+
model = ModelId.gpt_4o_2024_08_06,
666+
tools = Seq(
667+
FunctionTool(
668+
name = "get_current_weather",
669+
parameters = JsonSchema.Object(
670+
properties = Map(
671+
"location" -> JsonSchema.String(
672+
description = Some("The city and state, e.g. San Francisco, CA")
673+
),
674+
"unit" -> JsonSchema.String(
675+
`enum` = Seq("celsius", "fahrenheit")
676+
)
677+
),
678+
required = Seq("location", "unit")
679+
),
680+
description = Some("Get the current weather in a given location"),
681+
strict = true
682+
)
683+
),
684+
toolChoice = Some(ToolChoice.Mode.Auto)
685+
)
686+
)
687+
.map { response =>
688+
val functionCall = response.outputFunctionCalls.headOption
689+
.getOrElse(throw new RuntimeException("No function call output found"))
690+
691+
println(
692+
s"""Function Call Details:
693+
|Name: ${functionCall.name}
694+
|Arguments: ${functionCall.arguments}
695+
|Call ID: ${functionCall.callId}
696+
|ID: ${functionCall.id}
697+
|Status: ${functionCall.status}""".stripMargin
698+
)
699+
700+
val toolsUsed = response.tools.map(_.typeString)
701+
702+
println(s"${toolsUsed.size} tools used: ${toolsUsed.mkString(", ")}")
703+
}
704+
```
482705

483706
- Count expected used tokens before calling `createChatCompletions` or `createChatFunCompletions`, this helps you select proper model and reduce costs. This is an experimental feature and it may not work for all models. Requires `openai-scala-count-tokens` lib.
484707

@@ -598,7 +821,6 @@ Note that the adapters can be arbitrarily combined/stacked.
598821
Some(println(_)) // simple logging
599822
)
600823
```
601-
602824
- **Retry** on a specific function using [RetryHelpers](./openai-core/src/main/scala/io/cequence/openaiscala/RetryHelpers.scala) directly
603825

604826
```scala
@@ -710,4 +932,5 @@ This project is open-source and welcomes any contribution or feedback ([here](ht
710932

711933
Development of this library has been supported by [<img src="https://cequence.io/favicon-16x16.png"> - Cequence.io](https://cequence.io) - `The future of contracting`
712934

713-
Created and maintained by [Peter Banda](https://peterbanda.net).
935+
Created and maintained by [Peter Banda](https://peterbanda.net).
936+

0 commit comments

Comments
 (0)