Merge pull request #454 from simonx1/add-batches-endpoint-support

Add new /batches endpoint for batch of requests run asynchronously

Author: alexrudall

README.md

@@ -35,6 +35,7 @@ Stream text with GPT-4, transcribe and translate audio with Whisper, or create i
     - [Functions](#functions)
     - [Edits](#edits)
     - [Embeddings](#embeddings)
+    - [Batches](#batches)
     - [Files](#files)
     - [Finetunes](#finetunes)
     - [Assistants](#assistants)
@@ -486,6 +487,94 @@ puts response.dig("data", 0, "embedding")
 # => Vector representation of your embedding
 ```
 
+### Batches
+
+The Batches endpoint allows you to create and manage large batches of API requests to run asynchronously. Currently, only the `/v1/chat/completions` endpoint is supported for batches.
+
+To use the Batches endpoint, you need to first upload a JSONL file containing the batch requests using the Files endpoint. The file must be uploaded with the purpose set to `batch`. Each line in the JSONL file represents a single request and should have the following format:
+
+```json
+{
+  "custom_id": "request-1",
+  "method": "POST",
+  "url": "/v1/chat/completions",
+  "body": {
+    "model": "gpt-3.5-turbo",
+    "messages": [
+      { "role": "system", "content": "You are a helpful assistant." },
+      { "role": "user", "content": "What is 2+2?" }
+    ]
+  }
+}
+```
+
+Once you have uploaded the JSONL file, you can create a new batch by providing the file ID, endpoint, and completion window:
+
+```ruby
+response = client.batches.create(
+  parameters: {
+    input_file_id: "file-abc123",
+    endpoint: "/v1/chat/completions",
+    completion_window: "24h"
+  }
+)
+batch_id = response["id"]
+```
+
+You can retrieve information about a specific batch using its ID:
+
+```ruby
+batch = client.batches.retrieve(id: batch_id)
+```
+
+To cancel a batch that is in progress:
+
+```ruby
+client.batches.cancel(id: batch_id)
+```
+
+You can also list all the batches:
+
+```ruby
+client.batches.list
+```
+
+Once the batch["completed_at"] is present, you can fetch the output or error files:
+
+```ruby
+batch = client.batches.retrieve(id: batch_id)
+output_file_id = batch["output_file_id"]
+output_response = client.files.content(id: output_file_id)
+error_file_id = batch["error_file_id"]
+error_response = client.files.content(id: error_file_id)
+```
+
+These files are in JSONL format, with each line representing the output or error for a single request. The lines can be in any order:
+
+```json
+{
+  "id": "response-1",
+  "custom_id": "request-1",
+  "response": {
+    "id": "chatcmpl-abc123",
+    "object": "chat.completion",
+    "created": 1677858242,
+    "model": "gpt-3.5-turbo-0301",
+    "choices": [
+      {
+        "index": 0,
+        "message": {
+          "role": "assistant",
+          "content": "2+2 equals 4."
+        }
+      }
+    ]
+  }
+}
+```
+
+If a request fails with a non-HTTP error, the error object will contain more information about the cause of the failure.
+
 ### Files
 
 Put your data in a `.jsonl` file like this:

lib/openai.rb

@@ -14,6 +14,7 @@
 require_relative "openai/run_steps"
 require_relative "openai/audio"
 require_relative "openai/version"
+require_relative "openai/batches"
 
 module OpenAI
   class Error < StandardError; end

lib/openai/batches.rb

@@ -0,0 +1,23 @@
+module OpenAI
+  class Batches
+    def initialize(client:)
+      @client = client.beta(assistants: "v1")
+    end
+
+    def list
+      @client.get(path: "/batches")
+    end
+
+    def retrieve(id:)
+      @client.get(path: "/batches/#{id}")
+    end
+
+    def create(parameters: {})
+      @client.json_post(path: "/batches", parameters: parameters)
+    end
+
+    def cancel(id:)
+      @client.post(path: "/batches/#{id}/cancel")
+    end
+  end
+end

lib/openai/client.rb

@@ -78,6 +78,10 @@ def run_steps
       @run_steps ||= OpenAI::RunSteps.new(client: self)
     end
 
+    def batches
+      @batches ||= OpenAI::Batches.new(client: self)
+    end
+
     def moderations(parameters: {})
       json_post(path: "/moderations", parameters: parameters)
     end