Skip to content

Commit e0ca73a

Browse files
authored
builder secrets, builder config/data tab hooked up (#2581)
builder secrets, builder config/data tab hooked up, and tsunami cors config env var
1 parent 62e8ade commit e0ca73a

File tree

17 files changed

+599
-49
lines changed

17 files changed

+599
-49
lines changed

aiprompts/openai-request.md

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
# OpenAI Request Input Field Structure (On-the-Wire Format)
2+
3+
This document describes the actual JSON structure sent to the OpenAI API in the `input` field of [`OpenAIRequest`](../pkg/aiusechat/openai/openai-convertmessage.go:111).
4+
5+
## Overview
6+
7+
The `input` field is a JSON array containing one of three object types:
8+
9+
1. **Messages** (user/assistant) - `OpenAIMessage` objects
10+
2. **Function Calls** (tool invocations) - `OpenAIFunctionCallInput` objects
11+
3. **Function Call Results** (tool outputs) - `OpenAIFunctionCallOutputInput` objects
12+
13+
These are converted from [`OpenAIChatMessage`](../pkg/aiusechat/openai/openai-backend.go:46-52) internal format and cleaned before transmission ([see lines 485-494](../pkg/aiusechat/openai/openai-backend.go:485-494)).
14+
15+
## 1. Message Objects (User/Assistant)
16+
17+
User and assistant messages sent as [`OpenAIMessage`](../pkg/aiusechat/openai/openai-backend.go:54-57):
18+
19+
```json
20+
{
21+
"role": "user",
22+
"content": [
23+
{
24+
"type": "input_text",
25+
"text": "Hello, analyze this image"
26+
},
27+
{
28+
"type": "input_image",
29+
"image_url": "data:image/png;base64,iVBORw0KG..."
30+
}
31+
]
32+
}
33+
```
34+
35+
**Key Points:**
36+
- `role`: Always `"user"` or `"assistant"`
37+
- `content`: **Always an array** of content blocks (never a plain string)
38+
39+
### Content Block Types
40+
41+
#### Text Block
42+
```json
43+
{
44+
"type": "input_text",
45+
"text": "message content here"
46+
}
47+
```
48+
49+
#### Image Block
50+
```json
51+
{
52+
"type": "input_image",
53+
"image_url": "data:image/png;base64,..."
54+
}
55+
```
56+
- Can be a data URL or https:// URL
57+
- `filename` field is **removed** during cleaning
58+
59+
#### PDF File Block
60+
```json
61+
{
62+
"type": "input_file",
63+
"file_data": "JVBERi0xLjQKJeLjz9M...",
64+
"filename": "document.pdf"
65+
}
66+
```
67+
- `file_data`: Base64-encoded PDF content
68+
69+
#### Function Call Block (in assistant messages)
70+
```json
71+
{
72+
"type": "function_call",
73+
"call_id": "call_abc123",
74+
"name": "search_files",
75+
"arguments": {"query": "test"}
76+
}
77+
```
78+
79+
## 2. Function Call Objects (Tool Invocations)
80+
81+
Tool calls from the model sent as [`OpenAIFunctionCallInput`](../pkg/aiusechat/openai/openai-backend.go:59-67):
82+
83+
```json
84+
{
85+
"type": "function_call",
86+
"call_id": "call_abc123",
87+
"name": "search_files",
88+
"arguments": "{\"query\":\"test\",\"path\":\"./src\"}"
89+
}
90+
```
91+
92+
**Key Points:**
93+
- `type`: Always `"function_call"`
94+
- `call_id`: Unique identifier generated by model
95+
- `name`: Function name to execute
96+
- `arguments`: JSON-encoded string of parameters
97+
- `status`: Optional (`"in_progress"`, `"completed"`, `"incomplete"`)
98+
- Internal `toolusedata` field is **removed** during cleaning
99+
100+
## 3. Function Call Output Objects (Tool Results)
101+
102+
Tool execution results sent as [`OpenAIFunctionCallOutputInput`](../pkg/aiusechat/openai/openai-backend.go:69-75):
103+
104+
```json
105+
{
106+
"type": "function_call_output",
107+
"call_id": "call_abc123",
108+
"output": "Found 3 files matching query"
109+
}
110+
```
111+
112+
**Key Points:**
113+
- `type`: Always `"function_call_output"`
114+
- `call_id`: Must match the original function call's `call_id`
115+
- `output`: Can be text, image array, or error object
116+
117+
### Output Value Types
118+
119+
#### Text Output
120+
```json
121+
{
122+
"type": "function_call_output",
123+
"call_id": "call_abc123",
124+
"output": "Result text here"
125+
}
126+
```
127+
128+
#### Image Output
129+
```json
130+
{
131+
"type": "function_call_output",
132+
"call_id": "call_abc123",
133+
"output": [
134+
{
135+
"type": "input_image",
136+
"image_url": "data:image/png;base64,..."
137+
}
138+
]
139+
}
140+
```
141+
142+
#### Error Output
143+
```json
144+
{
145+
"type": "function_call_output",
146+
"call_id": "call_abc123",
147+
"output": "{\"ok\":\"false\",\"error\":\"File not found\"}"
148+
}
149+
```
150+
- Error output is a JSON-encoded string containing `ok` and `error` fields
151+
152+
## Complete Example
153+
154+
```json
155+
{
156+
"model": "gpt-4o",
157+
"input": [
158+
{
159+
"role": "user",
160+
"content": [
161+
{
162+
"type": "input_text",
163+
"text": "What files are in src/?"
164+
}
165+
]
166+
},
167+
{
168+
"type": "function_call",
169+
"call_id": "call_xyz789",
170+
"name": "list_files",
171+
"arguments": "{\"path\":\"src/\"}"
172+
},
173+
{
174+
"type": "function_call_output",
175+
"call_id": "call_xyz789",
176+
"output": "main.go\nutil.go\nconfig.go"
177+
},
178+
{
179+
"role": "assistant",
180+
"content": [
181+
{
182+
"type": "output_text",
183+
"text": "The src/ directory contains 3 files: main.go, util.go, and config.go"
184+
}
185+
]
186+
}
187+
],
188+
"stream": true,
189+
"max_output_tokens": 4096
190+
}
191+
```
192+
193+
## Cleaning Process
194+
195+
Before transmission, internal fields are removed ([cleanup code](../pkg/aiusechat/openai/openai-backend.go:485-494)):
196+
197+
- **Messages**: `previewurl` field removed, `filename` removed from `input_image` blocks
198+
- **Function Calls**: `toolusedata` field removed
199+
- **Function Outputs**: Sent as-is (no cleaning needed)
200+
201+
This ensures the API receives only the fields it expects.

frontend/builder/builder-apppanel.tsx

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,10 @@ import { TabRpcClient } from "@/app/store/wshrpcutil";
99
import { BuilderAppPanelModel, type TabType } from "@/builder/store/builder-apppanel-model";
1010
import { BuilderFocusManager } from "@/builder/store/builder-focusmanager";
1111
import { BuilderCodeTab } from "@/builder/tabs/builder-codetab";
12+
import { BuilderConfigDataTab } from "@/builder/tabs/builder-configdatatab";
1213
import { BuilderFilesTab, DeleteFileModal, RenameFileModal } from "@/builder/tabs/builder-filestab";
1314
import { BuilderPreviewTab } from "@/builder/tabs/builder-previewtab";
14-
import { BuilderEnvTab } from "@/builder/tabs/builder-secrettab";
15+
import { BuilderSecretTab } from "@/builder/tabs/builder-secrettab";
1516
import { builderAppHasSelection } from "@/builder/utils/builder-focus-utils";
1617
import { ErrorBoundary } from "@/element/errorboundary";
1718
import { atoms } from "@/store/global";
@@ -310,6 +311,13 @@ const BuilderAppPanel = memo(() => {
310311
isAppFocused={isAppFocused}
311312
onClick={() => handleTabClick("code")}
312313
/>
314+
<TabButton
315+
label="Config/Data"
316+
tabType="configdata"
317+
isActive={activeTab === "configdata"}
318+
isAppFocused={isAppFocused}
319+
onClick={() => handleTabClick("configdata")}
320+
/>
313321
<TabButton
314322
label="Files"
315323
tabType="files"
@@ -363,7 +371,12 @@ const BuilderAppPanel = memo(() => {
363371
</div>
364372
<div className="w-full h-full" style={{ display: activeTab === "secrets" ? "block" : "none" }}>
365373
<ErrorBoundary>
366-
<BuilderEnvTab />
374+
<BuilderSecretTab />
375+
</ErrorBoundary>
376+
</div>
377+
<div className="w-full h-full" style={{ display: activeTab === "configdata" ? "block" : "none" }}>
378+
<ErrorBoundary>
379+
<BuilderConfigDataTab />
367380
</ErrorBoundary>
368381
</div>
369382
</div>

frontend/builder/store/builder-apppanel-model.ts

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import { base64ToString, stringToBase64 } from "@/util/util";
1010
import { atom, type Atom, type PrimitiveAtom } from "jotai";
1111
import { debounce } from "throttle-debounce";
1212

13-
export type TabType = "preview" | "files" | "code" | "secrets";
13+
export type TabType = "preview" | "files" | "code" | "secrets" | "configdata";
1414

1515
export type EnvVar = {
1616
name: string;
@@ -121,6 +121,16 @@ export class BuilderAppPanelModel {
121121
}
122122
}
123123

124+
updateSecretBindings(newBindings: { [key: string]: string }) {
125+
const currentStatus = globalStore.get(this.builderStatusAtom);
126+
if (currentStatus) {
127+
globalStore.set(this.builderStatusAtom, {
128+
...currentStatus,
129+
secretbindings: newBindings,
130+
});
131+
}
132+
}
133+
124134
async loadEnvVars(builderId: string) {
125135
try {
126136
const rtInfo = await RpcApi.GetRTInfoCommand(TabRpcClient, {
@@ -215,7 +225,7 @@ export class BuilderAppPanelModel {
215225
async switchBuilderApp() {
216226
const builderId = globalStore.get(atoms.builderId);
217227
try {
218-
await RpcApi.StopBuilderCommand(TabRpcClient, builderId);
228+
await RpcApi.DeleteBuilderCommand(TabRpcClient, builderId);
219229
await new Promise((resolve) => setTimeout(resolve, 500));
220230
await RpcApi.SetRTInfoCommand(TabRpcClient, {
221231
oref: WOS.makeORef("builder", builderId),

0 commit comments

Comments
 (0)