Skip to content

Commit c34b3c3

Browse files
Preserve full BrowserGym observation data in metadata
For VisualWebArena and advanced training scenarios, users may need access to additional data like timestamps, browser state, or custom fields that BrowserGym provides. Changes: - Store full 'obs' and 'info' dicts from BrowserGym in observation.metadata - Accessible via metadata['browsergym_obs'] and metadata['browsergym_info'] - Common fields still extracted to top-level for convenience - Added documentation showing how to access timestamps, DOM objects, etc. This ensures no information loss while maintaining a clean API. The environment exposes everything BrowserGym provides - users can access whatever they need for training.
1 parent 68da1ae commit c34b3c3

File tree

2 files changed

+35
-0
lines changed

2 files changed

+35
-0
lines changed

src/envs/browsergym_env/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,33 @@ if obs.last_action_error:
348348
# Episode status
349349
print(obs.done) # True if episode ended
350350
print(obs.reward) # Reward for the step
351+
352+
# Access full BrowserGym data (includes timestamps, etc.)
353+
print(obs.metadata["browsergym_obs"]) # Full observation dict from BrowserGym
354+
print(obs.metadata["browsergym_info"]) # Full info dict (timestamps, page state, etc.)
355+
```
356+
357+
#### Advanced: Accessing Raw BrowserGym Data
358+
359+
For VisualWebArena or custom training, you may need additional data like timestamps or browser state. The full BrowserGym observation and info dicts are preserved in `metadata`:
360+
361+
```python
362+
result = env.step(action)
363+
364+
# Access timestamps (if available)
365+
info = result.observation.metadata["browsergym_info"]
366+
if "timestamp" in info:
367+
print(f"Action timestamp: {info['timestamp']}")
368+
369+
# Access additional observation fields
370+
obs_dict = result.observation.metadata["browsergym_obs"]
371+
if "dom_object" in obs_dict:
372+
dom = obs_dict["dom_object"]
373+
# Work with raw DOM object
374+
375+
# Access page performance data
376+
if "performance" in info:
377+
print(f"Page load time: {info['performance']}")
351378
```
352379

353380
### State

src/envs/browsergym_env/server/browsergym_environment.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -219,6 +219,13 @@ def _create_observation(
219219
axtree_txt = obs.get("axtree_txt", "") if isinstance(obs, dict) else ""
220220
pruned_html = obs.get("pruned_html", "") if isinstance(obs, dict) else ""
221221

222+
# Store full BrowserGym observation and info in metadata
223+
# This preserves timestamps, additional fields, and any future extensions
224+
browsergym_metadata = {
225+
"browsergym_obs": obs if isinstance(obs, dict) else {},
226+
"browsergym_info": info,
227+
}
228+
222229
return BrowserGymObservation(
223230
text=text,
224231
url=url,
@@ -230,6 +237,7 @@ def _create_observation(
230237
last_action_error=False,
231238
done=done,
232239
reward=reward,
240+
metadata=browsergym_metadata,
233241
)
234242

235243
@property

0 commit comments

Comments
 (0)