Skip to content

Commit 8bead08

Browse files
Fix Browser action deserialization by using OpenHandsModel
The root cause of 'Unexpected kind BrowserGetContentAction' errors during aggregation was that EvalOutput extended BaseModel instead of OpenHandsModel. When EvalOutput.model_validate() is called in iterative.py line 126 to deserialize output.jsonl results, the pydantic schema is already cached from the initial import. Since Browser classes are imported later in the import chain, they're not registered in the discriminated union schema. OpenHandsModel solves this by calling _rebuild_if_required() before validation, which regenerates all pydantic schemas to include any newly registered discriminated union types (like Browser actions/observations). This ensures that output.jsonl files containing Browser actions can be successfully deserialized during the aggregation step. Co-authored-by: openhands <openhands@all-hands.dev>
1 parent 384b924 commit 8bead08

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

benchmarks/utils/models.py

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
from openhands.sdk import LLM, Event, get_logger
66
from openhands.sdk.critic import CriticBase
77
from openhands.sdk.llm import Metrics
8+
from openhands.sdk.utils.models import OpenHandsModel
89

910

1011
logger = get_logger(__name__)
@@ -68,7 +69,15 @@ class EvalInstance(BaseModel):
6869
)
6970

7071

71-
class EvalOutput(BaseModel):
72+
class EvalOutput(OpenHandsModel):
73+
"""
74+
Evaluation output model.
75+
76+
Uses OpenHandsModel to ensure pydantic schemas are properly rebuilt when
77+
new discriminated union types (like Browser actions/observations) are registered.
78+
This prevents deserialization errors when loading results that contain
79+
dynamically registered event types.
80+
"""
7281
# NOTE: User-specified
7382
instance_id: str
7483
# output of the evaluation

0 commit comments

Comments
 (0)