Releases / Vision + policy

Filet o Fish

Shipped in openpilot 0.9.8. First public 7 Mar 2025.

Where it sits

Vision + policytwo networks

Filet o Fish Steam Powered North Nevada The Cool People's Dark Souls 2 WMI

MarAprMayJunJulAugSepOctNovDecJan 2026

Tests

G55 closed loopDistance from the recorded path at 2 snot run101520cmnot run
G55 recovery testsAfter a 2° heading error: distance at 2 snot run405060cmnot run
G55 open loopLateral error0.198 m/s² (0.162 to 0.207)0.150.200.25m/s²done 28 Oct 2026

Bar: this release's 95% interval, tick its mean. Grey dots: the means of the other releases in the Vision + policy family. Each axis spans every release the test has run on.

Tested 2026-10-28 04:04 UTC: protocol geval-1, harness d8eb40eeba.

Every metric, with paired differences

better, 95% interval excludes zero Better, 95% interval excludes zeroworse, 95% interval excludes zero Worse, interval excludes zerolower, 95% interval excludes zero; no better directionhigher, 95% interval excludes zero; no better direction Lower or higher, no better direction (action rate, lead)not separated, interval includes zero Not separatedno interval (too few drives) No interval: fewer than 10 drives

G55 closed loop

Not run for this model yet.

G55 recovery tests

Not run for this model yet.

G55 open loop

119 windows from 12 drives.

Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.

Metric	Mean	vs lebowski	vs North Dakota
Lateral errorm/s², lower is better	0.198[0.162, 0.207]median 0.190	worse, 95% interval excludes zero+0.025[+0.018, +0.037]better on 31/119	better, 95% interval excludes zero−0.016[−0.021, −0.002]better on 86/119
Lateral error at the model's own best leadm/s², lower is better	0.193[0.158, 0.203]median 0.185	worse, 95% interval excludes zero+0.029[+0.021, +0.040]better on 28/119	better, 95% interval excludes zero−0.017[−0.022, −0.002]better on 86/119
Lateral correlation with the car's later motionhigher is better	0.820[0.775, 0.841]median 0.899	better, 95% interval excludes zero+0.053[+0.003, +0.068]better on 85/119	better, 95% interval excludes zero+0.021[+0.013, +0.025]better on 95/119
Lateral gain (1 = same size as the car's motion)closer to 1 is better; the delta is the change in distance from 1	0.936[0.897, 1.019]median 0.918	not separated, interval includes zero−0.033[−0.060, +0.052]better on 70/119	not separated, interval includes zero+0.018[−0.003, +0.040]better on 50/119
Lateral action rate, RMS (smoothness)m/s³, no better direction	1.58[1.03, 1.74]median 1.35	lower, 95% interval excludes zero; no better direction−0.71[−0.97, −0.18]lower on 103/119	lower, 95% interval excludes zero; no better direction−3.75[−4.26, −1.87]lower on 119/119
Lead of the lateral request over the car's motions, no better direction	0.589[0.562, 0.651]median 0.600n = 117	not separated, interval includes zero−0.036[−0.087, +0.100]lower on 49/117	higher, 95% interval excludes zero; no better direction+0.028[+0.008, +0.059]lower on 34/117
Plan speed error at 3 sm/s, lower is better	2.43[1.68, 3.00]median 2.41	worse, 95% interval excludes zero+0.44[+0.14, +0.99]better on 29/119	better, 95% interval excludes zero−1.96[−2.36, −0.88]better on 109/119

By G55 camera setup (5 setups)

Metric	V14 (8 mm + IMX662)8 win, 2 drv	V13 (8 mm + IMX662)92 win, 3 drv	Stock aimed up4 win, 2 drv	Stock new case2 win, 1 drv	Stock dual13 win, 4 drv
Lateral errorm/s²	0.163med 0.159	0.207med 0.197	0.110med 0.110	0.207med 0.207	0.182med 0.171
Lateral error at the model's own best leadm/s²	0.158med 0.158	0.202med 0.193	0.108med 0.109	0.208med 0.208	0.177med 0.173
Lateral correlation with the car's later motion	0.837med 0.913	0.819med 0.907	0.810med 0.925	0.816med 0.816	0.825med 0.841
Lateral gain (1 = same size as the car's motion)	0.922med 0.961	0.932med 0.918	0.820med 0.924	0.874med 0.874	1.014med 0.890
Lateral action rate, RMS (smoothness)m/s³	0.99med 0.81	1.75med 1.54	0.90med 0.92	1.75med 1.75	0.97med 0.87
Lead of the lateral request over the car's motions	0.662med 0.600	0.583med 0.575	0.567med 0.550	0.350med 0.350	0.633med 0.675
Plan speed error at 3 sm/s	1.43med 1.45	2.68med 2.59	2.92med 2.96	1.94med 1.94	1.23med 1.20

Limits

Open-loop scores compare a model's action with what the recording driver or car did; a model that drives differently but well is scored as wrong. comma's RL residual makes the lateral action smoother, and earlier than its base's by 0.10 to 0.26 s (measured on the G55 and on 16 of comma's cars); both score as error here. Measured on the 2026-10-27 deep dive: open-loop accuracy did not predict closed-loop driving.
Closed-loop numbers come from a generative world model, not a car. In a separate test on the G55's windows, worldmodel-4B followed an added yaw command through 2 s but showed 0 to 20% of an added lateral offset 3 to 4 s later; past about 4 s the recorded anchor frames also pull every model back toward the recorded path. The front page reports 2 s.
Intervals are 95% percentile bootstraps that resample whole drives (the drive a public segment was cut from, the recorded drive a truck window comes from), so segments of one drive count once. Below 10 drives there is no interval: the braking-lead test comes from one drive and reports means and win counts only.
Paired deltas are model minus baseline on the units both have (for lateral gain, the change in its distance from 1); wins count units where the model is better in the metric's direction.
The lead of the lateral request is the shift, searched from 0 to 1.5 s, at which the request best correlates with the car's later lateral motion; a larger value means the request comes earlier. A value at either edge is censored and left out. Plan speed error at 3 s reads the plan head, which does not separate models built on one frozen base: it is identical on every segment for the four f22e06e5 models, and for Cinque v2 and v3.
Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.
A historical model, run outside comma's software with its own release's inputs, camera warp and command (engines/legacy.py). Checked against comma's CI model replay of this model (made on a comma device, 400 frames of one route): the plan's lateral position is 0.070 m RMS from comma's output; every other model here is 1.53 m or more from it on the same check.
Trained for the comma devices and cameras of its time, not for the G55's camera or for every camera among comma's public segments: of those 45 segments 26 record the OX03C10 sensor, 6 the AR0231, 4 the OS04C10 and 9 none; 30 were recorded 2021 to 2024 and 15 come from devices whose route names carry no date.
Not run in closed loop: the G55 closed-loop and recovery tests need this release's controller inside the world-model loop, which is not built yet.
No acceleration command: before openpilot 0.10 the longitudinal MPC, not the model, chose the acceleration, so the longitudinal action metrics are empty; the plan's speeds are scored.
The steering command is this release's (the model's desired-curvature head, unsmoothed); from 0.10 modeld smooths its command and the newest models are scored on their own action head, so lateral action rates are not comparable across these eras.

The model file

Model id: filet_o_fish
SHA-256: 897f80d0388250f99bba69b6a8434560cc0fd83157cbeb0bc134c67fe4e64624+98f0121ccb6f850077b04cc91bd33d370fc6cbdc2bd35f1ab55628a15a813f36
File tested: openpilot-master, d807b5c47665 (source), 2025-03-07
Parameters: not counted
GMAC per step: n/a
Notes: Shipped in openpilot 0.9.8 and 0.9.9. The first split model: a vision model and a policy on a 5 Hz feature history. The policy's desired-curvature head is the steering command. No acceleration command (the longitudinal MPC planned it), so no longitudinal scores. Vision 817b824f/500, policy e07ce1de/700.

Results file (results/filet_o_fish.json)

History of this page

28 Oct 2026Tests run (1 of 3), results published
7 Mar 2025Filet o Fish merged to openpilot master