G55 closed loop
Each model drives the G55 simulation inside comma's worldmodel-4B, which renders the next camera frame from the model's own steering, for 7.2 s from the start of a recorded window. Distance is to the driver's recorded path.
Distance from the recorded path at 2 s
- f22e06e55 Nov 2026, RL residual14.1 cm (10.9 to 17.1)
- f22b8b3429 Oct 2026, RL residual14.9 cm (11.4 to 17.8)
- 6b5b705f17 Oct 2026, RL residual13.9 cm (10.8 to 17.2)
- N16 nav model15 Oct 2026, RL residual13.6 cm (10.0 to 17.9)
- MD v3 / Flip Flop14 Oct 2026, RL residual13.7 cm (10.9 to 16.6)
- Mountain Dew10 Oct 2026, RL residual13.2 cm (10.2 to 15.8)
- 870a48238 Oct 2026, RL residual13.3 cm (10.7 to 15.4)
- Cinque v318 Sep 2026, Big RL15.0 cm (12.4 to 17.2)
- Cinque v211 Sep 2026, Big RL13.3 cm (10.7 to 15.6)
- lebowski1 Jul 2026, lebowski17.0 cm (13.7 to 20.3)
- Not run on the 9 releases before lebowski, 19 Jan 2026 to 7 Nov 2023: WMI, Dark Souls 2, The Cool People's, North Nevada, Steam Powered, Filet o Fish, North Dakota, Certified Herbalist, Farmville.
Distance from the recorded path at 4 s
- f22e06e55 Nov 2026, RL residual52.9 cm (40.4 to 66.7)
- f22b8b3429 Oct 2026, RL residual51.5 cm (37.2 to 65.1)
- 6b5b705f17 Oct 2026, RL residual57.7 cm (38.4 to 77.7)
- N16 nav model15 Oct 2026, RL residual55.5 cm (35.7 to 79.2)
- MD v3 / Flip Flop14 Oct 2026, RL residual53.9 cm (39.2 to 69.2)
- Mountain Dew10 Oct 2026, RL residual50.8 cm (37.4 to 65.3)
- 870a48238 Oct 2026, RL residual49.6 cm (36.7 to 63.8)
- Cinque v318 Sep 2026, Big RL49.8 cm (41.2 to 56.2)
- Cinque v211 Sep 2026, Big RL48.4 cm (39.3 to 55.7)
- lebowski1 Jul 2026, lebowski52.0 cm (37.3 to 69.1)
- Not run on the 9 releases before lebowski, 19 Jan 2026 to 7 Nov 2023: WMI, Dark Souls 2, The Cool People's, North Nevada, Steam Powered, Filet o Fish, North Dakota, Certified Herbalist, Farmville.
Requested lateral jerk, RMS
- f22e06e55 Nov 2026, RL residual0.302 m/s³ (0.253 to 0.354)
- f22b8b3429 Oct 2026, RL residual0.272 m/s³ (0.231 to 0.312)
- 6b5b705f17 Oct 2026, RL residual0.286 m/s³ (0.238 to 0.332)
- N16 nav model15 Oct 2026, RL residual0.263 m/s³ (0.226 to 0.298)
- MD v3 / Flip Flop14 Oct 2026, RL residual0.286 m/s³ (0.231 to 0.339)
- Mountain Dew10 Oct 2026, RL residual0.264 m/s³ (0.219 to 0.306)
- 870a48238 Oct 2026, RL residual0.266 m/s³ (0.218 to 0.318)
- Cinque v318 Sep 2026, Big RL0.230 m/s³ (0.205 to 0.257)
- Cinque v211 Sep 2026, Big RL0.222 m/s³ (0.197 to 0.248)
- lebowski1 Jul 2026, lebowski0.542 m/s³ (0.468 to 0.619)
- Not run on the 9 releases before lebowski, 19 Jan 2026 to 7 Nov 2023: WMI, Dark Souls 2, The Cool People's, North Nevada, Steam Powered, Filet o Fish, North Dakota, Certified Herbalist, Farmville.
The table: mean, interval and median
G55 closed loop
Each model drives the G55 simulation inside comma's worldmodel-4B, which renders the next camera frame from the model's own steering, for 7.2 s from the start of a recorded window. Distance is to the driver's recorded path. 98 windows from 13 drives for every model. Simulator g55-2026-10-31.
| Modelfirst public · status | Distance from the recorded path at 2 scm | Distance from the recorded path at 4 scm | Requested lateral jerk, RMSm/s³ |
|---|---|---|---|
f22e06e52026-11-05, Hugging Face checkpoint (unshipped) | 14.1 cm[10.9, 17.1]median 10.3 | 52.9 cm[40.4, 66.7]median 36.3 | 0.302 m/s³[0.253, 0.354]median 0.215 |
f22b8b342026-10-29, Hugging Face checkpoint (unshipped) | 14.9 cm[11.4, 17.8]median 9.9 | 51.5 cm[37.2, 65.1]median 31.0 | 0.272 m/s³[0.231, 0.312]median 0.186 |
6b5b705f2026-10-17, Hugging Face checkpoint (unshipped) | 13.9 cm[10.8, 17.2]median 9.5 | 57.7 cm[38.4, 77.7]median 30.0 | 0.286 m/s³[0.238, 0.332]median 0.188 |
N16 nav model 96244b1f2026-10-15, Hugging Face checkpoint (unshipped) | 13.6 cm[10.0, 17.9]median 8.3 | 55.5 cm[35.7, 79.2]median 28.4 | 0.263 m/s³[0.226, 0.298]median 0.193 |
MD v3 / Flip Flop 1365126a2026-10-14, merged to openpilot master (2026-10-14), updated 2026-10-24 | 13.7 cm[10.9, 16.6]median 8.5 | 53.9 cm[39.2, 69.2]median 31.8 | 0.286 m/s³[0.231, 0.339]median 0.178 |
Mountain Dew 8d7366832026-10-10, shipped in openpilot v0.11.2 (master 2026-10-10) | 13.2 cm[10.2, 15.8]median 9.1 | 50.8 cm[37.4, 65.3]median 32.8 | 0.264 m/s³[0.219, 0.306]median 0.179 |
870a48232026-10-08, Hugging Face checkpoint (unshipped) | 13.3 cm[10.7, 15.4]median 10.1 | 49.6 cm[36.7, 63.8]median 31.9 | 0.266 m/s³[0.218, 0.318]median 0.183 |
Cinque v3 f78ed37d2026-09-18, merged to openpilot master (2026-09-18) | 15.0 cm[12.4, 17.2]median 10.5 | 49.8 cm[41.2, 56.2]median 39.5 | 0.230 m/s³[0.205, 0.257]median 0.173 |
Cinque v2 1a4211752026-09-11, merged to openpilot master (2026-09-11) | 13.3 cm[10.7, 15.6]median 8.9 | 48.4 cm[39.3, 55.7]median 29.3 | 0.222 m/s³[0.197, 0.248]median 0.161 |
| lebowski2026-07-01, merged to openpilot master (2026-07-01) | 17.0 cm[13.7, 20.3]median 12.0 | 52.0 cm[37.3, 69.1]median 34.3 | 0.542 m/s³[0.468, 0.619]median 0.401 |
Conditions and limits
Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. The simulation starts from the truck's state (speed, yaw rate, steering, gear) in its CAN recording of the 2 s before the window; one window, whose 2 s before are not all valid CAN, starts steady at its recorded speed. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 98 windows (59 day, 39 night).
Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre. The pooled number mixes camera setups, so read the per-setup numbers too. On the V14 setup the wide frame can be up to 90 ms from the simulated one (its camera recorded 7-9 fps).
By camera setup
| Release | All setups | V14 (8 mm + IMX662) | V13 (8 mm + IMX662) | Stock aimed up | Stock new case | Stock dual |
|---|---|---|---|---|---|---|
f22e06e5 | 14.1 cm | 12.9 cm19 windows | 19.3 cm18 windows | 11.2 cm14 windows | 3.6 cm4 windows | 14.5 cm43 windows |
f22b8b34 | 14.9 cm | 14.0 cm19 windows | 18.6 cm18 windows | 13.6 cm14 windows | 3.3 cm4 windows | 15.2 cm43 windows |
6b5b705f | 13.9 cm | 12.8 cm19 windows | 19.6 cm18 windows | 12.2 cm14 windows | 4.5 cm4 windows | 13.4 cm43 windows |
N16 nav model 96244b1f | 13.6 cm | 12.1 cm19 windows | 21.5 cm18 windows | 9.1 cm14 windows | 5.2 cm4 windows | 13.2 cm43 windows |
MD v3 / Flip Flop 1365126a | 13.7 cm | 12.2 cm19 windows | 18.6 cm18 windows | 12.4 cm14 windows | 5.0 cm4 windows | 13.5 cm43 windows |
Mountain Dew 8d736683 | 13.2 cm | 12.7 cm19 windows | 17.2 cm18 windows | 12.1 cm14 windows | 5.1 cm4 windows | 12.8 cm43 windows |
870a4823 | 13.3 cm | 12.5 cm19 windows | 15.4 cm18 windows | 12.3 cm14 windows | 3.8 cm4 windows | 13.9 cm43 windows |
Cinque v3 f78ed37d | 15.0 cm | 12.7 cm19 windows | 15.6 cm18 windows | 13.8 cm14 windows | 10.7 cm4 windows | 16.5 cm43 windows |
Cinque v2 1a421175 | 13.3 cm | 11.4 cm19 windows | 16.0 cm18 windows | 11.8 cm14 windows | 6.7 cm4 windows | 14.1 cm43 windows |
| lebowski | 17.0 cm | 15.1 cm19 windows | 21.7 cm18 windows | 15.0 cm14 windows | 12.5 cm4 windows | 16.8 cm43 windows |
Conditions and limits
Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. The simulation starts from the truck's state (speed, yaw rate, steering, gear) in its CAN recording of the 2 s before the window; one window, whose 2 s before are not all valid CAN, starts steady at its recorded speed. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 98 windows (59 day, 39 night).
Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre. The pooled number mixes camera setups, so read the per-setup numbers too. On the V14 setup the wide frame can be up to 90 ms from the simulated one (its camera recorded 7-9 fps).
How this test has changed
- G55 closed loop and the recovery tests rerun for all 9 models on the current fit of the G55 simulation (steering centre and understeer refitted). Each window now starts from the truck's state in its CAN recording of the 2 s before it instead of a steady start; one window of 98 has no valid CAN there and starts steady. Windows and tests are unchanged. Medians at 2 s are now 8.3 to 12.0 cm (they were 12.0 to 15.3 cm), and 8 of 8 models are closer to the path than lebowski with the paired interval below zero (it was 5 of 8). On the G55 score the resamples no longer separate any model: one tier of 9 (it was 7 models, then Cinque v3 and lebowski). commit aebff69
- G55 closed loop and the heading and off-centre recovery tests now run on 98 windows from 13 drives and 5 camera setups (they were 18 windows from 4 drives; the braking lead keeps its 2 windows), with results per camera setup beside the pooled number. With 10 or more drives the closed-loop ranking now uses intervals. Drives whose recorded camera calibration was carried over from another drive are left out. The 8 current models rescored.
- comma's earlier driving models added, openpilot 0.9.5 (November 2023) to 0.11.1: nine models from Farmville to WMI, each run with its own release's inputs, camera warp and command and checked against comma's CI model replay of that model. Open loop on the G55 and on comma's segments; closed loop not yet. Their float16 feature normalisation runs in float32 here (in float16 it overflowed on some frames under ONNX Runtime). New over-time panel: G55 open-loop lateral error.