My Self-Driving G-Wagon

G55 recovery tests

The G55 closed loop with a disturbance at the start of the window, or on the windows where the car ahead braked.

100 windows from 14 drives. Run on 10 of 19 releases. Simulator g55-2026-10-31: G55 steering and powertrain simulation with the steering centre and understeer as refitted by 2026-10-31; each window starts from the truck's state in its CAN recording of the 2 s before it. When the test changes, every release is run again.

After a 2° heading error: distance at 2 s

Lower is better. Bar: 95% interval from resampling whole drives; tick: mean. Rows are releases, newest first; a gap between two releases smaller than their intervals does not separate them.
405060cm
  1. f22e06e55 Nov 2026, RL residual48.1 cm (39.1 to 59.6)
  2. f22b8b3429 Oct 2026, RL residual48.3 cm (39.2 to 59.5)
  3. 6b5b705f17 Oct 2026, RL residual50.0 cm (40.7 to 61.5)
  4. N16 nav model15 Oct 2026, RL residual48.4 cm (38.5 to 61.4)
  5. MD v3 / Flip Flop14 Oct 2026, RL residual51.7 cm (42.5 to 62.9)
  6. Mountain Dew10 Oct 2026, RL residual49.2 cm (39.9 to 60.5)
  7. 870a48238 Oct 2026, RL residual51.8 cm (42.3 to 63.4)
  8. Cinque v318 Sep 2026, Big RL51.0 cm (41.4 to 62.7)
  9. Cinque v211 Sep 2026, Big RL50.1 cm (40.4 to 62.2)
  10. lebowski1 Jul 2026, lebowski45.3 cm (38.8 to 53.7)
  11. Not run on the 9 releases before lebowski, 19 Jan 2026 to 7 Nov 2023: WMI, Dark Souls 2, The Cool People's, North Nevada, Steam Powered, Filet o Fish, North Dakota, Certified Herbalist, Farmville.

After a 0.5 m lateral offset: distance at 2 s

Lower is better. Bar: 95% interval from resampling whole drives; tick: mean. Rows are releases, newest first; a gap between two releases smaller than their intervals does not separate them.
22.525.027.530.0cm
  1. f22e06e55 Nov 2026, RL residual25.4 cm (22.8 to 28.1)
  2. f22b8b3429 Oct 2026, RL residual26.3 cm (23.8 to 28.6)
  3. 6b5b705f17 Oct 2026, RL residual26.7 cm (23.9 to 29.3)
  4. N16 nav model15 Oct 2026, RL residual25.8 cm (22.8 to 29.0)
  5. MD v3 / Flip Flop14 Oct 2026, RL residual28.5 cm (25.9 to 31.0)
  6. Mountain Dew10 Oct 2026, RL residual25.8 cm (23.4 to 28.2)
  7. 870a48238 Oct 2026, RL residual28.7 cm (26.3 to 31.1)
  8. Cinque v318 Sep 2026, Big RL28.2 cm (25.2 to 31.0)
  9. Cinque v211 Sep 2026, Big RL26.9 cm (24.3 to 29.4)
  10. lebowski1 Jul 2026, lebowski26.7 cm (22.5 to 30.2)
  11. Not run on the 9 releases before lebowski, 19 Jan 2026 to 7 Nov 2023: WMI, Dark Souls 2, The Cool People's, North Nevada, Steam Powered, Filet o Fish, North Dakota, Certified Herbalist, Farmville.

Braking lead: distance ahead of the recorded car at 4 s

Lower is better. Bar: 95% interval from resampling whole drives; tick: mean. Rows are releases, newest first; a gap between two releases smaller than their intervals does not separate them.
−1.0−0.50.0m
  1. f22e06e55 Nov 2026, RL residual−0.022 m
  2. f22b8b3429 Oct 2026, RL residual−0.553 m
  3. 6b5b705f17 Oct 2026, RL residual−0.369 m
  4. N16 nav model15 Oct 2026, RL residual−0.024 m
  5. MD v3 / Flip Flop14 Oct 2026, RL residual−0.565 m
  6. Mountain Dew10 Oct 2026, RL residual−0.317 m
  7. 870a48238 Oct 2026, RL residual−0.969 m
  8. Cinque v318 Sep 2026, Big RL−0.031 m
  9. Cinque v211 Sep 2026, Big RL−0.166 m
  10. lebowski1 Jul 2026, lebowski−1.34 m
  11. Not run on the 9 releases before lebowski, 19 Jan 2026 to 7 Nov 2023: WMI, Dark Souls 2, The Cool People's, North Nevada, Steam Powered, Filet o Fish, North Dakota, Certified Herbalist, Farmville.
The table: mean, interval and median

G55 recovery tests

The G55 closed loop with a disturbance at the start of the window, or on the windows where the car ahead braked. 100 windows from 14 drives for every model. Simulator g55-2026-10-31.

Mean over windows with its 95% interval (resampling whole drives), median beneath. Newest release first.
Modelfirst public · statusAfter a 2° heading error: distance at 2 scmAfter a 0.5 m lateral offset: distance at 2 scmBraking lead: distance ahead of the recorded car at 4 sm
f22e06e52026-11-05, Hugging Face checkpoint (unshipped)48.1 cm[39.1, 59.6]median 41.025.4 cm[22.8, 28.1]median 21.3−0.022 mno interval (1 drive)median −0.022
f22b8b342026-10-29, Hugging Face checkpoint (unshipped)48.3 cm[39.2, 59.5]median 40.026.3 cm[23.8, 28.6]median 22.0−0.553 mno interval (1 drive)median −0.553
6b5b705f2026-10-17, Hugging Face checkpoint (unshipped)50.0 cm[40.7, 61.5]median 42.126.7 cm[23.9, 29.3]median 22.0−0.369 mno interval (1 drive)median −0.369
N16 nav model 96244b1f2026-10-15, Hugging Face checkpoint (unshipped)48.4 cm[38.5, 61.4]median 38.825.8 cm[22.8, 29.0]median 20.8−0.024 mno interval (1 drive)median −0.024
MD v3 / Flip Flop 1365126a2026-10-14, merged to openpilot master (2026-10-14), updated 2026-10-2451.7 cm[42.5, 62.9]median 43.528.5 cm[25.9, 31.0]median 23.8−0.565 mno interval (1 drive)median −0.565
Mountain Dew 8d7366832026-10-10, shipped in openpilot v0.11.2 (master 2026-10-10)49.2 cm[39.9, 60.5]median 41.025.8 cm[23.4, 28.2]median 21.3−0.317 mno interval (1 drive)median −0.317
870a48232026-10-08, Hugging Face checkpoint (unshipped)51.8 cm[42.3, 63.4]median 44.228.7 cm[26.3, 31.1]median 24.4−0.969 mno interval (1 drive)median −0.969
Cinque v3 f78ed37d2026-09-18, merged to openpilot master (2026-09-18)51.0 cm[41.4, 62.7]median 42.128.2 cm[25.2, 31.0]median 24.3−0.031 mno interval (1 drive)median −0.031
Cinque v2 1a4211752026-09-11, merged to openpilot master (2026-09-11)50.1 cm[40.4, 62.2]median 40.226.9 cm[24.3, 29.4]median 23.9−0.166 mno interval (1 drive)median −0.166
lebowski2026-07-01, merged to openpilot master (2026-07-01)45.3 cm[38.8, 53.7]median 39.226.7 cm[22.5, 30.2]median 20.7−1.34 mno interval (1 drive)median −1.34
Conditions and limits

Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 98 windows (59 day, 39 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 98 windows (59 day, 39 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).

Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test has 2 windows from one drive, whose recorded calibration was carried over from an earlier drive.

By camera setup
Mean per camera setup; the G55's drives cover 5 mounts and lens combinations. Newest release first.
ReleaseAll setupsV14 (8 mm + IMX662)V13 (8 mm + IMX662)Stock aimed upStock new caseStock dual
f22e06e548.1 cm38.8 cm19 windows85.2 cm18 windows39.3 cm14 windows38.1 cm4 windows40.5 cm45 windows
f22b8b3448.3 cm39.7 cm19 windows85.3 cm18 windows38.9 cm14 windows35.2 cm4 windows41.0 cm45 windows
6b5b705f50.0 cm41.2 cm19 windows86.9 cm18 windows43.0 cm14 windows34.9 cm4 windows42.2 cm45 windows
N16 nav model 96244b1f48.4 cm38.9 cm19 windows89.8 cm18 windows36.4 cm14 windows36.6 cm4 windows40.3 cm45 windows
MD v3 / Flip Flop 1365126a51.7 cm43.4 cm19 windows88.5 cm18 windows41.6 cm14 windows37.9 cm4 windows44.6 cm45 windows
Mountain Dew 8d73668349.2 cm40.9 cm19 windows86.3 cm18 windows39.6 cm14 windows35.8 cm4 windows41.6 cm45 windows
870a482351.8 cm43.9 cm19 windows89.7 cm18 windows40.7 cm14 windows38.9 cm4 windows44.2 cm45 windows
Cinque v3 f78ed37d51.0 cm41.0 cm19 windows89.4 cm18 windows35.5 cm14 windows38.8 cm4 windows45.4 cm45 windows
Cinque v2 1a42117550.1 cm40.7 cm19 windows89.5 cm18 windows35.5 cm14 windows38.5 cm4 windows43.6 cm45 windows
lebowski45.3 cm35.5 cm19 windows69.3 cm18 windows39.2 cm14 windows45.1 cm4 windows41.6 cm45 windows
Conditions and limits

Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 98 windows (59 day, 39 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 98 windows (59 day, 39 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).

Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test has 2 windows from one drive, whose recorded calibration was carried over from an earlier drive.

How this test has changed

  1. G55 closed loop and the recovery tests rerun for all 9 models on the current fit of the G55 simulation (steering centre and understeer refitted). Each window now starts from the truck's state in its CAN recording of the 2 s before it instead of a steady start; one window of 98 has no valid CAN there and starts steady. Windows and tests are unchanged. Medians at 2 s are now 8.3 to 12.0 cm (they were 12.0 to 15.3 cm), and 8 of 8 models are closer to the path than lebowski with the paired interval below zero (it was 5 of 8). On the G55 score the resamples no longer separate any model: one tier of 9 (it was 7 models, then Cinque v3 and lebowski). commit aebff69
  2. G55 closed loop and the heading and off-centre recovery tests now run on 98 windows from 13 drives and 5 camera setups (they were 18 windows from 4 drives; the braking lead keeps its 2 windows), with results per camera setup beside the pooled number. With 10 or more drives the closed-loop ranking now uses intervals. Drives whose recorded camera calibration was carried over from another drive are left out. The 8 current models rescored.
  3. G55 recovery after a 0.5 m offset added, 18 windows. commit a98409f