My Self-Driving G-Wagon

G55 open loop

Each model runs on the G55's recorded camera frames. Its lateral request is compared with the truck's measured lateral acceleration; the action rate is how fast the request changes.

119 windows from 12 drives. Run on 19 of 19 releases. When the test changes, every release is run again.

Lateral error

Lower is better. Bar: 95% interval from resampling whole drives; tick: mean. Rows are releases, newest first; a gap between two releases smaller than their intervals does not separate them.
0.150.200.25m/s²
  1. f22e06e55 Nov 2026, RL residual0.181 m/s² (0.153 to 0.187)
  2. f22b8b3429 Oct 2026, RL residual0.177 m/s² (0.147 to 0.187)
  3. 6b5b705f17 Oct 2026, RL residual0.161 m/s² (0.135 to 0.171)
  4. N16 nav model15 Oct 2026, RL residual0.171 m/s² (0.144 to 0.183)
  5. MD v3 / Flip Flop14 Oct 2026, RL residual0.156 m/s² (0.126 to 0.168)
  6. Mountain Dew10 Oct 2026, RL residual0.164 m/s² (0.134 to 0.175)
  7. 870a48238 Oct 2026, RL residual0.157 m/s² (0.129 to 0.169)
  8. Cinque v318 Sep 2026, Big RL0.167 m/s² (0.143 to 0.178)
  9. Cinque v211 Sep 2026, Big RL0.161 m/s² (0.140 to 0.170)
  10. lebowski1 Jul 2026, lebowski0.173 m/s² (0.138 to 0.184)
  11. WMI19 Jan 2026, Vision + policy0.231 m/s² (0.198 to 0.259)
  12. Dark Souls 212 Dec 2025, Vision + policy0.172 m/s² (0.145 to 0.189)
  13. The Cool People's20 Oct 2025, Vision + policy0.168 m/s² (0.144 to 0.181)
  14. North Nevada9 Oct 2025, Vision + policy0.165 m/s² (0.140 to 0.172)
  15. Steam Powered17 Aug 2025, Vision + policy0.161 m/s² (0.136 to 0.169)
  16. Filet o Fish7 Mar 2025, Vision + policy0.198 m/s² (0.162 to 0.207)
  17. North Dakota29 Apr 2024, One network0.214 m/s² (0.168 to 0.226)
  18. Certified Herbalist19 Feb 2024, One network0.216 m/s² (0.171 to 0.229)
  19. Farmville7 Nov 2023, One network0.213 m/s² (0.178 to 0.222)

Lateral action rate, RMS

Lower is better. Bar: 95% interval from resampling whole drives; tick: mean. Rows are releases, newest first; a gap between two releases smaller than their intervals does not separate them.
0246m/s³
  1. f22e06e55 Nov 2026, RL residual1.18 m/s³ (0.67 to 1.43)
  2. f22b8b3429 Oct 2026, RL residual1.18 m/s³ (0.67 to 1.42)
  3. 6b5b705f17 Oct 2026, RL residual1.16 m/s³ (0.65 to 1.38)
  4. N16 nav model15 Oct 2026, RL residual1.08 m/s³ (0.64 to 1.31)
  5. MD v3 / Flip Flop14 Oct 2026, RL residual1.12 m/s³ (0.61 to 1.35)
  6. Mountain Dew10 Oct 2026, RL residual1.09 m/s³ (0.64 to 1.32)
  7. 870a48238 Oct 2026, RL residual1.08 m/s³ (0.60 to 1.32)
  8. Cinque v318 Sep 2026, Big RL0.920 m/s³ (0.591 to 1.084)
  9. Cinque v211 Sep 2026, Big RL0.879 m/s³ (0.608 to 1.012)
  10. lebowski1 Jul 2026, lebowski2.29 m/s³ (1.19 to 2.62)
  11. WMI19 Jan 2026, Vision + policy0.818 m/s³ (0.547 to 0.901)
  12. Dark Souls 212 Dec 2025, Vision + policy0.718 m/s³ (0.441 to 0.782)
  13. The Cool People's20 Oct 2025, Vision + policy0.781 m/s³ (0.427 to 0.864)
  14. North Nevada9 Oct 2025, Vision + policy0.728 m/s³ (0.411 to 0.806)
  15. Steam Powered17 Aug 2025, Vision + policy0.652 m/s³ (0.386 to 0.724)
  16. Filet o Fish7 Mar 2025, Vision + policy1.58 m/s³ (1.03 to 1.74)
  17. North Dakota29 Apr 2024, One network5.33 m/s³ (2.92 to 5.96)
  18. Certified Herbalist19 Feb 2024, One network5.87 m/s³ (3.07 to 6.61)
  19. Farmville7 Nov 2023, One network3.67 m/s³ (1.98 to 4.07)
The table: mean, interval and median

G55 open loop

Each model runs on the G55's recorded camera frames. Its lateral request is compared with the truck's measured lateral acceleration; the action rate is how fast the request changes. 119 windows from 12 drives for every model.

Mean over windows with its 95% interval (resampling whole drives), median beneath. Newest release first.
Modelfirst public · statusLateral errorm/s²Lateral action rate, RMSm/s³
f22e06e52026-11-05, Hugging Face checkpoint (unshipped)0.181 m/s²[0.153, 0.187]median 0.1761.18 m/s³[0.67, 1.43]median 0.90
f22b8b342026-10-29, Hugging Face checkpoint (unshipped)0.177 m/s²[0.147, 0.187]median 0.1661.18 m/s³[0.67, 1.42]median 0.89
6b5b705f2026-10-17, Hugging Face checkpoint (unshipped)0.161 m/s²[0.135, 0.171]median 0.1501.16 m/s³[0.65, 1.38]median 0.86
N16 nav model 96244b1f2026-10-15, Hugging Face checkpoint (unshipped)0.171 m/s²[0.144, 0.183]median 0.1581.08 m/s³[0.64, 1.31]median 0.82
MD v3 / Flip Flop 1365126a2026-10-14, merged to openpilot master (2026-10-14), updated 2026-10-240.156 m/s²[0.126, 0.168]median 0.1441.12 m/s³[0.61, 1.35]median 0.82
Mountain Dew 8d7366832026-10-10, shipped in openpilot v0.11.2 (master 2026-10-10)0.164 m/s²[0.134, 0.175]median 0.1541.09 m/s³[0.64, 1.32]median 0.82
870a48232026-10-08, Hugging Face checkpoint (unshipped)0.157 m/s²[0.129, 0.169]median 0.1491.08 m/s³[0.60, 1.32]median 0.81
Cinque v3 f78ed37d2026-09-18, merged to openpilot master (2026-09-18)0.167 m/s²[0.143, 0.178]median 0.1610.920 m/s³[0.591, 1.084]median 0.752
Cinque v2 1a4211752026-09-11, merged to openpilot master (2026-09-11)0.161 m/s²[0.140, 0.170]median 0.1550.879 m/s³[0.608, 1.012]median 0.733
lebowski2026-07-01, merged to openpilot master (2026-07-01)0.173 m/s²[0.138, 0.184]median 0.1682.29 m/s³[1.19, 2.62]median 2.10
WMI2026-01-19, merged to openpilot master (2026-01-19)0.231 m/s²[0.198, 0.259]median 0.2190.818 m/s³[0.547, 0.901]median 0.751
Dark Souls 22025-12-12, merged to openpilot master (2025-12-12)0.172 m/s²[0.145, 0.189]median 0.1630.718 m/s³[0.441, 0.782]median 0.635
The Cool People's2025-10-20, merged to openpilot master (2025-10-20)0.168 m/s²[0.144, 0.181]median 0.1600.781 m/s³[0.427, 0.864]median 0.683
North Nevada2025-10-09, merged to openpilot master (2025-10-09)0.165 m/s²[0.140, 0.172]median 0.1640.728 m/s³[0.411, 0.806]median 0.636
Steam Powered2025-08-17, merged to openpilot master (2025-08-17)0.161 m/s²[0.136, 0.169]median 0.1550.652 m/s³[0.386, 0.724]median 0.537
Filet o Fish2025-03-07, merged to openpilot master (2025-03-07)0.198 m/s²[0.162, 0.207]median 0.1901.58 m/s³[1.03, 1.74]median 1.35
North Dakota2024-04-29, merged to openpilot master (2024-04-29)0.214 m/s²[0.168, 0.226]median 0.2035.33 m/s³[2.92, 5.96]median 4.37
Certified Herbalist2024-02-19, merged to openpilot master (2024-02-19)0.216 m/s²[0.171, 0.229]median 0.2055.87 m/s³[3.07, 6.61]median 4.82
Farmville2023-11-07, merged to openpilot master (2023-11-07)0.213 m/s²[0.178, 0.222]median 0.2033.67 m/s³[1.98, 4.07]median 3.59
Conditions and limits

Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.

Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.

By camera setup
Mean per camera setup; the G55's drives cover 5 mounts and lens combinations. Newest release first.
ReleaseAll setupsV14 (8 mm + IMX662)V13 (8 mm + IMX662)Stock aimed upStock new caseStock dual
f22e06e50.181 m/s²0.174 m/s²8 windows0.189 m/s²92 windows0.116 m/s²4 windows0.133 m/s²2 windows0.156 m/s²13 windows
f22b8b340.177 m/s²0.175 m/s²8 windows0.182 m/s²92 windows0.116 m/s²4 windows0.137 m/s²2 windows0.163 m/s²13 windows
6b5b705f0.161 m/s²0.152 m/s²8 windows0.167 m/s²92 windows0.114 m/s²4 windows0.150 m/s²2 windows0.143 m/s²13 windows
N16 nav model 96244b1f0.171 m/s²0.170 m/s²8 windows0.174 m/s²92 windows0.136 m/s²4 windows0.149 m/s²2 windows0.163 m/s²13 windows
MD v3 / Flip Flop 1365126a0.156 m/s²0.144 m/s²8 windows0.162 m/s²92 windows0.119 m/s²4 windows0.141 m/s²2 windows0.132 m/s²13 windows
Mountain Dew 8d7366830.164 m/s²0.157 m/s²8 windows0.170 m/s²92 windows0.114 m/s²4 windows0.144 m/s²2 windows0.143 m/s²13 windows
870a48230.157 m/s²0.153 m/s²8 windows0.163 m/s²92 windows0.107 m/s²4 windows0.130 m/s²2 windows0.141 m/s²13 windows
Cinque v3 f78ed37d0.167 m/s²0.167 m/s²8 windows0.169 m/s²92 windows0.150 m/s²4 windows0.162 m/s²2 windows0.155 m/s²13 windows
Cinque v2 1a4211750.161 m/s²0.161 m/s²8 windows0.164 m/s²92 windows0.129 m/s²4 windows0.159 m/s²2 windows0.149 m/s²13 windows
lebowski0.173 m/s²0.141 m/s²8 windows0.182 m/s²92 windows0.141 m/s²4 windows0.134 m/s²2 windows0.145 m/s²13 windows
WMI0.231 m/s²0.188 m/s²8 windows0.239 m/s²92 windows0.267 m/s²4 windows0.176 m/s²2 windows0.192 m/s²13 windows
Dark Souls 20.172 m/s²0.153 m/s²8 windows0.180 m/s²92 windows0.116 m/s²4 windows0.174 m/s²2 windows0.142 m/s²13 windows
The Cool People's0.168 m/s²0.158 m/s²8 windows0.175 m/s²92 windows0.102 m/s²4 windows0.163 m/s²2 windows0.147 m/s²13 windows
North Nevada0.165 m/s²0.159 m/s²8 windows0.172 m/s²92 windows0.093 m/s²4 windows0.166 m/s²2 windows0.144 m/s²13 windows
Steam Powered0.161 m/s²0.150 m/s²8 windows0.167 m/s²92 windows0.108 m/s²4 windows0.166 m/s²2 windows0.133 m/s²13 windows
Filet o Fish0.198 m/s²0.163 m/s²8 windows0.207 m/s²92 windows0.110 m/s²4 windows0.207 m/s²2 windows0.182 m/s²13 windows
North Dakota0.214 m/s²0.184 m/s²8 windows0.226 m/s²92 windows0.133 m/s²4 windows0.229 m/s²2 windows0.172 m/s²13 windows
Certified Herbalist0.216 m/s²0.174 m/s²8 windows0.229 m/s²92 windows0.158 m/s²4 windows0.220 m/s²2 windows0.170 m/s²13 windows
Farmville0.213 m/s²0.197 m/s²8 windows0.220 m/s²92 windows0.161 m/s²4 windows0.291 m/s²2 windows0.176 m/s²13 windows
Conditions and limits

Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.

Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.

How this test has changed

  1. G55 open loop now runs on 119 windows of 30 s from 12 drives and 5 camera setups (it was 4 windows from 3 drives), with results per camera setup beside the pooled number. Both cameras feed the model; on the V13 setup the part of comma's narrow view the 8 mm lens misses comes from the wide camera. Drives whose recorded camera calibration was carried over from another drive are left out. All 17 models rescored.
  2. comma's earlier driving models added, openpilot 0.9.5 (November 2023) to 0.11.1: nine models from Farmville to WMI, each run with its own release's inputs, camera warp and command and checked against comma's CI model replay of that model. Open loop on the G55 and on comma's segments; closed loop not yet. Their float16 feature normalisation runs in float32 here (in float16 it overflowed on some frames under ONNX Runtime). New over-time panel: G55 open-loop lateral error.