{
 "schema": 1,
 "model": {
  "id": "1a421175_12864",
  "name": "Cinque v2",
  "sha256": "c7b300083b8dd6ce17545ef52edb6b577e3a805e6372f9f9343090604690242f",
  "source": {
   "kind": "huggingface",
   "ref": "c83630ca0ea166f74c89a67bbdf2a3a820126de6",
   "url": "https://huggingface.co/commaai/openpilot_driving_models/blob/c83630ca0ea166f74c89a67bbdf2a3a820126de6/1a421175-db71-4e3d-9d62-e2166421b02b/12864/big_driving_supercombo.onnx",
   "date": "2026-09-18"
  },
  "first_public": "2026-09-11",
  "on_master": "2026-09-11",
  "params_m": 381.59,
  "gmac": 46.7,
  "family": "big-rl",
  "notes": "On openpilot master 2026-09-11 (PR #38850). Vision base b9facbcc, history in the graph, no residual head."
 },
 "scored": {
  "date": "2026-10-31T23:59:21Z",
  "suite": "geval-1",
  "code": "d8eb40eeba87"
 },
 "baselines": [
  "lebowski_877m"
 ],
 "sections": {
  "public_open_loop": {
   "status": "ok",
   "unit_of_evidence": "segment",
   "n": 39,
   "clusters": 23,
   "condition": "Measured. comma's public two-camera segments (openpilot CI data), native intrinsics and the device's own calibration, both cameras; every model on the same decoded frames. Truth is what the recording car did next: carState speed and acceleration, yaw rate from the device's cameraOdometry. Segments where the car never moves have no data. 39 of the 45 listed segments have a score, from 23 drives and 20 cars.",
   "baselines": [
    "lebowski_877m"
   ],
   "metrics": [
    {
     "key": "lat_mae",
     "label": "Lateral error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1473,
     "median": 0.1204,
     "ci": [
      0.113,
      0.1826
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.03859,
       "median": 0.01926,
       "ci": [
        0.02044,
        0.05787
       ],
       "n": 39,
       "wins": 7,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "lat_mae_bl",
     "label": "Lateral error at the model's own best lead",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1378,
     "median": 0.1184,
     "ci": [
      0.1043,
      0.1732
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.03635,
       "median": 0.02002,
       "ci": [
        0.01686,
        0.05664
       ],
       "n": 39,
       "wins": 10,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "lat_corr",
     "label": "Lateral correlation with the car's later motion",
     "unit": "",
     "better": "higher",
     "value": 0.7951,
     "median": 0.9277,
     "ci": [
      0.6982,
      0.883
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.0625,
       "median": -0.01768,
       "ci": [
        -0.135,
        -0.00331
       ],
       "n": 39,
       "wins": 10,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "lat_gain",
     "label": "Lateral gain (1 = same size as the car's motion)",
     "unit": "",
     "better": "near1",
     "value": 0.7024,
     "median": 0.7585,
     "ci": [
      0.5997,
      0.8061
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.1183,
       "median": 0.07084,
       "ci": [
        0.02499,
        0.2142
       ],
       "n": 39,
       "wins": 11,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 5
      }
     }
    },
    {
     "key": "lat_rate_rms",
     "label": "Lateral action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.4121,
     "median": 0.3515,
     "ci": [
      0.3359,
      0.4985
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.2518,
       "median": -0.2406,
       "ci": [
        -0.3037,
        -0.1983
       ],
       "n": 39,
       "wins": 38,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 23
      }
     }
    },
    {
     "key": "lat_best_lag",
     "label": "Lead of the lateral request over the car's motion",
     "unit": "s",
     "better": "lower",
     "value": 0.4922,
     "median": 0.45,
     "ci": [
      0.3828,
      0.6028
     ],
     "n": 32,
     "clusters": 20,
     "paired": {
      "lebowski_877m": {
       "delta": 0.09194,
       "median": 0.1,
       "ci": [
        0.03148,
        0.1583
       ],
       "n": 31,
       "wins": 6,
       "ties": 3,
       "clusters": 20,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "lon_mae",
     "label": "Longitudinal error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.317,
     "median": 0.2842,
     "ci": [
      0.2724,
      0.3669
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.008842,
       "median": 0.01124,
       "ci": [
        -0.005507,
        0.02466
       ],
       "n": 39,
       "wins": 16,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 9
      }
     }
    },
    {
     "key": "lon_corr",
     "label": "Longitudinal correlation",
     "unit": "",
     "better": "higher",
     "value": 0.7173,
     "median": 0.8345,
     "ci": [
      0.608,
      0.8106
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.0004959,
       "median": 0.001123,
       "ci": [
        -0.02637,
        0.02398
       ],
       "n": 39,
       "wins": 21,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 12
      }
     }
    },
    {
     "key": "lon_brake_mae",
     "label": "Longitudinal error while braking",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.4407,
     "median": 0.421,
     "ci": [
      0.3729,
      0.5225
     ],
     "n": 30,
     "clusters": 21,
     "paired": {
      "lebowski_877m": {
       "delta": 0.006456,
       "median": 0.01612,
       "ci": [
        -0.02332,
        0.04304
       ],
       "n": 30,
       "wins": 11,
       "ties": 0,
       "clusters": 21,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "lon_rate_rms",
     "label": "Longitudinal action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.7678,
     "median": 0.6596,
     "ci": [
      0.6942,
      0.8519
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.5652,
       "median": -0.4338,
       "ci": [
        -0.7168,
        -0.4241
       ],
       "n": 39,
       "wins": 37,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 23
      }
     }
    },
    {
     "key": "v3_mae",
     "label": "Plan speed error at 3 s",
     "unit": "m/s",
     "better": "lower",
     "value": 0.7585,
     "median": 0.7436,
     "ci": [
      0.6582,
      0.8885
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.0427,
       "median": 0.02799,
       "ci": [
        -0.009648,
        0.09442
       ],
       "n": 39,
       "wins": 14,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 9
      }
     }
    }
   ],
   "per_unit_file": "per_unit/1a421175_12864/public_open_loop.csv"
  },
  "public_closed_loop": {
   "status": "ok",
   "unit_of_evidence": "segment",
   "n": 28,
   "clusters": 19,
   "condition": "Measured. comma's worldmodel-4B generates each next frame at 5 Hz from 9 history frames, 5 recorded anchor frames and the car's pose; the model drives comma's ideal car (the requested curvature and acceleration happen at once) for 7.2 s from the recorded start. Distance is to the recording car's own path. One window per segment, nearest the middle, where the car holds at least 5 m/s for the window and the 4.8 s before it; segments with no such stretch are left out. Seeds 0 and 1, averaged per segment. 28 of the 45 listed segments have a score, from 19 drives and 17 cars.",
   "baselines": [
    "lebowski_877m"
   ],
   "metrics": [
    {
     "key": "xtrack_2s",
     "label": "Distance from the recorded path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.1137,
     "median": 0.06276,
     "ci": [
      0.05285,
      0.2237
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.03397,
       "median": -0.009269,
       "ci": [
        -0.06483,
        -0.009618
       ],
       "n": 28,
       "wins": 19,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 15
      }
     }
    },
    {
     "key": "xtrack_4s",
     "label": "Distance from the recorded path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.2509,
     "median": 0.1134,
     "ci": [
      0.1133,
      0.4724
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02739,
       "median": 0.007581,
       "ci": [
        -0.06822,
        0.01239
       ],
       "n": 28,
       "wins": 11,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 10
      }
     }
    },
    {
     "key": "xtrack_7s",
     "label": "Distance from the recorded path at 7.2 s",
     "unit": "m",
     "better": "lower",
     "value": 1.755,
     "median": 0.3779,
     "ci": [
      0.5744,
      3.441
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.7111,
       "median": 0.01665,
       "ci": [
        -0.04963,
        1.911
       ],
       "n": 28,
       "wins": 13,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 9
      }
     }
    },
    {
     "key": "fail_1m",
     "label": "Share of episodes ever more than 1 m off the path",
     "unit": "",
     "better": "lower",
     "value": 0.25,
     "median": 0.0,
     "ci": [
      0.09259,
      0.4376
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.01786,
       "median": 0.0,
       "ci": [
        -0.04412,
        0.1111
       ],
       "n": 28,
       "wins": 1,
       "ties": 26,
       "clusters": 19,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "lat_jerk",
     "label": "Requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.1725,
     "median": 0.09977,
     "ci": [
      0.117,
      0.2322
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.1531,
       "median": -0.08097,
       "ci": [
        -0.2141,
        -0.09983
       ],
       "n": 28,
       "wins": 27,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 18
      }
     }
    },
    {
     "key": "long_jerk",
     "label": "Requested longitudinal jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.2691,
     "median": 0.2199,
     "ci": [
      0.2016,
      0.3507
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.07726,
       "median": -0.07915,
       "ci": [
        -0.115,
        -0.03183
       ],
       "n": 28,
       "wins": 26,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 18
      }
     }
    },
    {
     "key": "speed_err",
     "label": "Speed error against the recorded speed, RMS",
     "unit": "m/s",
     "better": "lower",
     "value": 1.435,
     "median": 0.8392,
     "ci": [
      1.004,
      1.919
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.3333,
       "median": 0.2393,
       "ci": [
        0.07923,
        0.6095
       ],
       "n": 28,
       "wins": 9,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 6
      }
     }
    }
   ],
   "per_unit_file": "per_unit/1a421175_12864/public_closed_loop.csv"
  },
  "truck_open_loop": {
   "status": "ok",
   "unit_of_evidence": "window",
   "n": 119,
   "clusters": 12,
   "condition": "Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.",
   "baselines": [
    "lebowski_877m"
   ],
   "metrics": [
    {
     "key": "lat_mae",
     "label": "Lateral error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1605,
     "median": 0.1548,
     "ci": [
      0.1396,
      0.1698
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -0.01224,
       "median": -0.008684,
       "ci": [
        -0.01789,
        0.007086
       ],
       "n": 119,
       "wins": 75,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 6
      }
     }
    },
    {
     "key": "lat_mae_bl",
     "label": "Lateral error at the model's own best lead",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1465,
     "median": 0.1403,
     "ci": [
      0.1261,
      0.1558
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -0.01796,
       "median": -0.01669,
       "ci": [
        -0.0234,
        -0.001124
       ],
       "n": 119,
       "wins": 91,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "lat_corr",
     "label": "Lateral correlation with the car's later motion",
     "unit": "",
     "better": "higher",
     "value": 0.8148,
     "median": 0.9046,
     "ci": [
      0.766,
      0.8344
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": 0.048,
       "median": 0.02074,
       "ci": [
        -0.00359,
        0.05862
       ],
       "n": 119,
       "wins": 92,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 5
      }
     }
    },
    {
     "key": "lat_gain",
     "label": "Lateral gain (1 = same size as the car's motion)",
     "unit": "",
     "better": "near1",
     "value": 0.8171,
     "median": 0.8394,
     "ci": [
      0.8031,
      0.8515
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02067,
       "median": -0.001727,
       "ci": [
        -0.03645,
        0.03838
       ],
       "n": 119,
       "wins": 66,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 5
      }
     }
    },
    {
     "key": "lat_rate_rms",
     "label": "Lateral action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.8794,
     "median": 0.7329,
     "ci": [
      0.6075,
      1.012
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -1.411,
       "median": -1.286,
       "ci": [
        -1.617,
        -0.5386
       ],
       "n": 119,
       "wins": 119,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 12
      }
     }
    },
    {
     "key": "lat_best_lag",
     "label": "Lead of the lateral request over the car's motion",
     "unit": "s",
     "better": "lower",
     "value": 0.6559,
     "median": 0.6,
     "ci": [
      0.619,
      0.7146
     ],
     "n": 118,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": 0.03941,
       "median": 0.05,
       "ci": [
        -0.0005324,
        0.1526
       ],
       "n": 118,
       "wins": 33,
       "ties": 17,
       "clusters": 12,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "lon_mae",
     "label": "Longitudinal error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.3108,
     "median": 0.2971,
     "ci": [
      0.2694,
      0.3481
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -0.05785,
       "median": -0.04622,
       "ci": [
        -0.08329,
        -0.0003201
       ],
       "n": 119,
       "wins": 95,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 8
      }
     }
    },
    {
     "key": "lon_corr",
     "label": "Longitudinal correlation",
     "unit": "",
     "better": "higher",
     "value": 0.4032,
     "median": 0.3972,
     "ci": [
      0.3341,
      0.6367
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": 0.07632,
       "median": 0.06769,
       "ci": [
        0.00605,
        0.09701
       ],
       "n": 119,
       "wins": 85,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "lon_brake_mae",
     "label": "Longitudinal error while braking",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.5878,
     "median": 0.5744,
     "ci": [
      0.5254,
      0.6296
     ],
     "n": 81,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": 0.08902,
       "median": 0.09478,
       "ci": [
        0.01014,
        0.1098
       ],
       "n": 81,
       "wins": 18,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "lon_rate_rms",
     "label": "Longitudinal action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 1.714,
     "median": 1.543,
     "ci": [
      1.364,
      1.937
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -3.896,
       "median": -3.31,
       "ci": [
        -4.922,
        -1.694
       ],
       "n": 119,
       "wins": 119,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 12
      }
     }
    },
    {
     "key": "v3_mae",
     "label": "Plan speed error at 3 s",
     "unit": "m/s",
     "better": "lower",
     "value": 1.831,
     "median": 1.841,
     "ci": [
      1.271,
      2.032
     ],
     "n": 119,
     "clusters": 12,
     "paired": {
      "lebowski_877m": {
       "delta": -0.1667,
       "median": -0.1689,
       "ci": [
        -0.2889,
        0.07267
       ],
       "n": 119,
       "wins": 76,
       "ties": 0,
       "clusters": 12,
       "cluster_wins": 7
      }
     }
    }
   ],
   "per_unit_file": null,
   "by_group": {
    "stock-aimedup": {
     "n": 4,
     "clusters": 2,
     "metrics": [
      {
       "key": "lat_mae",
       "value": 0.1289,
       "median": 0.1308,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.01226,
         "median": -0.01117,
         "ci": null,
         "n": 4,
         "wins": 3,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lat_mae_bl",
       "value": 0.1204,
       "median": 0.1258,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.01919,
         "median": -0.02067,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lat_corr",
       "value": 0.7948,
       "median": 0.8403,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.0003354,
         "median": 0.005528,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_gain",
       "value": 0.695,
       "median": 0.7414,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03105,
         "median": 0.02181,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_rate_rms",
       "value": 0.9038,
       "median": 0.9243,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2491,
         "median": -0.2286,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lat_best_lag",
       "value": 0.6375,
       "median": 0.625,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.125,
         "median": 0.1,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_mae",
       "value": 0.5623,
       "median": 0.5264,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2153,
         "median": -0.2516,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lon_corr",
       "value": 0.539,
       "median": 0.6781,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.003653,
         "median": -0.03281,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_brake_mae",
       "value": 0.3386,
       "median": 0.3386,
       "ci": null,
       "n": 2,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1506,
         "median": 0.1506,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_rate_rms",
       "value": 1.381,
       "median": 1.267,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -1.913,
         "median": -1.865,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "v3_mae",
       "value": 2.269,
       "median": 2.257,
       "ci": null,
       "n": 4,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.1204,
         "median": -0.2188,
         "ci": null,
         "n": 4,
         "wins": 3,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      }
     ]
    },
    "stock-dual": {
     "n": 13,
     "clusters": 4,
     "metrics": [
      {
       "key": "lat_mae",
       "value": 0.1491,
       "median": 0.1262,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": 0.003844,
         "median": 0.002255,
         "ci": null,
         "n": 13,
         "wins": 5,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_mae_bl",
       "value": 0.1346,
       "median": 0.1189,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": -0.004618,
         "median": -0.003351,
         "ci": null,
         "n": 13,
         "wins": 7,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lat_corr",
       "value": 0.8154,
       "median": 0.8259,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": 0.002069,
         "median": -0.004169,
         "ci": null,
         "n": 13,
         "wins": 6,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_gain",
       "value": 0.8818,
       "median": 0.8572,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": 0.04336,
         "median": 0.02993,
         "ci": null,
         "n": 13,
         "wins": 2,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_rate_rms",
       "value": 0.5657,
       "median": 0.5823,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": -0.6102,
         "median": -0.5815,
         "ci": null,
         "n": 13,
         "wins": 13,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 4
        }
       }
      },
      {
       "key": "lat_best_lag",
       "value": 0.6923,
       "median": 0.6,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1808,
         "median": 0.1,
         "ci": null,
         "n": 13,
         "wins": 0,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_mae",
       "value": 0.2559,
       "median": 0.2259,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": 0.02764,
         "median": 0.01754,
         "ci": null,
         "n": 13,
         "wins": 4,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_corr",
       "value": 0.6454,
       "median": 0.7549,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": 0.01121,
         "median": 0.03981,
         "ci": null,
         "n": 13,
         "wins": 8,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lon_brake_mae",
       "value": 0.6257,
       "median": 0.6606,
       "ci": null,
       "n": 6,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02161,
         "median": -0.03225,
         "ci": null,
         "n": 6,
         "wins": 3,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lon_rate_rms",
       "value": 1.047,
       "median": 1.06,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": -1.167,
         "median": -1.124,
         "ci": null,
         "n": 13,
         "wins": 13,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 4
        }
       }
      },
      {
       "key": "v3_mae",
       "value": 0.7788,
       "median": 0.6927,
       "ci": null,
       "n": 13,
       "clusters": 4,
       "paired": {
        "lebowski_877m": {
         "delta": -0.06241,
         "median": -0.03433,
         "ci": null,
         "n": 13,
         "wins": 8,
         "ties": 0,
         "clusters": 4,
         "cluster_wins": 3
        }
       }
      }
     ]
    },
    "stock-newcase": {
     "n": 2,
     "clusters": 1,
     "metrics": [
      {
       "key": "lat_mae",
       "value": 0.1589,
       "median": 0.1589,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.02463,
         "median": 0.02463,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_mae_bl",
       "value": 0.1547,
       "median": 0.1547,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.01952,
         "median": 0.01952,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_corr",
       "value": 0.8256,
       "median": 0.8256,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02196,
         "median": -0.02196,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_gain",
       "value": 0.7352,
       "median": 0.7352,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.0547,
         "median": 0.0547,
         "ci": null,
         "n": 2,
         "wins": 1,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_rate_rms",
       "value": 0.8108,
       "median": 0.8108,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.4648,
         "median": -0.4648,
         "ci": null,
         "n": 2,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_best_lag",
       "value": 0.525,
       "median": 0.525,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.025,
         "median": 0.025,
         "ci": null,
         "n": 2,
         "wins": 1,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_mae",
       "value": 0.3457,
       "median": 0.3457,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.01677,
         "median": -0.01677,
         "ci": null,
         "n": 2,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_corr",
       "value": 0.5894,
       "median": 0.5894,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1028,
         "median": 0.1028,
         "ci": null,
         "n": 2,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_brake_mae",
       "value": 0.4106,
       "median": 0.4106,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.09267,
         "median": -0.09267,
         "ci": null,
         "n": 2,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_rate_rms",
       "value": 1.697,
       "median": 1.697,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -1.66,
         "median": -1.66,
         "ci": null,
         "n": 2,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "v3_mae",
       "value": 2.496,
       "median": 2.496,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.4021,
         "median": 0.4021,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      }
     ]
    },
    "v13-8mm": {
     "n": 92,
     "clusters": 3,
     "metrics": [
      {
       "key": "lat_mae",
       "value": 0.1635,
       "median": 0.1561,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.01806,
         "median": -0.01341,
         "ci": null,
         "n": 92,
         "wins": 66,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lat_mae_bl",
       "value": 0.1491,
       "median": 0.1409,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02302,
         "median": -0.02198,
         "ci": null,
         "n": 92,
         "wins": 78,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lat_corr",
       "value": 0.8125,
       "median": 0.9164,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.06404,
         "median": 0.02833,
         "ci": null,
         "n": 92,
         "wins": 81,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lat_gain",
       "value": 0.811,
       "median": 0.8399,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03939,
         "median": -0.02305,
         "ci": null,
         "n": 92,
         "wins": 58,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lat_rate_rms",
       "value": 0.946,
       "median": 0.8222,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -1.672,
         "median": -1.516,
         "ci": null,
         "n": 92,
         "wins": 92,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lat_best_lag",
       "value": 0.6577,
       "median": 0.6,
       "ci": null,
       "n": 91,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.004396,
         "median": 0.0,
         "ci": null,
         "n": 91,
         "wins": 30,
         "ties": 16,
         "clusters": 3,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_mae",
       "value": 0.3052,
       "median": 0.2973,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.07079,
         "median": -0.06246,
         "ci": null,
         "n": 92,
         "wins": 81,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lon_corr",
       "value": 0.3302,
       "median": 0.3638,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.09653,
         "median": 0.08308,
         "ci": null,
         "n": 92,
         "wins": 70,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lon_brake_mae",
       "value": 0.591,
       "median": 0.5876,
       "ci": null,
       "n": 65,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1019,
         "median": 0.1088,
         "ci": null,
         "n": 65,
         "wins": 11,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_rate_rms",
       "value": 1.754,
       "median": 1.564,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -4.579,
         "median": -4.015,
         "ci": null,
         "n": 92,
         "wins": 92,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "v3_mae",
       "value": 2.004,
       "median": 1.963,
       "ci": null,
       "n": 92,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2299,
         "median": -0.289,
         "ci": null,
         "n": 92,
         "wins": 65,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 2
        }
       }
      }
     ]
    },
    "v14-8mm": {
     "n": 8,
     "clusters": 2,
     "metrics": [
      {
       "key": "lat_mae",
       "value": 0.1607,
       "median": 0.1675,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.01927,
         "median": 0.01785,
         "ci": null,
         "n": 8,
         "wins": 1,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_mae_bl",
       "value": 0.1469,
       "median": 0.1461,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.009802,
         "median": 0.008727,
         "ci": null,
         "n": 8,
         "wins": 2,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_corr",
       "value": 0.8478,
       "median": 0.9226,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02014,
         "median": -0.003379,
         "ci": null,
         "n": 8,
         "wins": 3,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_gain",
       "value": 0.8648,
       "median": 0.8458,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.07688,
         "median": 0.05096,
         "ci": null,
         "n": 8,
         "wins": 3,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_rate_rms",
       "value": 0.6282,
       "median": 0.5471,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.5274,
         "median": -0.4788,
         "ci": null,
         "n": 8,
         "wins": 8,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lat_best_lag",
       "value": 0.6188,
       "median": 0.55,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1687,
         "median": 0.15,
         "ci": null,
         "n": 8,
         "wins": 0,
         "ties": 1,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_mae",
       "value": 0.3299,
       "median": 0.2743,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.02054,
         "median": 0.008992,
         "ci": null,
         "n": 8,
         "wins": 4,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_corr",
       "value": 0.7341,
       "median": 0.7653,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02057,
         "median": -0.006125,
         "ci": null,
         "n": 8,
         "wins": 3,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lon_brake_mae",
       "value": 0.6576,
       "median": 0.6573,
       "ci": null,
       "n": 6,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1,
         "median": 0.03616,
         "ci": null,
         "n": 6,
         "wins": 2,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lon_rate_rms",
       "value": 2.506,
       "median": 1.844,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -2.027,
         "median": -1.988,
         "ci": null,
         "n": 8,
         "wins": 8,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "v3_mae",
       "value": 1.158,
       "median": 1.185,
       "ci": null,
       "n": 8,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.2253,
         "median": 0.1725,
         "ci": null,
         "n": 8,
         "wins": 0,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      }
     ]
    }
   }
  },
  "truck_closed_loop": {
   "status": "ok",
   "unit_of_evidence": "window",
   "n": 98,
   "clusters": 13,
   "condition": "Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. The simulation starts from the truck's state (speed, yaw rate, steering, gear) in its CAN recording of the 2 s before the window; one window, whose 2 s before are not all valid CAN, starts steady at its recorded speed. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 98 windows (59 day, 39 night).",
   "baselines": [
    "lebowski_877m"
   ],
   "metrics": [
    {
     "key": "xtrack_2s",
     "label": "Distance from the recorded path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.133,
     "median": 0.08905,
     "ci": [
      0.1068,
      0.1563
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.03655,
       "median": -0.034,
       "ci": [
        -0.0523,
        -0.02151
       ],
       "n": 98,
       "wins": 65,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 12
      }
     }
    },
    {
     "key": "xtrack_4s",
     "label": "Distance from the recorded path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.4845,
     "median": 0.2928,
     "ci": [
      0.3933,
      0.5573
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.03555,
       "median": -0.005007,
       "ci": [
        -0.1707,
        0.05102
       ],
       "n": 98,
       "wins": 52,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 5
      }
     }
    },
    {
     "key": "xtrack_7s",
     "label": "Distance from the recorded path at 7.2 s",
     "unit": "m",
     "better": "lower",
     "value": 1.935,
     "median": 0.795,
     "ci": [
      1.451,
      2.478
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.0448,
       "median": -0.001474,
       "ci": [
        -0.7158,
        0.6279
       ],
       "n": 98,
       "wins": 49,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 6
      }
     }
    },
    {
     "key": "fail_1m",
     "label": "Share of episodes ever more than 1 m off the path",
     "unit": "",
     "better": "lower",
     "value": 0.4184,
     "median": 0.0,
     "ci": [
      0.3278,
      0.5208
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.0102,
       "median": 0.0,
       "ci": [
        -0.06383,
        0.05661
       ],
       "n": 98,
       "wins": 13,
       "ties": 76,
       "clusters": 13,
       "cluster_wins": 6
      }
     }
    },
    {
     "key": "lat_jerk",
     "label": "Requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.2216,
     "median": 0.1613,
     "ci": [
      0.1971,
      0.2479
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.3203,
       "median": -0.2206,
       "ci": [
        -0.3838,
        -0.2595
       ],
       "n": 98,
       "wins": 97,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 13
      }
     }
    },
    {
     "key": "long_jerk",
     "label": "Requested longitudinal jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.2124,
     "median": 0.1841,
     "ci": [
      0.1841,
      0.2355
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.1739,
       "median": -0.1564,
       "ci": [
        -0.211,
        -0.1406
       ],
       "n": 98,
       "wins": 94,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 13
      }
     }
    },
    {
     "key": "speed_err",
     "label": "Speed error against the recorded speed, RMS",
     "unit": "m/s",
     "better": "lower",
     "value": 1.151,
     "median": 1.011,
     "ci": [
      0.9247,
      1.392
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": 0.06844,
       "median": 0.06296,
       "ci": [
        -0.122,
        0.2397
       ],
       "n": 98,
       "wins": 41,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 5
      }
     }
    }
   ],
   "per_unit_file": null,
   "by_group": {
    "stock-aimedup": {
     "n": 14,
     "clusters": 2,
     "metrics": [
      {
       "key": "xtrack_2s",
       "value": 0.1176,
       "median": 0.07986,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03276,
         "median": -0.02796,
         "ci": null,
         "n": 14,
         "wins": 9,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "xtrack_4s",
       "value": 0.5269,
       "median": 0.3386,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.07578,
         "median": 0.04471,
         "ci": null,
         "n": 14,
         "wins": 6,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "xtrack_7s",
       "value": 2.895,
       "median": 2.156,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.9899,
         "median": 0.1726,
         "ci": null,
         "n": 14,
         "wins": 7,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "fail_1m",
       "value": 0.7143,
       "median": 1.0,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.03571,
         "median": 0.0,
         "ci": null,
         "n": 14,
         "wins": 2,
         "ties": 8,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_jerk",
       "value": 0.3022,
       "median": 0.2456,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2686,
         "median": -0.2324,
         "ci": null,
         "n": 14,
         "wins": 14,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "long_jerk",
       "value": 0.2292,
       "median": 0.1988,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2655,
         "median": -0.2476,
         "ci": null,
         "n": 14,
         "wins": 14,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "speed_err",
       "value": 1.696,
       "median": 1.661,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.06741,
         "median": -0.2932,
         "ci": null,
         "n": 14,
         "wins": 8,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      }
     ]
    },
    "stock-dual": {
     "n": 43,
     "clusters": 5,
     "metrics": [
      {
       "key": "xtrack_2s",
       "value": 0.1412,
       "median": 0.1091,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02691,
         "median": -0.04432,
         "ci": null,
         "n": 43,
         "wins": 28,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 4
        }
       }
      },
      {
       "key": "xtrack_4s",
       "value": 0.4662,
       "median": 0.2765,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": 0.01397,
         "median": -0.00628,
         "ci": null,
         "n": 43,
         "wins": 24,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "xtrack_7s",
       "value": 1.68,
       "median": 0.578,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.3394,
         "median": -0.003365,
         "ci": null,
         "n": 43,
         "wins": 22,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "fail_1m",
       "value": 0.3372,
       "median": 0.0,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": 0.0,
         "median": 0.0,
         "ci": null,
         "n": 43,
         "wins": 6,
         "ties": 34,
         "clusters": 5,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "lat_jerk",
       "value": 0.2139,
       "median": 0.1577,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.3388,
         "median": -0.1903,
         "ci": null,
         "n": 43,
         "wins": 42,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 5
        }
       }
      },
      {
       "key": "long_jerk",
       "value": 0.2226,
       "median": 0.2088,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.1516,
         "median": -0.1339,
         "ci": null,
         "n": 43,
         "wins": 42,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 5
        }
       }
      },
      {
       "key": "speed_err",
       "value": 1.161,
       "median": 1.065,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": 0.07733,
         "median": 0.06042,
         "ci": null,
         "n": 43,
         "wins": 18,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 2
        }
       }
      }
     ]
    },
    "stock-newcase": {
     "n": 4,
     "clusters": 1,
     "metrics": [
      {
       "key": "xtrack_2s",
       "value": 0.06689,
       "median": 0.05782,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.05797,
         "median": -0.05892,
         "ci": null,
         "n": 4,
         "wins": 3,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "xtrack_4s",
       "value": 0.207,
       "median": 0.1482,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03367,
         "median": 0.01054,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "xtrack_7s",
       "value": 1.572,
       "median": 1.743,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.06417,
         "median": 0.04556,
         "ci": null,
         "n": 4,
         "wins": 1,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "fail_1m",
       "value": 0.625,
       "median": 0.75,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.0,
         "median": 0.0,
         "ci": null,
         "n": 4,
         "wins": 0,
         "ties": 4,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "lat_jerk",
       "value": 0.283,
       "median": 0.2185,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.4567,
         "median": -0.3322,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "long_jerk",
       "value": 0.1512,
       "median": 0.1596,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.238,
         "median": -0.1714,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "speed_err",
       "value": 0.9628,
       "median": 0.9525,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.284,
         "median": 0.2585,
         "ci": null,
         "n": 4,
         "wins": 1,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      }
     ]
    },
    "v13-8mm": {
     "n": 18,
     "clusters": 3,
     "metrics": [
      {
       "key": "xtrack_2s",
       "value": 0.1599,
       "median": 0.08159,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.0574,
         "median": -0.0586,
         "ci": null,
         "n": 18,
         "wins": 13,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "xtrack_4s",
       "value": 0.4758,
       "median": 0.2323,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.3263,
         "median": -0.04698,
         "ci": null,
         "n": 18,
         "wins": 12,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "xtrack_7s",
       "value": 1.345,
       "median": 0.8557,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -1.28,
         "median": 0.01542,
         "ci": null,
         "n": 18,
         "wins": 8,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "fail_1m",
       "value": 0.3889,
       "median": 0.0,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.0,
         "median": 0.0,
         "ci": null,
         "n": 18,
         "wins": 1,
         "ties": 16,
         "clusters": 3,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "lat_jerk",
       "value": 0.1691,
       "median": 0.1483,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2103,
         "median": -0.1654,
         "ci": null,
         "n": 18,
         "wins": 18,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "long_jerk",
       "value": 0.1423,
       "median": 0.1248,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.1622,
         "median": -0.1135,
         "ci": null,
         "n": 18,
         "wins": 18,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "speed_err",
       "value": 0.657,
       "median": 0.5213,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.2876,
         "median": 0.003953,
         "ci": null,
         "n": 18,
         "wins": 9,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 2
        }
       }
      }
     ]
    },
    "v14-8mm": {
     "n": 19,
     "clusters": 2,
     "metrics": [
      {
       "key": "xtrack_2s",
       "value": 0.114,
       "median": 0.07316,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03689,
         "median": -0.01644,
         "ci": null,
         "n": 19,
         "wins": 12,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "xtrack_4s",
       "value": 0.5611,
       "median": 0.3994,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.04539,
         "median": 0.02619,
         "ci": null,
         "n": 19,
         "wins": 8,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "xtrack_7s",
       "value": 2.439,
       "median": 0.6679,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 1.034,
         "median": -0.08086,
         "ci": null,
         "n": 19,
         "wins": 11,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "fail_1m",
       "value": 0.3684,
       "median": 0.0,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.07895,
         "median": 0.0,
         "ci": null,
         "n": 19,
         "wins": 4,
         "ties": 14,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "lat_jerk",
       "value": 0.2162,
       "median": 0.1904,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.3923,
         "median": -0.2666,
         "ci": null,
         "n": 19,
         "wins": 19,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "long_jerk",
       "value": 0.2559,
       "median": 0.2182,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.1545,
         "median": -0.1301,
         "ci": null,
         "n": 19,
         "wins": 16,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "speed_err",
       "value": 1.235,
       "median": 1.031,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.4403,
         "median": 0.357,
         "ci": null,
         "n": 19,
         "wins": 5,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      }
     ]
    }
   },
   "simulator": "g55-2026-10-31"
  },
  "truck_recovery": {
   "status": "ok",
   "unit_of_evidence": "window",
   "n": 100,
   "clusters": 14,
   "condition": "Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 98 windows (59 day, 39 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 98 windows (59 day, 39 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).",
   "baselines": [
    "lebowski_877m"
   ],
   "metrics": [
    {
     "key": "heading_xtrack_2s",
     "label": "2° off heading: distance from the path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.5012,
     "median": 0.4022,
     "ci": [
      0.4038,
      0.6216
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": 0.0483,
       "median": 0.03563,
       "ci": [
        0.007664,
        0.0942
       ],
       "n": 98,
       "wins": 43,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 5
      }
     }
    },
    {
     "key": "heading_xtrack_4s",
     "label": "2° off heading: distance from the path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.7497,
     "median": 0.5054,
     "ci": [
      0.5699,
      0.9721
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": 0.04719,
       "median": 0.05267,
       "ci": [
        -0.01703,
        0.1081
       ],
       "n": 98,
       "wins": 40,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "heading_lat_jerk",
     "label": "2° off heading: requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.3528,
     "median": 0.3163,
     "ci": [
      0.3183,
      0.3836
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.5341,
       "median": -0.4822,
       "ci": [
        -0.5889,
        -0.4756
       ],
       "n": 98,
       "wins": 98,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 13
      }
     }
    },
    {
     "key": "offcentre_xtrack_2s",
     "label": "0.5 m off centre: distance from the path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.2691,
     "median": 0.2389,
     "ci": [
      0.2426,
      0.2938
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": 0.002118,
       "median": 0.004116,
       "ci": [
        -0.01623,
        0.02294
       ],
       "n": 98,
       "wins": 47,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "offcentre_xtrack_4s",
     "label": "0.5 m off centre: distance from the path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.6509,
     "median": 0.4119,
     "ci": [
      0.5275,
      0.7686
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.01199,
       "median": -0.02061,
       "ci": [
        -0.08099,
        0.03616
       ],
       "n": 98,
       "wins": 52,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "offcentre_lat_jerk",
     "label": "0.5 m off centre: requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.4456,
     "median": 0.4219,
     "ci": [
      0.425,
      0.4653
     ],
     "n": 98,
     "clusters": 13,
     "paired": {
      "lebowski_877m": {
       "delta": -0.6648,
       "median": -0.615,
       "ci": [
        -0.7251,
        -0.5954
       ],
       "n": 98,
       "wins": 98,
       "ties": 0,
       "clusters": 13,
       "cluster_wins": 13
      }
     }
    },
    {
     "key": "brake_ahead_4s",
     "label": "Braking lead: distance ahead of the recorded car at 4 s",
     "unit": "m",
     "better": "lower",
     "value": -0.1657,
     "median": -0.1657,
     "ci": null,
     "n": 2,
     "clusters": 1,
     "paired": {
      "lebowski_877m": {
       "delta": 1.172,
       "median": 1.172,
       "ci": null,
       "n": 2,
       "wins": 0,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "brake_gap_min",
     "label": "Braking lead: smallest gap to the lead (derived)",
     "unit": "m",
     "better": "higher",
     "value": 21.93,
     "median": 21.93,
     "ci": null,
     "n": 2,
     "clusters": 1,
     "paired": {
      "lebowski_877m": {
       "delta": -5.53,
       "median": -5.53,
       "ci": null,
       "n": 2,
       "wins": 0,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "brake_speed_err",
     "label": "Braking lead: speed error against the recorded speed, RMS",
     "unit": "m/s",
     "better": "lower",
     "value": 1.153,
     "median": 1.153,
     "ci": null,
     "n": 2,
     "clusters": 1,
     "paired": {
      "lebowski_877m": {
       "delta": -0.05087,
       "median": -0.05087,
       "ci": null,
       "n": 2,
       "wins": 1,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 1
      }
     }
    }
   ],
   "per_unit_file": null,
   "by_group": {
    "stock-aimedup": {
     "n": 14,
     "clusters": 2,
     "metrics": [
      {
       "key": "heading_xtrack_2s",
       "value": 0.3549,
       "median": 0.3387,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03686,
         "median": -0.04573,
         "ci": null,
         "n": 14,
         "wins": 10,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "heading_xtrack_4s",
       "value": 0.5165,
       "median": 0.4773,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03651,
         "median": -0.0296,
         "ci": null,
         "n": 14,
         "wins": 8,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "heading_lat_jerk",
       "value": 0.3619,
       "median": 0.308,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.4061,
         "median": -0.362,
         "ci": null,
         "n": 14,
         "wins": 14,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "offcentre_xtrack_2s",
       "value": 0.229,
       "median": 0.238,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.03392,
         "median": -0.02852,
         "ci": null,
         "n": 14,
         "wins": 10,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "offcentre_xtrack_4s",
       "value": 0.5359,
       "median": 0.4626,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.05389,
         "median": -0.04008,
         "ci": null,
         "n": 14,
         "wins": 7,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "offcentre_lat_jerk",
       "value": 0.508,
       "median": 0.4914,
       "ci": null,
       "n": 14,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.5538,
         "median": -0.5789,
         "ci": null,
         "n": 14,
         "wins": 14,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      }
     ]
    },
    "stock-dual": {
     "n": 45,
     "clusters": 6,
     "metrics": [
      {
       "key": "heading_xtrack_2s",
       "value": 0.4363,
       "median": 0.3923,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": 0.02071,
         "median": -0.001991,
         "ci": null,
         "n": 43,
         "wins": 23,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "heading_xtrack_4s",
       "value": 0.7018,
       "median": 0.519,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": 0.06558,
         "median": 0.04856,
         "ci": null,
         "n": 43,
         "wins": 19,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "heading_lat_jerk",
       "value": 0.3296,
       "median": 0.2937,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.5668,
         "median": -0.4841,
         "ci": null,
         "n": 43,
         "wins": 43,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 5
        }
       }
      },
      {
       "key": "offcentre_xtrack_2s",
       "value": 0.2939,
       "median": 0.2471,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.005393,
         "median": -0.007592,
         "ci": null,
         "n": 43,
         "wins": 23,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "offcentre_xtrack_4s",
       "value": 0.7085,
       "median": 0.4564,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": 0.03811,
         "median": -0.01909,
         "ci": null,
         "n": 43,
         "wins": 23,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "offcentre_lat_jerk",
       "value": 0.4418,
       "median": 0.4111,
       "ci": null,
       "n": 43,
       "clusters": 5,
       "paired": {
        "lebowski_877m": {
         "delta": -0.7348,
         "median": -0.6937,
         "ci": null,
         "n": 43,
         "wins": 43,
         "ties": 0,
         "clusters": 5,
         "cluster_wins": 5
        }
       }
      },
      {
       "key": "brake_ahead_4s",
       "value": -0.1657,
       "median": -0.1657,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 1.172,
         "median": 1.172,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "brake_gap_min",
       "value": 21.93,
       "median": 21.93,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -5.53,
         "median": -5.53,
         "ci": null,
         "n": 2,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "brake_speed_err",
       "value": 1.153,
       "median": 1.153,
       "ci": null,
       "n": 2,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.05087,
         "median": -0.05087,
         "ci": null,
         "n": 2,
         "wins": 1,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      }
     ]
    },
    "stock-newcase": {
     "n": 4,
     "clusters": 1,
     "metrics": [
      {
       "key": "heading_xtrack_2s",
       "value": 0.3846,
       "median": 0.3806,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.06596,
         "median": -0.08693,
         "ci": null,
         "n": 4,
         "wins": 3,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "heading_xtrack_4s",
       "value": 0.4011,
       "median": 0.3415,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": 0.1814,
         "median": 0.09102,
         "ci": null,
         "n": 4,
         "wins": 0,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "heading_lat_jerk",
       "value": 0.3544,
       "median": 0.3146,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.6178,
         "median": -0.605,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "offcentre_xtrack_2s",
       "value": 0.2719,
       "median": 0.2376,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.02794,
         "median": -0.02917,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "offcentre_xtrack_4s",
       "value": 0.7241,
       "median": 0.5878,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.003526,
         "median": -0.005466,
         "ci": null,
         "n": 4,
         "wins": 2,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "offcentre_lat_jerk",
       "value": 0.5142,
       "median": 0.5281,
       "ci": null,
       "n": 4,
       "clusters": 1,
       "paired": {
        "lebowski_877m": {
         "delta": -0.8831,
         "median": -0.584,
         "ci": null,
         "n": 4,
         "wins": 4,
         "ties": 0,
         "clusters": 1,
         "cluster_wins": 1
        }
       }
      }
     ]
    },
    "v13-8mm": {
     "n": 18,
     "clusters": 3,
     "metrics": [
      {
       "key": "heading_xtrack_2s",
       "value": 0.8955,
       "median": 0.8541,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.2025,
         "median": 0.2209,
         "ci": null,
         "n": 18,
         "wins": 2,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "heading_xtrack_4s",
       "value": 1.233,
       "median": 0.8155,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.001217,
         "median": 0.1051,
         "ci": null,
         "n": 18,
         "wins": 7,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "heading_lat_jerk",
       "value": 0.418,
       "median": 0.3767,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.455,
         "median": -0.4182,
         "ci": null,
         "n": 18,
         "wins": 18,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "offcentre_xtrack_2s",
       "value": 0.2797,
       "median": 0.2419,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": 0.02166,
         "median": 0.06343,
         "ci": null,
         "n": 18,
         "wins": 7,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 1
        }
       }
      },
      {
       "key": "offcentre_xtrack_4s",
       "value": 0.525,
       "median": 0.254,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.168,
         "median": -0.06758,
         "ci": null,
         "n": 18,
         "wins": 11,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      },
      {
       "key": "offcentre_lat_jerk",
       "value": 0.3959,
       "median": 0.3783,
       "ci": null,
       "n": 18,
       "clusters": 3,
       "paired": {
        "lebowski_877m": {
         "delta": -0.4859,
         "median": -0.4939,
         "ci": null,
         "n": 18,
         "wins": 18,
         "ties": 0,
         "clusters": 3,
         "cluster_wins": 3
        }
       }
      }
     ]
    },
    "v14-8mm": {
     "n": 19,
     "clusters": 2,
     "metrics": [
      {
       "key": "heading_xtrack_2s",
       "value": 0.407,
       "median": 0.4004,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.05147,
         "median": 0.04509,
         "ci": null,
         "n": 19,
         "wins": 5,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "heading_xtrack_4s",
       "value": 0.646,
       "median": 0.3697,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.08252,
         "median": 0.04882,
         "ci": null,
         "n": 19,
         "wins": 6,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "heading_lat_jerk",
       "value": 0.3366,
       "median": 0.2848,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.6118,
         "median": -0.552,
         "ci": null,
         "n": 19,
         "wins": 19,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      },
      {
       "key": "offcentre_xtrack_2s",
       "value": 0.232,
       "median": 0.1926,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.03349,
         "median": 0.06057,
         "ci": null,
         "n": 19,
         "wins": 5,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "offcentre_xtrack_4s",
       "value": 0.709,
       "median": 0.4073,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": 0.05152,
         "median": 0.009246,
         "ci": null,
         "n": 19,
         "wins": 9,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 0
        }
       }
      },
      {
       "key": "offcentre_lat_jerk",
       "value": 0.441,
       "median": 0.4167,
       "ci": null,
       "n": 19,
       "clusters": 2,
       "paired": {
        "lebowski_877m": {
         "delta": -0.7116,
         "median": -0.6498,
         "ci": null,
         "n": 19,
         "wins": 19,
         "ties": 0,
         "clusters": 2,
         "cluster_wins": 2
        }
       }
      }
     ]
    }
   },
   "simulator": "g55-2026-10-31"
  }
 },
 "stills": [],
 "limits": [
  "Open-loop scores compare a model's action with what the recording driver or car did; a model that drives differently but well is scored as wrong. comma's RL residual makes the lateral action smoother, and earlier than its base's by 0.10 to 0.26 s (measured on the G55 and on 16 of comma's cars); both score as error here. Measured on the 2026-10-27 deep dive: open-loop accuracy did not predict closed-loop driving.",
  "Closed-loop numbers come from a generative world model, not a car. In a separate test on the G55's windows, worldmodel-4B followed an added yaw command through 2 s but showed 0 to 20% of an added lateral offset 3 to 4 s later; past about 4 s the recorded anchor frames also pull every model back toward the recorded path. The front page reports 2 s.",
  "Intervals are 95% percentile bootstraps that resample whole drives (the drive a public segment was cut from, the recorded drive a truck window comes from), so segments of one drive count once. Below 10 drives there is no interval: the braking-lead test comes from one drive and reports means and win counts only.",
  "Paired deltas are model minus baseline on the units both have (for lateral gain, the change in its distance from 1); wins count units where the model is better in the metric's direction.",
  "The lead of the lateral request is the shift, searched from 0 to 1.5 s, at which the request best correlates with the car's later lateral motion; a larger value means the request comes earlier. A value at either edge is censored and left out. Plan speed error at 3 s reads the plan head, which does not separate models built on one frozen base: it is identical on every segment for the four f22e06e5 models, and for Cinque v2 and v3.",
  "Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.",
  "Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre. The pooled number mixes camera setups, so read the per-setup numbers too. On the V14 setup the wide frame can be up to 90 ms from the simulated one (its camera recorded 7-9 fps).",
  "Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test has 2 windows from one drive, whose recorded calibration was carried over from an earlier drive."
 ]
}