{ "schema": 1, "model": { "id": "870a4823_12864", "name": null, "sha256": "c392ba1aa1b4b89e850366024fa6c1beac1f1fdc3a7182e3c3b0234c55e08eeb", "source": { "kind": "huggingface", "ref": "af05983f67bf9adb4df004dd8382d2aae5763ed4", "url": "https://huggingface.co/commaai/openpilot_driving_models/blob/af05983f67bf9adb4df004dd8382d2aae5763ed4/870a4823-d2ec-4b9a-8015-b947e9901fc7/12864/big_driving_supercombo.onnx", "date": "2026-10-08" }, "first_public": "2026-10-08", "on_master": null, "params_m": 382.9, "gmac": 50.1, "family": "residual", "notes": "Uploaded as \"mountain dew model\"; never shipped. Vision base f22e06e5 with an RL residual head." }, "scored": { "date": "2026-10-31T23:59:30Z", "suite": "geval-1", "code": "d8eb40eeba87" }, "baselines": [ "lebowski_877m", "f78ed37d_12864" ], "sections": { "public_open_loop": { "status": "ok", "unit_of_evidence": "segment", "n": 39, "clusters": 23, "condition": "Measured. comma's public two-camera segments (openpilot CI data), native intrinsics and the device's own calibration, both cameras; every model on the same decoded frames. Truth is what the recording car did next: carState speed and acceleration, yaw rate from the device's cameraOdometry. Segments where the car never moves have no data. 39 of the 45 listed segments have a score, from 23 drives and 20 cars.", "baselines": [ "lebowski_877m", "f78ed37d_12864" ], "metrics": [ { "key": "lat_mae", "label": "Lateral error", "unit": "m/s²", "better": "lower", "value": 0.1337, "median": 0.1188, "ci": [ 0.1063, 0.1629 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.02496, "median": 0.01769, "ci": [ 0.01642, 0.03338 ], "n": 39, "wins": 8, "ties": 0, "clusters": 23, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": -0.00279, "median": -0.0009645, "ci": [ -0.007407, 0.001925 ], "n": 39, "wins": 21, "ties": 0, "clusters": 23, "cluster_wins": 13 } } }, { "key": "lat_mae_bl", "label": "Lateral error at the model's own best lead", "unit": "m/s²", "better": "lower", "value": 0.126, "median": 0.1112, "ci": [ 0.09889, 0.1544 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.02456, "median": 0.01718, "ci": [ 0.01407, 0.03454 ], "n": 39, "wins": 9, "ties": 0, "clusters": 23, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": -0.002312, "median": 0.0004215, "ci": [ -0.00746, 0.002915 ], "n": 39, "wins": 19, "ties": 0, "clusters": 23, "cluster_wins": 10 } } }, { "key": "lat_corr", "label": "Lateral correlation with the car's later motion", "unit": "", "better": "higher", "value": 0.8406, "median": 0.9447, "ci": [ 0.7617, 0.912 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.01702, "median": -0.01057, "ci": [ -0.04895, 0.01102 ], "n": 39, "wins": 8, "ties": 0, "clusters": 23, "cluster_wins": 7 }, "f78ed37d_12864": { "delta": 0.01354, "median": 0.00104, "ci": [ -0.003716, 0.03612 ], "n": 39, "wins": 25, "ties": 0, "clusters": 23, "cluster_wins": 15 } } }, { "key": "lat_gain", "label": "Lateral gain (1 = same size as the car's motion)", "unit": "", "better": "near1", "value": 0.7608, "median": 0.7654, "ci": [ 0.684, 0.8436 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.06112, "median": 0.03989, "ci": [ 0.00375, 0.117 ], "n": 39, "wins": 11, "ties": 0, "clusters": 23, "cluster_wins": 6 }, "f78ed37d_12864": { "delta": -0.003537, "median": -0.004933, "ci": [ -0.03556, 0.03096 ], "n": 39, "wins": 20, "ties": 0, "clusters": 23, "cluster_wins": 10 } } }, { "key": "lat_rate_rms", "label": "Lateral action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.4383, "median": 0.3907, "ci": [ 0.3455, 0.546 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.2256, "median": -0.2414, "ci": [ -0.2721, -0.1762 ], "n": 39, "wins": 37, "ties": 0, "clusters": 23, "cluster_wins": 22 }, "f78ed37d_12864": { "delta": 0.01599, "median": -0.008388, "ci": [ -0.01343, 0.05159 ], "n": 39, "wins": 23, "ties": 0, "clusters": 23, "cluster_wins": 13 } } }, { "key": "lat_best_lag", "label": "Lead of the lateral request over the car's motion", "unit": "s", "better": "lower", "value": 0.5139, "median": 0.4, "ci": [ 0.4139, 0.6265 ], "n": 36, "clusters": 22, "paired": { "lebowski_877m": { "delta": 0.1229, "median": 0.05, "ci": [ 0.04545, 0.2194 ], "n": 35, "wins": 10, "ties": 5, "clusters": 22, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.005882, "median": 0.0, "ci": [ -0.05789, 0.08194 ], "n": 34, "wins": 11, "ties": 8, "clusters": 20, "cluster_wins": 10 } } }, { "key": "lon_mae", "label": "Longitudinal error", "unit": "m/s²", "better": "lower", "value": 0.3091, "median": 0.2858, "ci": [ 0.2692, 0.3546 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.0009908, "median": 0.002596, "ci": [ -0.01393, 0.01852 ], "n": 39, "wins": 18, "ties": 0, "clusters": 23, "cluster_wins": 10 }, "f78ed37d_12864": { "delta": 0.007458, "median": 0.006105, "ci": [ -0.0008081, 0.01707 ], "n": 39, "wins": 16, "ties": 0, "clusters": 23, "cluster_wins": 9 } } }, { "key": "lon_corr", "label": "Longitudinal correlation", "unit": "", "better": "higher", "value": 0.7319, "median": 0.8473, "ci": [ 0.6342, 0.8182 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.01415, "median": 0.01157, "ci": [ -0.01263, 0.03922 ], "n": 39, "wins": 22, "ties": 0, "clusters": 23, "cluster_wins": 12 }, "f78ed37d_12864": { "delta": 0.002636, "median": 0.002321, "ci": [ -0.01561, 0.02212 ], "n": 39, "wins": 22, "ties": 0, "clusters": 23, "cluster_wins": 14 } } }, { "key": "lon_brake_mae", "label": "Longitudinal error while braking", "unit": "m/s²", "better": "lower", "value": 0.4116, "median": 0.3857, "ci": [ 0.3503, 0.4829 ], "n": 30, "clusters": 21, "paired": { "lebowski_877m": { "delta": -0.02264, "median": -0.001033, "ci": [ -0.04418, 0.001824 ], "n": 30, "wins": 15, "ties": 0, "clusters": 21, "cluster_wins": 12 }, "f78ed37d_12864": { "delta": 0.003995, "median": -0.01227, "ci": [ -0.01947, 0.02957 ], "n": 30, "wins": 17, "ties": 0, "clusters": 21, "cluster_wins": 13 } } }, { "key": "lon_rate_rms", "label": "Longitudinal action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.8937, "median": 0.7763, "ci": [ 0.7786, 1.028 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.4393, "median": -0.3884, "ci": [ -0.5715, -0.3257 ], "n": 39, "wins": 36, "ties": 0, "clusters": 23, "cluster_wins": 21 }, "f78ed37d_12864": { "delta": 0.0132, "median": -0.01202, "ci": [ -0.048, 0.08622 ], "n": 39, "wins": 21, "ties": 0, "clusters": 23, "cluster_wins": 13 } } }, { "key": "v3_mae", "label": "Plan speed error at 3 s", "unit": "m/s", "better": "lower", "value": 0.7602, "median": 0.7121, "ci": [ 0.6567, 0.888 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.04442, "median": 0.04624, "ci": [ 0.009032, 0.07746 ], "n": 39, "wins": 12, "ties": 0, "clusters": 23, "cluster_wins": 8 }, "f78ed37d_12864": { "delta": 0.001718, "median": 0.01899, "ci": [ -0.03573, 0.03288 ], "n": 39, "wins": 13, "ties": 0, "clusters": 23, "cluster_wins": 9 } } } ], "per_unit_file": "per_unit/870a4823_12864/public_open_loop.csv" }, "public_closed_loop": { "status": "ok", "unit_of_evidence": "segment", "n": 28, "clusters": 19, "condition": "Measured. comma's worldmodel-4B generates each next frame at 5 Hz from 9 history frames, 5 recorded anchor frames and the car's pose; the model drives comma's ideal car (the requested curvature and acceleration happen at once) for 7.2 s from the recorded start. Distance is to the recording car's own path. One window per segment, nearest the middle, where the car holds at least 5 m/s for the window and the 4.8 s before it; segments with no such stretch are left out. Seeds 0 and 1, averaged per segment. 28 of the 45 listed segments have a score, from 19 drives and 17 cars.", "baselines": [ "lebowski_877m", "f78ed37d_12864" ], "metrics": [ { "key": "xtrack_2s", "label": "Distance from the recorded path at 2 s", "unit": "m", "better": "lower", "value": 0.09792, "median": 0.05565, "ci": [ 0.0528, 0.1689 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.04977, "median": -0.01049, "ci": [ -0.1168, -0.002068 ], "n": 28, "wins": 17, "ties": 0, "clusters": 19, "cluster_wins": 13 }, "f78ed37d_12864": { "delta": -0.02984, "median": -0.005753, "ci": [ -0.08265, 0.007196 ], "n": 28, "wins": 15, "ties": 0, "clusters": 19, "cluster_wins": 12 } } }, { "key": "xtrack_4s", "label": "Distance from the recorded path at 4 s", "unit": "m", "better": "lower", "value": 0.1731, "median": 0.0938, "ci": [ 0.1054, 0.2574 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.1052, "median": 0.00328, "ci": [ -0.2689, 0.004132 ], "n": 28, "wins": 14, "ties": 0, "clusters": 19, "cluster_wins": 10 }, "f78ed37d_12864": { "delta": -0.111, "median": -0.03133, "ci": [ -0.2689, -0.01397 ], "n": 28, "wins": 20, "ties": 0, "clusters": 19, "cluster_wins": 13 } } }, { "key": "xtrack_7s", "label": "Distance from the recorded path at 7.2 s", "unit": "m", "better": "lower", "value": 0.9405, "median": 0.3574, "ci": [ 0.3398, 1.702 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.1035, "median": 0.02126, "ci": [ -0.6146, 0.3414 ], "n": 28, "wins": 13, "ties": 0, "clusters": 19, "cluster_wins": 8 }, "f78ed37d_12864": { "delta": -0.4477, "median": -0.03905, "ci": [ -1.267, 0.0404 ], "n": 28, "wins": 17, "ties": 0, "clusters": 19, "cluster_wins": 13 } } }, { "key": "fail_1m", "label": "Share of episodes ever more than 1 m off the path", "unit": "", "better": "lower", "value": 0.2143, "median": 0.0, "ci": [ 0.07143, 0.375 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.01786, "median": 0.0, "ci": [ -0.1111, 0.05172 ], "n": 28, "wins": 1, "ties": 26, "clusters": 19, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.05357, "median": 0.0, "ci": [ -0.1739, 0.03704 ], "n": 28, "wins": 2, "ties": 25, "clusters": 19, "cluster_wins": 2 } } }, { "key": "lat_jerk", "label": "Requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.1818, "median": 0.09545, "ci": [ 0.1174, 0.2544 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.1438, "median": -0.09866, "ci": [ -0.1886, -0.09916 ], "n": 28, "wins": 26, "ties": 0, "clusters": 19, "cluster_wins": 17 }, "f78ed37d_12864": { "delta": 0.01084, "median": -0.007119, "ci": [ -0.01679, 0.04663 ], "n": 28, "wins": 16, "ties": 0, "clusters": 19, "cluster_wins": 10 } } }, { "key": "long_jerk", "label": "Requested longitudinal jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3171, "median": 0.2772, "ci": [ 0.2428, 0.4087 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.02933, "median": -0.05022, "ci": [ -0.07063, 0.02328 ], "n": 28, "wins": 21, "ties": 0, "clusters": 19, "cluster_wins": 14 }, "f78ed37d_12864": { "delta": 0.01151, "median": 0.00638, "ci": [ -0.044, 0.07929 ], "n": 28, "wins": 12, "ties": 0, "clusters": 19, "cluster_wins": 8 } } }, { "key": "speed_err", "label": "Speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.234, "median": 0.9326, "ci": [ 0.8866, 1.646 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": 0.1326, "median": 0.1724, "ci": [ -0.06616, 0.3449 ], "n": 28, "wins": 9, "ties": 0, "clusters": 19, "cluster_wins": 7 }, "f78ed37d_12864": { "delta": -0.1243, "median": -0.006293, "ci": [ -0.3508, 0.09063 ], "n": 28, "wins": 14, "ties": 0, "clusters": 19, "cluster_wins": 11 } } } ], "per_unit_file": "per_unit/870a4823_12864/public_closed_loop.csv" }, "truck_open_loop": { "status": "ok", "unit_of_evidence": "window", "n": 119, "clusters": 12, "condition": "Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.", "baselines": [ "lebowski_877m", "f78ed37d_12864" ], "metrics": [ { "key": "lat_mae", "label": "Lateral error", "unit": "m/s²", "better": "lower", "value": 0.1573, "median": 0.1491, "ci": [ 0.1286, 0.1687 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.01546, "median": -0.01306, "ci": [ -0.02156, -0.00391 ], "n": 119, "wins": 86, "ties": 0, "clusters": 12, "cluster_wins": 8 }, "f78ed37d_12864": { "delta": -0.009318, "median": -0.009106, "ci": [ -0.02154, -0.006384 ], "n": 119, "wins": 85, "ties": 0, "clusters": 12, "cluster_wins": 11 } } }, { "key": "lat_mae_bl", "label": "Lateral error at the model's own best lead", "unit": "m/s²", "better": "lower", "value": 0.1438, "median": 0.1292, "ci": [ 0.1188, 0.1536 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.02068, "median": -0.01854, "ci": [ -0.02495, -0.009532 ], "n": 119, "wins": 96, "ties": 0, "clusters": 12, "cluster_wins": 9 }, "f78ed37d_12864": { "delta": -0.01262, "median": -0.01206, "ci": [ -0.02235, -0.0083 ], "n": 119, "wins": 93, "ties": 0, "clusters": 12, "cluster_wins": 12 } } }, { "key": "lat_corr", "label": "Lateral correlation with the car's later motion", "unit": "", "better": "higher", "value": 0.8169, "median": 0.9106, "ci": [ 0.7759, 0.8447 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.05013, "median": 0.0197, "ci": [ 0.01873, 0.05944 ], "n": 119, "wins": 93, "ties": 0, "clusters": 12, "cluster_wins": 9 }, "f78ed37d_12864": { "delta": -0.003051, "median": -0.003433, "ci": [ -0.008442, 0.01938 ], "n": 119, "wins": 42, "ties": 0, "clusters": 12, "cluster_wins": 7 } } }, { "key": "lat_gain", "label": "Lateral gain (1 = same size as the car's motion)", "unit": "", "better": "near1", "value": 0.8087, "median": 0.8373, "ci": [ 0.7921, 0.8786 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.01465, "median": -0.00623, "ci": [ -0.02963, 0.01649 ], "n": 119, "wins": 63, "ties": 0, "clusters": 12, "cluster_wins": 6 }, "f78ed37d_12864": { "delta": -0.005709, "median": -0.007839, "ci": [ -0.04607, 0.009177 ], "n": 119, "wins": 66, "ties": 0, "clusters": 12, "cluster_wins": 10 } } }, { "key": "lat_rate_rms", "label": "Lateral action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 1.084, "median": 0.8072, "ci": [ 0.5967, 1.321 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -1.206, "median": -1.135, "ci": [ -1.332, -0.5569 ], "n": 119, "wins": 118, "ties": 0, "clusters": 12, "cluster_wins": 12 }, "f78ed37d_12864": { "delta": 0.1647, "median": 0.07102, "ci": [ -0.03871, 0.2398 ], "n": 119, "wins": 39, "ties": 0, "clusters": 12, "cluster_wins": 7 } } }, { "key": "lat_best_lag", "label": "Lead of the lateral request over the car's motion", "unit": "s", "better": "lower", "value": 0.6504, "median": 0.6, "ci": [ 0.5778, 0.702 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.02815, "median": 0.05, "ci": [ -0.04205, 0.109 ], "n": 119, "wins": 36, "ties": 23, "clusters": 12, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.02966, "median": 0.0, "ci": [ -0.03273, 0.05448 ], "n": 118, "wins": 18, "ties": 46, "clusters": 12, "cluster_wins": 4 } } }, { "key": "lon_mae", "label": "Longitudinal error", "unit": "m/s²", "better": "lower", "value": 0.358, "median": 0.3533, "ci": [ 0.2771, 0.4087 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.01069, "median": -0.01331, "ci": [ -0.0232, 0.01638 ], "n": 119, "wins": 69, "ties": 0, "clusters": 12, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": 0.03727, "median": 0.02946, "ci": [ 0.009631, 0.05358 ], "n": 119, "wins": 19, "ties": 0, "clusters": 12, "cluster_wins": 0 } } }, { "key": "lon_corr", "label": "Longitudinal correlation", "unit": "", "better": "higher", "value": 0.3376, "median": 0.2921, "ci": [ 0.2835, 0.5978 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.01072, "median": 0.02445, "ci": [ -0.0241, 0.04796 ], "n": 119, "wins": 68, "ties": 0, "clusters": 12, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": -0.05462, "median": -0.03671, "ci": [ -0.08073, -0.001 ], "n": 119, "wins": 40, "ties": 0, "clusters": 12, "cluster_wins": 3 } } }, { "key": "lon_brake_mae", "label": "Longitudinal error while braking", "unit": "m/s²", "better": "lower", "value": 0.4722, "median": 0.4651, "ci": [ 0.4401, 0.5886 ], "n": 81, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.02665, "median": -0.03469, "ci": [ -0.05073, 0.03018 ], "n": 81, "wins": 51, "ties": 0, "clusters": 12, "cluster_wins": 7 }, "f78ed37d_12864": { "delta": -0.01847, "median": -0.02161, "ci": [ -0.0317, 0.02596 ], "n": 81, "wins": 51, "ties": 0, "clusters": 12, "cluster_wins": 5 } } }, { "key": "lon_rate_rms", "label": "Longitudinal action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 4.463, "median": 4.332, "ci": [ 2.069, 5.795 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -1.147, "median": -1.238, "ci": [ -1.714, -0.7655 ], "n": 119, "wins": 90, "ties": 0, "clusters": 12, "cluster_wins": 11 }, "f78ed37d_12864": { "delta": 2.089, "median": 1.828, "ci": [ 0.334, 3.155 ], "n": 119, "wins": 13, "ties": 0, "clusters": 12, "cluster_wins": 3 } } }, { "key": "v3_mae", "label": "Plan speed error at 3 s", "unit": "m/s", "better": "lower", "value": 1.889, "median": 1.838, "ci": [ 1.152, 2.301 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.1086, "median": -0.1249, "ci": [ -0.3518, 0.306 ], "n": 119, "wins": 78, "ties": 0, "clusters": 12, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": 0.05808, "median": 0.03675, "ci": [ -0.1388, 0.3123 ], "n": 119, "wins": 52, "ties": 0, "clusters": 12, "cluster_wins": 7 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 4, "clusters": 2, "metrics": [ { "key": "lat_mae", "value": 0.1067, "median": 0.101, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.03441, "median": -0.04299, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.04368, "median": -0.04471, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_mae_bl", "value": 0.1017, "median": 0.1002, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.03791, "median": -0.0483, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.03987, "median": -0.03391, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_corr", "value": 0.8343, "median": 0.903, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.03917, "median": 0.02948, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.09025, "median": 0.1123, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_gain", "value": 0.7722, "median": 0.8463, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1082, "median": -0.0831, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.1754, "median": -0.2323, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_rate_rms", "value": 0.8354, "median": 0.7923, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3175, "median": -0.3364, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.2664, "median": -0.2844, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.575, "median": 0.525, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.0625, "median": 0.125, "ci": null, "n": 4, "wins": 1, "ties": 1, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.1375, "median": -0.225, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_mae", "value": 0.6866, "median": 0.6522, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.091, "median": -0.1014, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.1047, "median": 0.1222, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.5942, "median": 0.6775, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.05877, "median": 0.04534, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.0408, "median": 0.04342, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_brake_mae", "value": 0.2525, "median": 0.2525, "ci": null, "n": 2, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.0645, "median": 0.0645, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.02647, "median": -0.02647, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_rate_rms", "value": 2.041, "median": 1.925, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -1.253, "median": -1.184, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.4827, "median": 0.438, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 1.443, "median": 1.507, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.9457, "median": -0.9688, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.8253, "median": -0.75, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 } } } ] }, "stock-dual": { "n": 13, "clusters": 4, "metrics": [ { "key": "lat_mae", "value": 0.1406, "median": 0.1245, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.004567, "median": -0.002744, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.01426, "median": -0.01058, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lat_mae_bl", "value": 0.1288, "median": 0.1162, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.01042, "median": -0.007014, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.01673, "median": -0.01432, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 4 } } }, { "key": "lat_corr", "value": 0.8409, "median": 0.8586, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.02755, "median": 0.001447, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.008941, "median": 0.001106, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lat_gain", "value": 0.9181, "median": 0.8719, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.02616, "median": 0.02494, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.02754, "median": -0.01214, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 4 } } }, { "key": "lat_rate_rms", "value": 0.5937, "median": 0.5646, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.5822, "median": -0.5462, "ci": null, "n": 13, "wins": 13, "ties": 0, "clusters": 4, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.04914, "median": 0.03037, "ci": null, "n": 13, "wins": 3, "ties": 0, "clusters": 4, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.65, "median": 0.65, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.1385, "median": 0.1, "ci": null, "n": 13, "wins": 2, "ties": 2, "clusters": 4, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.05385, "median": 0.0, "ci": null, "n": 13, "wins": 2, "ties": 5, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lon_mae", "value": 0.2654, "median": 0.2676, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.0372, "median": 0.02865, "ci": null, "n": 13, "wins": 2, "ties": 0, "clusters": 4, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.02303, "median": 0.01841, "ci": null, "n": 13, "wins": 2, "ties": 0, "clusters": 4, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.5899, "median": 0.7324, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.04425, "median": 0.004358, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.07229, "median": -0.0266, "ci": null, "n": 13, "wins": 3, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lon_brake_mae", "value": 0.6002, "median": 0.6584, "ci": null, "n": 6, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.04712, "median": -0.04838, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.005405, "median": 0.00716, "ci": null, "n": 6, "wins": 2, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lon_rate_rms", "value": 1.181, "median": 1.185, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -1.033, "median": -1.156, "ci": null, "n": 13, "wins": 13, "ties": 0, "clusters": 4, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.01051, "median": 0.01901, "ci": null, "n": 13, "wins": 6, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "v3_mae", "value": 0.8066, "median": 0.8323, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.03457, "median": -0.06044, "ci": null, "n": 13, "wins": 8, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.02784, "median": 0.04065, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 } } } ] }, "stock-newcase": { "n": 2, "clusters": 1, "metrics": [ { "key": "lat_mae", "value": 0.1304, "median": 0.1304, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.003891, "median": -0.003891, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.03135, "median": -0.03135, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_mae_bl", "value": 0.1259, "median": 0.1259, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.009307, "median": -0.009307, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.03421, "median": -0.03421, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_corr", "value": 0.8351, "median": 0.8351, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.01243, "median": -0.01243, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.0001777, "median": 0.0001777, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_gain", "value": 0.8176, "median": 0.8176, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.02766, "median": -0.02766, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.05329, "median": -0.05329, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "value": 0.774, "median": 0.774, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.5016, "median": -0.5016, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.04244, "median": -0.04244, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_best_lag", "value": 0.475, "median": 0.475, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.025, "median": -0.025, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 2, "wins": 0, "ties": 2, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.3675, "median": 0.3675, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.005027, "median": 0.005027, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.0165, "median": 0.0165, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.5575, "median": 0.5575, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.07087, "median": 0.07087, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.02508, "median": -0.02508, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_brake_mae", "value": 0.3833, "median": 0.3833, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.12, "median": -0.12, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.02548, "median": 0.02548, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_rate_rms", "value": 2.425, "median": 2.425, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.9314, "median": -0.9314, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.491, "median": 0.491, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 2.129, "median": 2.129, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.03487, "median": 0.03487, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.3672, "median": -0.3672, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } } ] }, "v13-8mm": { "n": 92, "clusters": 3, "metrics": [ { "key": "lat_mae", "value": 0.1628, "median": 0.1564, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.01877, "median": -0.01561, "ci": null, "n": 92, "wins": 73, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.006191, "median": -0.007323, "ci": null, "n": 92, "wins": 63, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "lat_mae_bl", "value": 0.1484, "median": 0.133, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.02381, "median": -0.02146, "ci": null, "n": 92, "wins": 80, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.01007, "median": -0.01062, "ci": null, "n": 92, "wins": 71, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "lat_corr", "value": 0.8089, "median": 0.9117, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.06044, "median": 0.02504, "ci": null, "n": 92, "wins": 79, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.009206, "median": -0.006426, "ci": null, "n": 92, "wins": 27, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.7875, "median": 0.8175, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.02121, "median": -0.007247, "ci": null, "n": 92, "wins": 50, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.007271, "median": -0.0006248, "ci": null, "n": 92, "wins": 46, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "value": 1.216, "median": 0.9481, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.401, "median": -1.307, "ci": null, "n": 92, "wins": 91, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.2201, "median": 0.09504, "ci": null, "n": 92, "wins": 25, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "value": 0.6652, "median": 0.6, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.004891, "median": 0.0, "ci": null, "n": 92, "wins": 32, "ties": 17, "clusters": 3, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.03681, "median": 0.0, "ci": null, "n": 91, "wins": 11, "ties": 35, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lon_mae", "value": 0.3573, "median": 0.3704, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.01871, "median": -0.01868, "ci": null, "n": 92, "wins": 59, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.03693, "median": 0.03017, "ci": null, "n": 92, "wins": 15, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.2553, "median": 0.2378, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.02157, "median": 0.03745, "ci": null, "n": 92, "wins": 55, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.05634, "median": -0.04196, "ci": null, "n": 92, "wins": 32, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lon_brake_mae", "value": 0.4537, "median": 0.4651, "ci": null, "n": 65, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.03544, "median": -0.04038, "ci": null, "n": 65, "wins": 43, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.02666, "median": -0.02422, "ci": null, "n": 65, "wins": 44, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "lon_rate_rms", "value": 5.065, "median": 5.093, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.268, "median": -1.538, "ci": null, "n": 92, "wins": 68, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 2.514, "median": 2.575, "ci": null, "n": 92, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 2.125, "median": 1.978, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1094, "median": -0.2267, "ci": null, "n": 92, "wins": 63, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1205, "median": 0.05814, "ci": null, "n": 92, "wins": 36, "ties": 0, "clusters": 3, "cluster_wins": 1 } } } ] }, "v14-8mm": { "n": 8, "clusters": 2, "metrics": [ { "key": "lat_mae", "value": 0.1528, "median": 0.1562, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.01142, "median": 0.01173, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.01455, "median": -0.01226, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_mae_bl", "value": 0.1416, "median": 0.1404, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.004464, "median": 0.004159, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.01621, "median": -0.01408, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_corr", "value": 0.8573, "median": 0.9252, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.01066, "median": -0.0001918, "ci": null, "n": 8, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.0007968, "median": -0.003292, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lat_gain", "value": 0.8899, "median": 0.8782, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.04456, "median": 0.03097, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.02278, "median": -0.03079, "ci": null, "n": 8, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_rate_rms", "value": 0.5633, "median": 0.4946, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.5922, "median": -0.5059, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.01697, "median": -0.02519, "ci": null, "n": 8, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.5625, "median": 0.5, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1125, "median": 0.1, "ci": null, "n": 8, "wins": 0, "ties": 3, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -2.082e-17, "median": 0.0, "ci": null, "n": 8, "wins": 2, "ties": 4, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_mae", "value": 0.3494, "median": 0.2838, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.04007, "median": 0.03585, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.03567, "median": 0.02767, "ci": null, "n": 8, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.6909, "median": 0.7232, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.06377, "median": -0.03522, "ci": null, "n": 8, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.0613, "median": -0.05105, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_brake_mae", "value": 0.6475, "median": 0.7275, "ci": null, "n": 6, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.08986, "median": 0.04654, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.03432, "median": 0.005476, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_rate_rms", "value": 4.594, "median": 4.453, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.06123, "median": 0.1742, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 1.786, "median": 2.022, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 1.095, "median": 1.065, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1622, "median": 0.1656, "ci": null, "n": 8, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.0631, "median": -0.08635, "ci": null, "n": 8, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 2 } } } ] } } }, "truck_closed_loop": { "status": "ok", "unit_of_evidence": "window", "n": 98, "clusters": 13, "condition": "Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. The simulation starts from the truck's state (speed, yaw rate, steering, gear) in its CAN recording of the 2 s before the window; one window, whose 2 s before are not all valid CAN, starts steady at its recorded speed. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 98 windows (59 day, 39 night).", "baselines": [ "lebowski_877m", "f78ed37d_12864" ], "metrics": [ { "key": "xtrack_2s", "label": "Distance from the recorded path at 2 s", "unit": "m", "better": "lower", "value": 0.1326, "median": 0.1013, "ci": [ 0.1068, 0.154 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.03695, "median": -0.03562, "ci": [ -0.05566, -0.02092 ], "n": 98, "wins": 63, "ties": 0, "clusters": 13, "cluster_wins": 12 }, "f78ed37d_12864": { "delta": -0.0174, "median": -0.008779, "ci": [ -0.02868, -0.006913 ], "n": 98, "wins": 55, "ties": 0, "clusters": 13, "cluster_wins": 10 } } }, { "key": "xtrack_4s", "label": "Distance from the recorded path at 4 s", "unit": "m", "better": "lower", "value": 0.4958, "median": 0.3189, "ci": [ 0.3673, 0.638 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.02421, "median": -0.008212, "ci": [ -0.207, 0.1345 ], "n": 98, "wins": 51, "ties": 0, "clusters": 13, "cluster_wins": 8 }, "f78ed37d_12864": { "delta": -0.002332, "median": -0.0374, "ci": [ -0.08842, 0.1187 ], "n": 98, "wins": 56, "ties": 0, "clusters": 13, "cluster_wins": 10 } } }, { "key": "xtrack_7s", "label": "Distance from the recorded path at 7.2 s", "unit": "m", "better": "lower", "value": 2.037, "median": 0.7023, "ci": [ 1.383, 3.011 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.05802, "median": -0.0005321, "ci": [ -0.8235, 1.106 ], "n": 98, "wins": 49, "ties": 0, "clusters": 13, "cluster_wins": 6 }, "f78ed37d_12864": { "delta": 0.1669, "median": -0.05482, "ci": [ -0.1641, 0.6802 ], "n": 98, "wins": 58, "ties": 0, "clusters": 13, "cluster_wins": 8 } } }, { "key": "fail_1m", "label": "Share of episodes ever more than 1 m off the path", "unit": "", "better": "lower", "value": 0.398, "median": 0.0, "ci": [ 0.3125, 0.4884 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.03061, "median": 0.0, "ci": [ -0.06881, 0.01765 ], "n": 98, "wins": 15, "ties": 73, "clusters": 13, "cluster_wins": 8 }, "f78ed37d_12864": { "delta": -0.04082, "median": 0.0, "ci": [ -0.07747, -0.009257 ], "n": 98, "wins": 9, "ties": 84, "clusters": 13, "cluster_wins": 6 } } }, { "key": "lat_jerk", "label": "Requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.2658, "median": 0.1828, "ci": [ 0.2182, 0.3182 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.2762, "median": -0.1792, "ci": [ -0.3618, -0.1982 ], "n": 98, "wins": 89, "ties": 0, "clusters": 13, "cluster_wins": 12 }, "f78ed37d_12864": { "delta": 0.0362, "median": 0.002695, "ci": [ -0.003333, 0.07374 ], "n": 98, "wins": 46, "ties": 0, "clusters": 13, "cluster_wins": 4 } } }, { "key": "long_jerk", "label": "Requested longitudinal jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3758, "median": 0.2961, "ci": [ 0.3191, 0.4394 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.01053, "median": -0.04583, "ci": [ -0.08063, 0.06575 ], "n": 98, "wins": 66, "ties": 0, "clusters": 13, "cluster_wins": 7 }, "f78ed37d_12864": { "delta": 0.1234, "median": 0.04032, "ci": [ 0.07467, 0.1782 ], "n": 98, "wins": 37, "ties": 0, "clusters": 13, "cluster_wins": 1 } } }, { "key": "speed_err", "label": "Speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.26, "median": 1.007, "ci": [ 1.018, 1.509 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.1769, "median": 0.1411, "ci": [ 0.02857, 0.3269 ], "n": 98, "wins": 34, "ties": 0, "clusters": 13, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.1396, "median": 0.02316, "ci": [ 0.06591, 0.2064 ], "n": 98, "wins": 46, "ties": 0, "clusters": 13, "cluster_wins": 3 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 14, "clusters": 2, "metrics": [ { "key": "xtrack_2s", "value": 0.1235, "median": 0.08771, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02684, "median": -0.005324, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.01497, "median": -0.005104, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "xtrack_4s", "value": 0.5552, "median": 0.4282, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1041, "median": 0.04203, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.0575, "median": -0.01344, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_7s", "value": 2.164, "median": 1.512, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2592, "median": 0.1621, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.1887, "median": -0.11, "ci": null, "n": 14, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "fail_1m", "value": 0.6786, "median": 1.0, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.0, "median": 0.0, "ci": null, "n": 14, "wins": 3, "ties": 7, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.03571, "median": 0.0, "ci": null, "n": 14, "wins": 2, "ties": 11, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lat_jerk", "value": 0.4544, "median": 0.2177, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1164, "median": -0.1187, "ci": null, "n": 14, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1504, "median": 0.02928, "ci": null, "n": 14, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "long_jerk", "value": 0.3295, "median": 0.2811, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1652, "median": -0.1098, "ci": null, "n": 14, "wins": 13, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.07555, "median": 0.09254, "ci": null, "n": 14, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 1.934, "median": 1.958, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1702, "median": 0.2248, "ci": null, "n": 14, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.2814, "median": 0.192, "ci": null, "n": 14, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 } } } ] }, "stock-dual": { "n": 43, "clusters": 5, "metrics": [ { "key": "xtrack_2s", "value": 0.1385, "median": 0.1122, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.02959, "median": -0.03836, "ci": null, "n": 43, "wins": 30, "ties": 0, "clusters": 5, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": -0.02684, "median": -0.03488, "ci": null, "n": 43, "wins": 26, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "xtrack_4s", "value": 0.4199, "median": 0.2689, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.03238, "median": -0.01809, "ci": null, "n": 43, "wins": 25, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.1137, "median": -0.09136, "ci": null, "n": 43, "wins": 30, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "xtrack_7s", "value": 1.78, "median": 0.5595, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.2388, "median": 0.003853, "ci": null, "n": 43, "wins": 20, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.03803, "median": -0.08151, "ci": null, "n": 43, "wins": 28, "ties": 0, "clusters": 5, "cluster_wins": 3 } } }, { "key": "fail_1m", "value": 0.3256, "median": 0.0, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.01163, "median": 0.0, "ci": null, "n": 43, "wins": 5, "ties": 34, "clusters": 5, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.0814, "median": 0.0, "ci": null, "n": 43, "wins": 5, "ties": 37, "clusters": 5, "cluster_wins": 4 } } }, { "key": "lat_jerk", "value": 0.262, "median": 0.1724, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.2907, "median": -0.1982, "ci": null, "n": 43, "wins": 39, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": 0.03978, "median": 0.01295, "ci": null, "n": 43, "wins": 17, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "long_jerk", "value": 0.416, "median": 0.3115, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.04177, "median": -0.03324, "ci": null, "n": 43, "wins": 28, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.1519, "median": 0.01613, "ci": null, "n": 43, "wins": 18, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "speed_err", "value": 1.236, "median": 0.9889, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.1528, "median": 0.05748, "ci": null, "n": 43, "wins": 18, "ties": 0, "clusters": 5, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1253, "median": -0.03477, "ci": null, "n": 43, "wins": 23, "ties": 0, "clusters": 5, "cluster_wins": 1 } } } ] }, "stock-newcase": { "n": 4, "clusters": 1, "metrics": [ { "key": "xtrack_2s", "value": 0.03803, "median": 0.02548, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.08684, "median": -0.07424, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.06872, "median": -0.06934, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "xtrack_4s", "value": 0.2054, "median": 0.1653, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.03522, "median": 0.05566, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.09374, "median": -0.06978, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "xtrack_7s", "value": 1.236, "median": 1.027, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.3994, "median": -0.1004, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.08392, "median": 0.000266, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "fail_1m", "value": 0.5, "median": 0.5, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.125, "median": 0.0, "ci": null, "n": 4, "wins": 1, "ties": 3, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.125, "median": 0.0, "ci": null, "n": 4, "wins": 1, "ties": 2, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_jerk", "value": 0.258, "median": 0.194, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.4817, "median": -0.3209, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.02849, "median": 0.03215, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "long_jerk", "value": 0.5034, "median": 0.4118, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.1142, "median": 0.09075, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.3249, "median": 0.2237, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 0.947, "median": 0.9194, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.2682, "median": 0.3887, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.1698, "median": -0.001116, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ] }, "v13-8mm": { "n": 18, "clusters": 3, "metrics": [ { "key": "xtrack_2s", "value": 0.1542, "median": 0.09663, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.0631, "median": -0.04748, "ci": null, "n": 18, "wins": 13, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.001908, "median": -0.003298, "ci": null, "n": 18, "wins": 9, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "xtrack_4s", "value": 0.4143, "median": 0.2173, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.3878, "median": -0.04362, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.01832, "median": 0.006191, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "xtrack_7s", "value": 1.175, "median": 0.8455, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.45, "median": -0.07089, "ci": null, "n": 18, "wins": 10, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.07103, "median": -0.05068, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "fail_1m", "value": 0.3333, "median": 0.0, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.05556, "median": 0.0, "ci": null, "n": 18, "wins": 3, "ties": 14, "clusters": 3, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.02778, "median": 0.0, "ci": null, "n": 18, "wins": 0, "ties": 17, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.1951, "median": 0.1431, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1843, "median": -0.1434, "ci": null, "n": 18, "wins": 17, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.0123, "median": -0.005673, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "long_jerk", "value": 0.3084, "median": 0.189, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.003805, "median": -0.02068, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1316, "median": 0.03037, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 0.7913, "median": 0.5547, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1533, "median": -0.004014, "ci": null, "n": 18, "wins": 9, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": 0.06588, "median": 0.05043, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 1 } } } ] }, "v14-8mm": { "n": 19, "clusters": 2, "metrics": [ { "key": "xtrack_2s", "value": 0.1251, "median": 0.09598, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02579, "median": 0.01097, "ci": null, "n": 19, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.001715, "median": 0.01233, "ci": null, "n": 19, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_4s", "value": 0.7623, "median": 0.4028, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2465, "median": 0.03463, "ci": null, "n": 19, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.2401, "median": -0.00594, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_7s", "value": 3.511, "median": 0.6849, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 2.106, "median": -0.04083, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 1.171, "median": 0.02014, "ci": null, "n": 19, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "fail_1m", "value": 0.3947, "median": 0.0, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.05263, "median": 0.0, "ci": null, "n": 19, "wins": 3, "ties": 15, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 19, "wins": 1, "ties": 17, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.2037, "median": 0.1963, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.4049, "median": -0.2679, "ci": null, "n": 19, "wins": 18, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.03184, "median": -0.01804, "ci": null, "n": 19, "wins": 12, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "long_jerk", "value": 0.3557, "median": 0.3326, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.05475, "median": -0.03618, "ci": null, "n": 19, "wins": 12, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.04392, "median": 0.001009, "ci": null, "n": 19, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 1.325, "median": 0.9894, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.5303, "median": 0.4375, "ci": null, "n": 19, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.1309, "median": -0.06803, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 } } } ] } }, "simulator": "g55-2026-10-31" }, "truck_recovery": { "status": "ok", "unit_of_evidence": "window", "n": 100, "clusters": 14, "condition": "Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 98 windows (59 day, 39 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 98 windows (59 day, 39 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).", "baselines": [ "lebowski_877m", "f78ed37d_12864" ], "metrics": [ { "key": "heading_xtrack_2s", "label": "2° off heading: distance from the path at 2 s", "unit": "m", "better": "lower", "value": 0.5179, "median": 0.4425, "ci": [ 0.4232, 0.6336 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.06499, "median": 0.05592, "ci": [ 0.02548, 0.1086 ], "n": 98, "wins": 37, "ties": 0, "clusters": 13, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.008271, "median": 0.01089, "ci": [ -0.005523, 0.02315 ], "n": 98, "wins": 36, "ties": 0, "clusters": 13, "cluster_wins": 7 } } }, { "key": "heading_xtrack_4s", "label": "2° off heading: distance from the path at 4 s", "unit": "m", "better": "lower", "value": 0.7409, "median": 0.4902, "ci": [ 0.5559, 0.9328 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.03841, "median": 0.0208, "ci": [ -0.1129, 0.168 ], "n": 98, "wins": 43, "ties": 0, "clusters": 13, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.0182, "median": -0.0151, "ci": [ -0.1407, 0.08444 ], "n": 98, "wins": 53, "ties": 0, "clusters": 13, "cluster_wins": 7 } } }, { "key": "heading_lat_jerk", "label": "2° off heading: requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3656, "median": 0.3096, "ci": [ 0.3254, 0.4035 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.5213, "median": -0.4706, "ci": [ -0.602, -0.4312 ], "n": 98, "wins": 96, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "f78ed37d_12864": { "delta": 0.008916, "median": -0.01946, "ci": [ -0.03491, 0.05606 ], "n": 98, "wins": 71, "ties": 0, "clusters": 13, "cluster_wins": 8 } } }, { "key": "offcentre_xtrack_2s", "label": "0.5 m off centre: distance from the path at 2 s", "unit": "m", "better": "lower", "value": 0.287, "median": 0.2444, "ci": [ 0.263, 0.3105 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.02002, "median": 0.03696, "ci": [ -0.00233, 0.04376 ], "n": 98, "wins": 36, "ties": 0, "clusters": 13, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": 0.004493, "median": 0.004787, "ci": [ -0.01084, 0.02186 ], "n": 98, "wins": 41, "ties": 0, "clusters": 13, "cluster_wins": 6 } } }, { "key": "offcentre_xtrack_4s", "label": "0.5 m off centre: distance from the path at 4 s", "unit": "m", "better": "lower", "value": 0.6337, "median": 0.3706, "ci": [ 0.5171, 0.752 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.02913, "median": -0.05153, "ci": [ -0.1256, 0.06501 ], "n": 98, "wins": 61, "ties": 0, "clusters": 13, "cluster_wins": 9 }, "f78ed37d_12864": { "delta": -0.06843, "median": -0.05895, "ci": [ -0.1647, 0.03102 ], "n": 98, "wins": 61, "ties": 0, "clusters": 13, "cluster_wins": 8 } } }, { "key": "offcentre_lat_jerk", "label": "0.5 m off centre: requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.533, "median": 0.4181, "ci": [ 0.4685, 0.5942 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.5774, "median": -0.5876, "ci": [ -0.6497, -0.5042 ], "n": 98, "wins": 95, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "f78ed37d_12864": { "delta": 0.0797, "median": -0.01487, "ci": [ 0.0263, 0.1297 ], "n": 98, "wins": 58, "ties": 0, "clusters": 13, "cluster_wins": 5 } } }, { "key": "brake_ahead_4s", "label": "Braking lead: distance ahead of the recorded car at 4 s", "unit": "m", "better": "lower", "value": -0.9686, "median": -0.9686, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.3689, "median": 0.3689, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.9376, "median": -0.9376, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "brake_gap_min", "label": "Braking lead: smallest gap to the lead (derived)", "unit": "m", "better": "higher", "value": 27.81, "median": 27.81, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.3494, "median": 0.3494, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 5.525, "median": 5.525, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "brake_speed_err", "label": "Braking lead: speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.066, "median": 1.066, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.1372, "median": -0.1372, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1605, "median": 0.1605, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 14, "clusters": 2, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.4072, "median": 0.3803, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.01537, "median": 0.008627, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.05187, "median": 0.05356, "ci": null, "n": 14, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "heading_xtrack_4s", "value": 0.7026, "median": 0.5763, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1496, "median": -0.007462, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.1465, "median": 0.08951, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "heading_lat_jerk", "value": 0.4374, "median": 0.3381, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3306, "median": -0.2849, "ci": null, "n": 14, "wins": 13, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.05483, "median": -0.01352, "ci": null, "n": 14, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2829, "median": 0.2702, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.02002, "median": 0.02186, "ci": null, "n": 14, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.03482, "median": 0.05509, "ci": null, "n": 14, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.7039, "median": 0.5117, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1141, "median": 0.09518, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1298, "median": 0.05644, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_lat_jerk", "value": 0.6922, "median": 0.5959, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3695, "median": -0.5903, "ci": null, "n": 14, "wins": 12, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": 0.1648, "median": 0.0503, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 } } } ] }, "stock-dual": { "n": 45, "clusters": 6, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.4421, "median": 0.389, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.02652, "median": -0.01158, "ci": null, "n": 43, "wins": 22, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.01194, "median": -0.007672, "ci": null, "n": 43, "wins": 23, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "heading_xtrack_4s", "value": 0.6151, "median": 0.4252, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.02112, "median": -0.01253, "ci": null, "n": 43, "wins": 22, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.1423, "median": -0.09241, "ci": null, "n": 43, "wins": 30, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "heading_lat_jerk", "value": 0.3807, "median": 0.2899, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.5157, "median": -0.4757, "ci": null, "n": 43, "wins": 42, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": 0.0468, "median": -0.007905, "ci": null, "n": 43, "wins": 26, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2895, "median": 0.2443, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.009873, "median": -0.001892, "ci": null, "n": 43, "wins": 22, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.02321, "median": -0.0115, "ci": null, "n": 43, "wins": 28, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "offcentre_xtrack_4s", "value": 0.6103, "median": 0.3702, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.06012, "median": -0.05857, "ci": null, "n": 43, "wins": 29, "ties": 0, "clusters": 5, "cluster_wins": 4 }, "f78ed37d_12864": { "delta": -0.2016, "median": -0.1671, "ci": null, "n": 43, "wins": 29, "ties": 0, "clusters": 5, "cluster_wins": 4 } } }, { "key": "offcentre_lat_jerk", "value": 0.5848, "median": 0.4816, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.5919, "median": -0.5925, "ci": null, "n": 43, "wins": 42, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "f78ed37d_12864": { "delta": 0.1358, "median": 0.02405, "ci": null, "n": 43, "wins": 18, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "brake_ahead_4s", "value": -0.9686, "median": -0.9686, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.3689, "median": 0.3689, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.9376, "median": -0.9376, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "brake_gap_min", "value": 27.81, "median": 27.81, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.3494, "median": 0.3494, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 5.525, "median": 5.525, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "brake_speed_err", "value": 1.066, "median": 1.066, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.1372, "median": -0.1372, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1605, "median": 0.1605, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ] }, "stock-newcase": { "n": 4, "clusters": 1, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.3891, "median": 0.4161, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.06143, "median": -0.0827, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.001208, "median": 0.00163, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "heading_xtrack_4s", "value": 0.2457, "median": 0.1942, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.02607, "median": 0.09184, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": -0.2568, "median": -0.16, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "heading_lat_jerk", "value": 0.2662, "median": 0.2144, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.706, "median": -0.7133, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.07051, "median": -0.0798, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_2s", "value": 0.3576, "median": 0.2513, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.0578, "median": 0.06833, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.07559, "median": 0.02246, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.5554, "median": 0.3707, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.1723, "median": -0.1847, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": -0.2813, "median": -0.2512, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.6422, "median": 0.4319, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.7552, "median": -0.668, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.1268, "median": -0.09174, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ] }, "v13-8mm": { "n": 18, "clusters": 3, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.8973, "median": 0.8716, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.2043, "median": 0.2381, "ci": null, "n": 18, "wins": 2, "ties": 0, "clusters": 3, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.003012, "median": -0.0007176, "ci": null, "n": 18, "wins": 10, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "heading_xtrack_4s", "value": 1.128, "median": 0.8199, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1038, "median": 0.09939, "ci": null, "n": 18, "wins": 8, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.04204, "median": 0.02957, "ci": null, "n": 18, "wins": 4, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "heading_lat_jerk", "value": 0.3861, "median": 0.3836, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.4869, "median": -0.4291, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.02711, "median": -0.02551, "ci": null, "n": 18, "wins": 14, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2999, "median": 0.245, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.04186, "median": 0.06722, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.01423, "median": 0.004787, "ci": null, "n": 18, "wins": 5, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_4s", "value": 0.4983, "median": 0.2244, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1947, "median": -0.08205, "ci": null, "n": 18, "wins": 12, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.01018, "median": -0.008579, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "offcentre_lat_jerk", "value": 0.3812, "median": 0.3345, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.5006, "median": -0.5115, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "f78ed37d_12864": { "delta": -0.0163, "median": -0.03602, "ci": null, "n": 18, "wins": 14, "ties": 0, "clusters": 3, "cluster_wins": 2 } } } ] }, "v14-8mm": { "n": 19, "clusters": 2, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.4388, "median": 0.4422, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.08327, "median": 0.06992, "ci": null, "n": 19, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.02836, "median": 0.03258, "ci": null, "n": 19, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "heading_xtrack_4s", "value": 0.792, "median": 0.4397, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2285, "median": 0.08262, "ci": null, "n": 19, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.1345, "median": 0.01045, "ci": null, "n": 19, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "heading_lat_jerk", "value": 0.2802, "median": 0.2602, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.6682, "median": -0.5478, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.0598, "median": -0.03876, "ci": null, "n": 19, "wins": 18, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2575, "median": 0.2425, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.05904, "median": 0.08326, "ci": null, "n": 19, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "f78ed37d_12864": { "delta": 0.02064, "median": 0.03491, "ci": null, "n": 19, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.7799, "median": 0.371, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1225, "median": -0.01315, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "f78ed37d_12864": { "delta": 0.07646, "median": -0.01723, "ci": null, "n": 19, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.4196, "median": 0.4185, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.733, "median": -0.6564, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "f78ed37d_12864": { "delta": -0.02889, "median": -0.05227, "ci": null, "n": 19, "wins": 16, "ties": 0, "clusters": 2, "cluster_wins": 2 } } } ] } }, "simulator": "g55-2026-10-31" } }, "stills": [], "limits": [ "Open-loop scores compare a model's action with what the recording driver or car did; a model that drives differently but well is scored as wrong. comma's RL residual makes the lateral action smoother, and earlier than its base's by 0.10 to 0.26 s (measured on the G55 and on 16 of comma's cars); both score as error here. Measured on the 2026-10-27 deep dive: open-loop accuracy did not predict closed-loop driving.", "Closed-loop numbers come from a generative world model, not a car. In a separate test on the G55's windows, worldmodel-4B followed an added yaw command through 2 s but showed 0 to 20% of an added lateral offset 3 to 4 s later; past about 4 s the recorded anchor frames also pull every model back toward the recorded path. The front page reports 2 s.", "Intervals are 95% percentile bootstraps that resample whole drives (the drive a public segment was cut from, the recorded drive a truck window comes from), so segments of one drive count once. Below 10 drives there is no interval: the braking-lead test comes from one drive and reports means and win counts only.", "Paired deltas are model minus baseline on the units both have (for lateral gain, the change in its distance from 1); wins count units where the model is better in the metric's direction.", "The lead of the lateral request is the shift, searched from 0 to 1.5 s, at which the request best correlates with the car's later lateral motion; a larger value means the request comes earlier. A value at either edge is censored and left out. Plan speed error at 3 s reads the plan head, which does not separate models built on one frozen base: it is identical on every segment for the four f22e06e5 models, and for Cinque v2 and v3.", "Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.", "Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre. The pooled number mixes camera setups, so read the per-setup numbers too. On the V14 setup the wide frame can be up to 90 ms from the simulated one (its camera recorded 7-9 fps).", "Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test has 2 windows from one drive, whose recorded calibration was carried over from an earlier drive." ] }