{ "schema": 1, "model": { "id": "8d736683_12864", "name": "Mountain Dew", "sha256": "87fe86ca0813c099efddf9173ba503be2e40ea4c5362c36812a44141f4cb07d4", "source": { "kind": "release", "ref": "v0.11.2", "url": "https://github.com/commaai/openpilot/releases/tag/v0.11.2", "date": "2026-10-23" }, "first_public": "2026-10-10", "on_master": "2026-10-10", "params_m": 382.9, "gmac": 50.1, "family": "residual", "notes": "Shipped in openpilot 0.11.2 (master 2026-10-10, PR #39047). Scored from the Hugging Face upload of 2026-10-14, whose tensors match the release's compiled model byte for byte. Vision base f22e06e5 with an RL residual head." }, "scored": { "date": "2026-10-31T23:59:14Z", "suite": "geval-1", "code": "d8eb40eeba87" }, "baselines": [ "lebowski_877m", "870a4823_12864" ], "sections": { "public_open_loop": { "status": "ok", "unit_of_evidence": "segment", "n": 39, "clusters": 23, "condition": "Measured. comma's public two-camera segments (openpilot CI data), native intrinsics and the device's own calibration, both cameras; every model on the same decoded frames. Truth is what the recording car did next: carState speed and acceleration, yaw rate from the device's cameraOdometry. Segments where the car never moves have no data. 39 of the 45 listed segments have a score, from 23 drives and 20 cars.", "baselines": [ "lebowski_877m", "870a4823_12864" ], "metrics": [ { "key": "lat_mae", "label": "Lateral error", "unit": "m/s²", "better": "lower", "value": 0.1435, "median": 0.1273, "ci": [ 0.113, 0.1758 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.03476, "median": 0.02054, "ci": [ 0.02333, 0.04654 ], "n": 39, "wins": 3, "ties": 0, "clusters": 23, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.009801, "median": 0.00607, "ci": [ 0.005152, 0.01479 ], "n": 39, "wins": 7, "ties": 0, "clusters": 23, "cluster_wins": 3 } } }, { "key": "lat_mae_bl", "label": "Lateral error at the model's own best lead", "unit": "m/s²", "better": "lower", "value": 0.1321, "median": 0.1137, "ci": [ 0.1026, 0.1636 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.03073, "median": 0.01596, "ci": [ 0.01767, 0.04412 ], "n": 39, "wins": 7, "ties": 0, "clusters": 23, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.006167, "median": 0.002033, "ci": [ 0.001465, 0.01113 ], "n": 39, "wins": 13, "ties": 0, "clusters": 23, "cluster_wins": 6 } } }, { "key": "lat_corr", "label": "Lateral correlation with the car's later motion", "unit": "", "better": "higher", "value": 0.8207, "median": 0.9291, "ci": [ 0.7258, 0.9013 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.03696, "median": -0.02256, "ci": [ -0.08437, -0.003298 ], "n": 39, "wins": 7, "ties": 0, "clusters": 23, "cluster_wins": 4 }, "870a4823_12864": { "delta": -0.01994, "median": -0.008982, "ci": [ -0.04023, -0.003784 ], "n": 39, "wins": 4, "ties": 0, "clusters": 23, "cluster_wins": 2 } } }, { "key": "lat_gain", "label": "Lateral gain (1 = same size as the car's motion)", "unit": "", "better": "near1", "value": 0.7322, "median": 0.7394, "ci": [ 0.6417, 0.8258 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.08866, "median": 0.07854, "ci": [ 0.02561, 0.1525 ], "n": 39, "wins": 11, "ties": 0, "clusters": 23, "cluster_wins": 7 }, "870a4823_12864": { "delta": 0.02754, "median": 0.01426, "ci": [ 0.007847, 0.0478 ], "n": 39, "wins": 13, "ties": 0, "clusters": 23, "cluster_wins": 8 } } }, { "key": "lat_rate_rms", "label": "Lateral action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.474, "median": 0.4218, "ci": [ 0.3745, 0.5946 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.1899, "median": -0.216, "ci": [ -0.2388, -0.1342 ], "n": 39, "wins": 35, "ties": 0, "clusters": 23, "cluster_wins": 22 }, "870a4823_12864": { "delta": 0.03575, "median": 0.02577, "ci": [ 0.02575, 0.05288 ], "n": 39, "wins": 2, "ties": 0, "clusters": 23, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "label": "Lead of the lateral request over the car's motion", "unit": "s", "better": "lower", "value": 0.6153, "median": 0.5, "ci": [ 0.5042, 0.736 ], "n": 36, "clusters": 22, "paired": { "lebowski_877m": { "delta": 0.2257, "median": 0.15, "ci": [ 0.1382, 0.3281 ], "n": 35, "wins": 1, "ties": 3, "clusters": 22, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.1014, "median": 0.075, "ci": [ 0.07105, 0.1458 ], "n": 36, "wins": 0, "ties": 2, "clusters": 22, "cluster_wins": 0 } } }, { "key": "lon_mae", "label": "Longitudinal error", "unit": "m/s²", "better": "lower", "value": 0.3131, "median": 0.2945, "ci": [ 0.2735, 0.3577 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.004976, "median": 0.0009126, "ci": [ -0.01115, 0.02325 ], "n": 39, "wins": 16, "ties": 0, "clusters": 23, "cluster_wins": 9 }, "870a4823_12864": { "delta": 0.003985, "median": 0.005507, "ci": [ -0.001318, 0.009218 ], "n": 39, "wins": 14, "ties": 0, "clusters": 23, "cluster_wins": 9 } } }, { "key": "lon_corr", "label": "Longitudinal correlation", "unit": "", "better": "higher", "value": 0.724, "median": 0.8485, "ci": [ 0.6177, 0.8172 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.006207, "median": 0.005531, "ci": [ -0.02059, 0.03093 ], "n": 39, "wins": 20, "ties": 0, "clusters": 23, "cluster_wins": 11 }, "870a4823_12864": { "delta": -0.007943, "median": -0.0028, "ci": [ -0.02001, 0.001518 ], "n": 39, "wins": 14, "ties": 0, "clusters": 23, "cluster_wins": 8 } } }, { "key": "lon_brake_mae", "label": "Longitudinal error while braking", "unit": "m/s²", "better": "lower", "value": 0.4275, "median": 0.3716, "ci": [ 0.3667, 0.5 ], "n": 30, "clusters": 21, "paired": { "lebowski_877m": { "delta": -0.006826, "median": 0.004398, "ci": [ -0.0304, 0.02003 ], "n": 30, "wins": 13, "ties": 0, "clusters": 21, "cluster_wins": 11 }, "870a4823_12864": { "delta": 0.01582, "median": 0.01423, "ci": [ 0.006791, 0.02434 ], "n": 30, "wins": 10, "ties": 0, "clusters": 21, "cluster_wins": 6 } } }, { "key": "lon_rate_rms", "label": "Longitudinal action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.897, "median": 0.7682, "ci": [ 0.7812, 1.038 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.436, "median": -0.3464, "ci": [ -0.5781, -0.3121 ], "n": 39, "wins": 36, "ties": 0, "clusters": 23, "cluster_wins": 22 }, "870a4823_12864": { "delta": 0.003298, "median": -0.002988, "ci": [ -0.01381, 0.02031 ], "n": 39, "wins": 20, "ties": 0, "clusters": 23, "cluster_wins": 10 } } }, { "key": "v3_mae", "label": "Plan speed error at 3 s", "unit": "m/s", "better": "lower", "value": 0.7602, "median": 0.7121, "ci": [ 0.6567, 0.888 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.04442, "median": 0.04624, "ci": [ 0.009032, 0.07746 ], "n": 39, "wins": 12, "ties": 0, "clusters": 23, "cluster_wins": 8 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": [ 0.0, 0.0 ], "n": 39, "wins": 0, "ties": 39, "clusters": 23, "cluster_wins": 0 } } } ], "per_unit_file": "per_unit/8d736683_12864/public_open_loop.csv" }, "public_closed_loop": { "status": "ok", "unit_of_evidence": "segment", "n": 28, "clusters": 19, "condition": "Measured. comma's worldmodel-4B generates each next frame at 5 Hz from 9 history frames, 5 recorded anchor frames and the car's pose; the model drives comma's ideal car (the requested curvature and acceleration happen at once) for 7.2 s from the recorded start. Distance is to the recording car's own path. One window per segment, nearest the middle, where the car holds at least 5 m/s for the window and the 4.8 s before it; segments with no such stretch are left out. Seeds 0 and 1, averaged per segment. 28 of the 45 listed segments have a score, from 19 drives and 17 cars.", "baselines": [ "lebowski_877m", "870a4823_12864" ], "metrics": [ { "key": "xtrack_2s", "label": "Distance from the recorded path at 2 s", "unit": "m", "better": "lower", "value": 0.09638, "median": 0.05582, "ci": [ 0.04802, 0.1817 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.0513, "median": -0.01454, "ci": [ -0.102, -0.01419 ], "n": 28, "wins": 19, "ties": 0, "clusters": 19, "cluster_wins": 16 }, "870a4823_12864": { "delta": -0.001537, "median": 0.004524, "ci": [ -0.02222, 0.01911 ], "n": 28, "wins": 13, "ties": 0, "clusters": 19, "cluster_wins": 8 } } }, { "key": "xtrack_4s", "label": "Distance from the recorded path at 4 s", "unit": "m", "better": "lower", "value": 0.221, "median": 0.1087, "ci": [ 0.1034, 0.4074 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.05729, "median": 0.0114, "ci": [ -0.1255, -0.00137 ], "n": 28, "wins": 11, "ties": 0, "clusters": 19, "cluster_wins": 9 }, "870a4823_12864": { "delta": 0.04791, "median": 0.007159, "ci": [ -0.02675, 0.1584 ], "n": 28, "wins": 12, "ties": 0, "clusters": 19, "cluster_wins": 7 } } }, { "key": "xtrack_7s", "label": "Distance from the recorded path at 7.2 s", "unit": "m", "better": "lower", "value": 1.025, "median": 0.3086, "ci": [ 0.3446, 1.901 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.01929, "median": -0.02165, "ci": [ -0.5859, 0.5312 ], "n": 28, "wins": 15, "ties": 0, "clusters": 19, "cluster_wins": 10 }, "870a4823_12864": { "delta": 0.08425, "median": -0.0137, "ci": [ -0.0814, 0.2853 ], "n": 28, "wins": 16, "ties": 0, "clusters": 19, "cluster_wins": 12 } } }, { "key": "fail_1m", "label": "Share of episodes ever more than 1 m off the path", "unit": "", "better": "lower", "value": 0.1786, "median": 0.0, "ci": [ 0.03846, 0.3333 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.05357, "median": 0.0, "ci": [ -0.1522, 0.0 ], "n": 28, "wins": 2, "ties": 26, "clusters": 19, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.03571, "median": 0.0, "ci": [ -0.1, 0.0 ], "n": 28, "wins": 2, "ties": 26, "clusters": 19, "cluster_wins": 2 } } }, { "key": "lat_jerk", "label": "Requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.2021, "median": 0.1191, "ci": [ 0.1355, 0.2723 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.1235, "median": -0.07406, "ci": [ -0.1793, -0.06923 ], "n": 28, "wins": 24, "ties": 0, "clusters": 19, "cluster_wins": 16 }, "870a4823_12864": { "delta": 0.02025, "median": 0.01056, "ci": [ -0.005336, 0.04187 ], "n": 28, "wins": 6, "ties": 0, "clusters": 19, "cluster_wins": 2 } } }, { "key": "long_jerk", "label": "Requested longitudinal jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.341, "median": 0.289, "ci": [ 0.2615, 0.4334 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.005382, "median": -0.02769, "ci": [ -0.04512, 0.04191 ], "n": 28, "wins": 17, "ties": 0, "clusters": 19, "cluster_wins": 11 }, "870a4823_12864": { "delta": 0.02395, "median": 0.009417, "ci": [ 0.001739, 0.04831 ], "n": 28, "wins": 11, "ties": 0, "clusters": 19, "cluster_wins": 7 } } }, { "key": "speed_err", "label": "Speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.241, "median": 0.8609, "ci": [ 0.8733, 1.644 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": 0.14, "median": 0.1712, "ci": [ -0.09488, 0.3612 ], "n": 28, "wins": 8, "ties": 0, "clusters": 19, "cluster_wins": 7 }, "870a4823_12864": { "delta": 0.007365, "median": -0.002964, "ci": [ -0.08448, 0.07858 ], "n": 28, "wins": 15, "ties": 0, "clusters": 19, "cluster_wins": 10 } } } ], "per_unit_file": "per_unit/8d736683_12864/public_closed_loop.csv" }, "truck_open_loop": { "status": "ok", "unit_of_evidence": "window", "n": 119, "clusters": 12, "condition": "Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.", "baselines": [ "lebowski_877m", "870a4823_12864" ], "metrics": [ { "key": "lat_mae", "label": "Lateral error", "unit": "m/s²", "better": "lower", "value": 0.164, "median": 0.1537, "ci": [ 0.1344, 0.1751 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.008735, "median": -0.007474, "ci": [ -0.0145, 0.001726 ], "n": 119, "wins": 69, "ties": 0, "clusters": 12, "cluster_wins": 7 }, "870a4823_12864": { "delta": 0.006727, "median": 0.006625, "ci": [ 0.004309, 0.008145 ], "n": 119, "wins": 22, "ties": 0, "clusters": 12, "cluster_wins": 1 } } }, { "key": "lat_mae_bl", "label": "Lateral error at the model's own best lead", "unit": "m/s²", "better": "lower", "value": 0.1456, "median": 0.1299, "ci": [ 0.1201, 0.1547 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.0189, "median": -0.01574, "ci": [ -0.02269, -0.008663 ], "n": 119, "wins": 93, "ties": 0, "clusters": 12, "cluster_wins": 8 }, "870a4823_12864": { "delta": 0.001778, "median": 0.00151, "ci": [ -0.0004789, 0.0037 ], "n": 119, "wins": 50, "ties": 0, "clusters": 12, "cluster_wins": 5 } } }, { "key": "lat_corr", "label": "Lateral correlation with the car's later motion", "unit": "", "better": "higher", "value": 0.803, "median": 0.8997, "ci": [ 0.7599, 0.8288 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.03616, "median": 0.01449, "ci": [ -0.001582, 0.04606 ], "n": 119, "wins": 83, "ties": 0, "clusters": 12, "cluster_wins": 6 }, "870a4823_12864": { "delta": -0.01396, "median": -0.007845, "ci": [ -0.02107, -0.01167 ], "n": 119, "wins": 15, "ties": 0, "clusters": 12, "cluster_wins": 0 } } }, { "key": "lat_gain", "label": "Lateral gain (1 = same size as the car's motion)", "unit": "", "better": "near1", "value": 0.7822, "median": 0.8105, "ci": [ 0.7657, 0.8382 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.008747, "median": 0.01361, "ci": [ -0.007337, 0.04251 ], "n": 119, "wins": 49, "ties": 0, "clusters": 12, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.0234, "median": 0.01803, "ci": [ 0.0173, 0.03759 ], "n": 119, "wins": 33, "ties": 0, "clusters": 12, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "label": "Lateral action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 1.093, "median": 0.8181, "ci": [ 0.6358, 1.316 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -1.197, "median": -1.121, "ci": [ -1.334, -0.4889 ], "n": 119, "wins": 118, "ties": 0, "clusters": 12, "cluster_wins": 12 }, "870a4823_12864": { "delta": 0.00872, "median": 0.008049, "ci": [ -0.006323, 0.06323 ], "n": 119, "wins": 46, "ties": 0, "clusters": 12, "cluster_wins": 3 } } }, { "key": "lat_best_lag", "label": "Lead of the lateral request over the car's motion", "unit": "s", "better": "lower", "value": 0.7165, "median": 0.7, "ci": [ 0.6549, 0.7588 ], "n": 118, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.1, "median": 0.1, "ci": [ 0.03451, 0.19 ], "n": 118, "wins": 20, "ties": 16, "clusters": 12, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.07288, "median": 0.05, "ci": [ 0.06853, 0.09 ], "n": 118, "wins": 1, "ties": 28, "clusters": 12, "cluster_wins": 0 } } }, { "key": "lon_mae", "label": "Longitudinal error", "unit": "m/s²", "better": "lower", "value": 0.3542, "median": 0.3567, "ci": [ 0.2773, 0.4004 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.01449, "median": -0.01135, "ci": [ -0.03501, 0.01018 ], "n": 119, "wins": 73, "ties": 0, "clusters": 12, "cluster_wins": 5 }, "870a4823_12864": { "delta": -0.0038, "median": -0.001121, "ci": [ -0.0174, 0.001501 ], "n": 119, "wins": 65, "ties": 0, "clusters": 12, "cluster_wins": 5 } } }, { "key": "lon_corr", "label": "Longitudinal correlation", "unit": "", "better": "higher", "value": 0.3391, "median": 0.2915, "ci": [ 0.2852, 0.5846 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.01223, "median": 0.01559, "ci": [ -0.03365, 0.04104 ], "n": 119, "wins": 68, "ties": 0, "clusters": 12, "cluster_wins": 6 }, "870a4823_12864": { "delta": 0.001508, "median": 0.001838, "ci": [ -0.01963, 0.01022 ], "n": 119, "wins": 64, "ties": 0, "clusters": 12, "cluster_wins": 7 } } }, { "key": "lon_brake_mae", "label": "Longitudinal error while braking", "unit": "m/s²", "better": "lower", "value": 0.4759, "median": 0.4603, "ci": [ 0.4449, 0.59 ], "n": 81, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.02287, "median": -0.0254, "ci": [ -0.04279, 0.03585 ], "n": 81, "wins": 47, "ties": 0, "clusters": 12, "cluster_wins": 7 }, "870a4823_12864": { "delta": 0.003772, "median": 0.007128, "ci": [ -0.004738, 0.02188 ], "n": 81, "wins": 34, "ties": 0, "clusters": 12, "cluster_wins": 5 } } }, { "key": "lon_rate_rms", "label": "Longitudinal action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 4.484, "median": 4.299, "ci": [ 2.05, 5.85 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -1.125, "median": -1.314, "ci": [ -1.741, -0.7443 ], "n": 119, "wins": 91, "ties": 0, "clusters": 12, "cluster_wins": 11 }, "870a4823_12864": { "delta": 0.02123, "median": -0.01015, "ci": [ -0.04212, 0.05202 ], "n": 119, "wins": 62, "ties": 0, "clusters": 12, "cluster_wins": 8 } } }, { "key": "v3_mae", "label": "Plan speed error at 3 s", "unit": "m/s", "better": "lower", "value": 1.889, "median": 1.838, "ci": [ 1.152, 2.301 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.1086, "median": -0.1249, "ci": [ -0.3518, 0.306 ], "n": 119, "wins": 78, "ties": 0, "clusters": 12, "cluster_wins": 5 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": [ 0.0, 0.0 ], "n": 119, "wins": 0, "ties": 119, "clusters": 12, "cluster_wins": 0 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 4, "clusters": 2, "metrics": [ { "key": "lat_mae", "value": 0.1144, "median": 0.1082, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.0267, "median": -0.0344, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.007707, "median": 0.008411, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1048, "median": 0.1038, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.03479, "median": -0.0454, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.003126, "median": 0.003346, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.8116, "median": 0.8918, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.01643, "median": -0.002299, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.02274, "median": -0.01334, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.7125, "median": 0.7992, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.04855, "median": -0.036, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.0597, "median": 0.05741, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_rate_rms", "value": 0.8162, "median": 0.781, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3367, "median": -0.3469, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.01919, "median": -0.01129, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lat_best_lag", "value": 0.675, "median": 0.625, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1625, "median": 0.225, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.1, "median": 0.1, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.5767, "median": 0.5213, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.201, "median": -0.2042, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.11, "median": -0.1135, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_corr", "value": 0.5359, "median": 0.5468, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.0005326, "median": 0.01419, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.05824, "median": -0.03115, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_brake_mae", "value": 0.3392, "median": 0.3392, "ci": null, "n": 2, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1513, "median": 0.1513, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.08676, "median": 0.08676, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_rate_rms", "value": 1.895, "median": 1.835, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -1.399, "median": -1.314, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.146, "median": -0.1621, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "v3_mae", "value": 1.443, "median": 1.507, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.9457, "median": -0.9688, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 4, "wins": 0, "ties": 4, "clusters": 2, "cluster_wins": 0 } } } ] }, "stock-dual": { "n": 13, "clusters": 4, "metrics": [ { "key": "lat_mae", "value": 0.1431, "median": 0.1286, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.00212, "median": -0.01026, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.002447, "median": 0.004189, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lat_mae_bl", "value": 0.126, "median": 0.1131, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.01328, "median": -0.01562, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.002861, "median": -0.003037, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lat_corr", "value": 0.8174, "median": 0.8256, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.004121, "median": -0.003423, "ci": null, "n": 13, "wins": 6, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.02343, "median": -0.0163, "ci": null, "n": 13, "wins": 0, "ties": 0, "clusters": 4, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.8692, "median": 0.8385, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.04898, "median": 0.04293, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.02283, "median": 0.01632, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 0 } } }, { "key": "lat_rate_rms", "value": 0.7021, "median": 0.6727, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.4738, "median": -0.4494, "ci": null, "n": 13, "wins": 13, "ties": 0, "clusters": 4, "cluster_wins": 4 }, "870a4823_12864": { "delta": 0.1084, "median": 0.1012, "ci": null, "n": 13, "wins": 0, "ties": 0, "clusters": 4, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "value": 0.7385, "median": 0.65, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.2269, "median": 0.2, "ci": null, "n": 13, "wins": 0, "ties": 1, "clusters": 4, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.08846, "median": 0.1, "ci": null, "n": 13, "wins": 0, "ties": 2, "clusters": 4, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.2714, "median": 0.2662, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.04317, "median": 0.03394, "ci": null, "n": 13, "wins": 2, "ties": 0, "clusters": 4, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.005974, "median": 0.003198, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lon_corr", "value": 0.5901, "median": 0.7168, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.04408, "median": -0.01, "ci": null, "n": 13, "wins": 6, "ties": 0, "clusters": 4, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.0001749, "median": 0.01119, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lon_brake_mae", "value": 0.5978, "median": 0.6265, "ci": null, "n": 6, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.04947, "median": -0.03602, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.002357, "median": 0.008325, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lon_rate_rms", "value": 1.211, "median": 1.21, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -1.003, "median": -1.052, "ci": null, "n": 13, "wins": 13, "ties": 0, "clusters": 4, "cluster_wins": 4 }, "870a4823_12864": { "delta": 0.03032, "median": 0.027, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "v3_mae", "value": 0.8066, "median": 0.8323, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.03457, "median": -0.06044, "ci": null, "n": 13, "wins": 8, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 13, "wins": 0, "ties": 13, "clusters": 4, "cluster_wins": 0 } } } ] }, "stock-newcase": { "n": 2, "clusters": 1, "metrics": [ { "key": "lat_mae", "value": 0.1442, "median": 0.1442, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.009863, "median": 0.009863, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.01375, "median": 0.01375, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1371, "median": 0.1371, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.001918, "median": 0.001918, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.01123, "median": 0.01123, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.8141, "median": 0.8141, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.03349, "median": -0.03349, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.02105, "median": -0.02105, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.7754, "median": 0.7754, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.01452, "median": 0.01452, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.04218, "median": 0.04218, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_rate_rms", "value": 0.8053, "median": 0.8053, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.4703, "median": -0.4703, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.03128, "median": 0.03128, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "value": 0.6, "median": 0.6, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.1, "median": 0.1, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.125, "median": 0.125, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.3761, "median": 0.3761, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.01363, "median": 0.01363, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.008606, "median": 0.008606, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.5398, "median": 0.5398, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.05321, "median": 0.05321, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.01766, "median": -0.01766, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_brake_mae", "value": 0.389, "median": 0.389, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.1143, "median": -0.1143, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.005638, "median": 0.005638, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_rate_rms", "value": 2.35, "median": 2.35, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -1.006, "median": -1.006, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.07508, "median": -0.07508, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "v3_mae", "value": 2.129, "median": 2.129, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.03487, "median": 0.03487, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 2, "wins": 0, "ties": 2, "clusters": 1, "cluster_wins": 0 } } } ] }, "v13-8mm": { "n": 92, "clusters": 3, "metrics": [ { "key": "lat_mae", "value": 0.1702, "median": 0.1659, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.0114, "median": -0.008981, "ci": null, "n": 92, "wins": 58, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.007369, "median": 0.006573, "ci": null, "n": 92, "wins": 14, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1508, "median": 0.1323, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.02137, "median": -0.0196, "ci": null, "n": 92, "wins": 77, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.002448, "median": 0.001745, "ci": null, "n": 92, "wins": 37, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.7968, "median": 0.9016, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.04832, "median": 0.01798, "ci": null, "n": 92, "wins": 72, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.01212, "median": -0.006941, "ci": null, "n": 92, "wins": 13, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.7667, "median": 0.8057, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.001452, "median": 0.01329, "ci": null, "n": 92, "wins": 37, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.02266, "median": 0.0176, "ci": null, "n": 92, "wins": 24, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_rate_rms", "value": 1.208, "median": 0.9415, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.409, "median": -1.356, "ci": null, "n": 92, "wins": 91, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.008041, "median": 0.0006478, "ci": null, "n": 92, "wins": 44, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.7247, "median": 0.7, "ci": null, "n": 91, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.07143, "median": 0.1, "ci": null, "n": 91, "wins": 19, "ties": 15, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.06813, "median": 0.05, "ci": null, "n": 91, "wins": 1, "ties": 24, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.3565, "median": 0.3683, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.01953, "median": -0.01972, "ci": null, "n": 92, "wins": 64, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.0008197, "median": -0.001297, "ci": null, "n": 92, "wins": 52, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lon_corr", "value": 0.2605, "median": 0.2525, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.02678, "median": 0.03207, "ci": null, "n": 92, "wins": 56, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.005216, "median": 0.002278, "ci": null, "n": 92, "wins": 50, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "lon_brake_mae", "value": 0.4571, "median": 0.4585, "ci": null, "n": 65, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.03201, "median": -0.02986, "ci": null, "n": 65, "wins": 39, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.003438, "median": 0.009269, "ci": null, "n": 65, "wins": 27, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lon_rate_rms", "value": 5.099, "median": 5.221, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.234, "median": -1.531, "ci": null, "n": 92, "wins": 69, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.0339, "median": 0.00301, "ci": null, "n": 92, "wins": 44, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "v3_mae", "value": 2.125, "median": 1.978, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1094, "median": -0.2267, "ci": null, "n": 92, "wins": 63, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 92, "wins": 0, "ties": 92, "clusters": 3, "cluster_wins": 0 } } } ] }, "v14-8mm": { "n": 8, "clusters": 2, "metrics": [ { "key": "lat_mae", "value": 0.1569, "median": 0.1593, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.01548, "median": 0.01253, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.004061, "median": 0.00468, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1401, "median": 0.1361, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.003039, "median": 0.001307, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.001425, "median": -0.0001784, "ci": null, "n": 8, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_corr", "value": 0.8438, "median": 0.9132, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02422, "median": -0.009128, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.01356, "median": -0.0102, "ci": null, "n": 8, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.8562, "median": 0.8512, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.05445, "median": 0.02537, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.00989, "median": 0.002489, "ci": null, "n": 8, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "value": 0.6112, "median": 0.5248, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.5443, "median": -0.4624, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.04787, "median": 0.03191, "ci": null, "n": 8, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "value": 0.6375, "median": 0.575, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1875, "median": 0.2, "ci": null, "n": 8, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.075, "median": 0.1, "ci": null, "n": 8, "wins": 0, "ties": 2, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.3455, "median": 0.2851, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.03609, "median": 0.03497, "ci": null, "n": 8, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.003976, "median": -0.0008029, "ci": null, "n": 8, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_corr", "value": 0.6866, "median": 0.706, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.06806, "median": -0.03637, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.004298, "median": -0.005137, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_brake_mae", "value": 0.6327, "median": 0.6929, "ci": null, "n": 6, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.07509, "median": 0.03092, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.01477, "median": -0.01562, "ci": null, "n": 6, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lon_rate_rms", "value": 4.563, "median": 4.42, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.02959, "median": 0.2078, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.03164, "median": -0.03598, "ci": null, "n": 8, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "v3_mae", "value": 1.095, "median": 1.065, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1622, "median": 0.1656, "ci": null, "n": 8, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 8, "wins": 0, "ties": 8, "clusters": 2, "cluster_wins": 0 } } } ] } } }, "truck_closed_loop": { "status": "ok", "unit_of_evidence": "window", "n": 98, "clusters": 13, "condition": "Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. The simulation starts from the truck's state (speed, yaw rate, steering, gear) in its CAN recording of the 2 s before the window; one window, whose 2 s before are not all valid CAN, starts steady at its recorded speed. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 98 windows (59 day, 39 night).", "baselines": [ "lebowski_877m", "870a4823_12864" ], "metrics": [ { "key": "xtrack_2s", "label": "Distance from the recorded path at 2 s", "unit": "m", "better": "lower", "value": 0.1317, "median": 0.09121, "ci": [ 0.1025, 0.1581 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.03779, "median": -0.02195, "ci": [ -0.05493, -0.02288 ], "n": 98, "wins": 67, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "870a4823_12864": { "delta": -0.0008397, "median": -0.003732, "ci": [ -0.01049, 0.009215 ], "n": 98, "wins": 53, "ties": 0, "clusters": 13, "cluster_wins": 9 } } }, { "key": "xtrack_4s", "label": "Distance from the recorded path at 4 s", "unit": "m", "better": "lower", "value": 0.5081, "median": 0.3275, "ci": [ 0.3739, 0.6534 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.01195, "median": -0.01167, "ci": [ -0.1794, 0.1418 ], "n": 98, "wins": 51, "ties": 0, "clusters": 13, "cluster_wins": 8 }, "870a4823_12864": { "delta": 0.01227, "median": -0.006419, "ci": [ -0.01, 0.03709 ], "n": 98, "wins": 50, "ties": 0, "clusters": 13, "cluster_wins": 4 } } }, { "key": "xtrack_7s", "label": "Distance from the recorded path at 7.2 s", "unit": "m", "better": "lower", "value": 2.046, "median": 0.7946, "ci": [ 1.375, 3.042 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.06676, "median": 0.02241, "ci": [ -0.7992, 1.119 ], "n": 98, "wins": 46, "ties": 0, "clusters": 13, "cluster_wins": 5 }, "870a4823_12864": { "delta": 0.008737, "median": 0.01833, "ci": [ -0.1057, 0.1387 ], "n": 98, "wins": 44, "ties": 0, "clusters": 13, "cluster_wins": 6 } } }, { "key": "fail_1m", "label": "Share of episodes ever more than 1 m off the path", "unit": "", "better": "lower", "value": 0.4031, "median": 0.0, "ci": [ 0.3261, 0.4943 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.02551, "median": 0.0, "ci": [ -0.08421, 0.03615 ], "n": 98, "wins": 13, "ties": 76, "clusters": 13, "cluster_wins": 7 }, "870a4823_12864": { "delta": 0.005102, "median": 0.0, "ci": [ -0.03686, 0.04593 ], "n": 98, "wins": 7, "ties": 84, "clusters": 13, "cluster_wins": 4 } } }, { "key": "lat_jerk", "label": "Requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.264, "median": 0.1794, "ci": [ 0.219, 0.3058 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.2779, "median": -0.2069, "ci": [ -0.3644, -0.1968 ], "n": 98, "wins": 89, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "870a4823_12864": { "delta": -0.001762, "median": -0.006794, "ci": [ -0.02716, 0.01951 ], "n": 98, "wins": 58, "ties": 0, "clusters": 13, "cluster_wins": 8 } } }, { "key": "long_jerk", "label": "Requested longitudinal jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3708, "median": 0.2948, "ci": [ 0.3166, 0.4349 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.01546, "median": -0.05707, "ci": [ -0.08628, 0.06208 ], "n": 98, "wins": 64, "ties": 0, "clusters": 13, "cluster_wins": 8 }, "870a4823_12864": { "delta": -0.004934, "median": 0.003193, "ci": [ -0.01948, 0.006349 ], "n": 98, "wins": 46, "ties": 0, "clusters": 13, "cluster_wins": 8 } } }, { "key": "speed_err", "label": "Speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.217, "median": 0.9263, "ci": [ 0.9907, 1.437 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.1342, "median": 0.1061, "ci": [ -0.04422, 0.303 ], "n": 98, "wins": 39, "ties": 0, "clusters": 13, "cluster_wins": 5 }, "870a4823_12864": { "delta": -0.04275, "median": -0.01326, "ci": [ -0.1069, 0.007462 ], "n": 98, "wins": 51, "ties": 0, "clusters": 13, "cluster_wins": 8 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 14, "clusters": 2, "metrics": [ { "key": "xtrack_2s", "value": 0.1211, "median": 0.08018, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02921, "median": -0.01034, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.00237, "median": -0.003778, "ci": null, "n": 14, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "xtrack_4s", "value": 0.5433, "median": 0.4166, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.09215, "median": -0.004499, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.0119, "median": -0.01125, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_7s", "value": 2.536, "median": 2.194, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.6307, "median": -0.1259, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.3714, "median": -0.01419, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "fail_1m", "value": 0.6786, "median": 1.0, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.0, "median": 0.0, "ci": null, "n": 14, "wins": 2, "ties": 9, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 14, "wins": 2, "ties": 10, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lat_jerk", "value": 0.3843, "median": 0.2352, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1865, "median": -0.2165, "ci": null, "n": 14, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.0701, "median": -0.01065, "ci": null, "n": 14, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "long_jerk", "value": 0.33, "median": 0.2761, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1647, "median": -0.1213, "ci": null, "n": 14, "wins": 13, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.0004805, "median": -0.00171, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "speed_err", "value": 1.738, "median": 1.851, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02605, "median": 0.09553, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.1962, "median": -0.2616, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } } ] }, "stock-dual": { "n": 43, "clusters": 5, "metrics": [ { "key": "xtrack_2s", "value": 0.1278, "median": 0.08363, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.04026, "median": -0.04313, "ci": null, "n": 43, "wins": 35, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "870a4823_12864": { "delta": -0.01066, "median": -0.01218, "ci": null, "n": 43, "wins": 28, "ties": 0, "clusters": 5, "cluster_wins": 4 } } }, { "key": "xtrack_4s", "value": 0.4207, "median": 0.3238, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.03152, "median": -0.02542, "ci": null, "n": 43, "wins": 25, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.0008651, "median": -0.03308, "ci": null, "n": 43, "wins": 26, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "xtrack_7s", "value": 1.626, "median": 0.5751, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.3934, "median": 0.02371, "ci": null, "n": 43, "wins": 19, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.1545, "median": -0.01823, "ci": null, "n": 43, "wins": 24, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "fail_1m", "value": 0.3372, "median": 0.0, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.0, "median": 0.0, "ci": null, "n": 43, "wins": 5, "ties": 33, "clusters": 5, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.01163, "median": 0.0, "ci": null, "n": 43, "wins": 3, "ties": 37, "clusters": 5, "cluster_wins": 2 } } }, { "key": "lat_jerk", "value": 0.2776, "median": 0.1892, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.2751, "median": -0.1886, "ci": null, "n": 43, "wins": 39, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "870a4823_12864": { "delta": 0.01553, "median": -0.006327, "ci": null, "n": 43, "wins": 25, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "long_jerk", "value": 0.4215, "median": 0.3099, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.04731, "median": -0.04148, "ci": null, "n": 43, "wins": 26, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.005539, "median": 0.009858, "ci": null, "n": 43, "wins": 16, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "speed_err", "value": 1.222, "median": 0.9263, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.1384, "median": 0.09756, "ci": null, "n": 43, "wins": 19, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.0144, "median": -0.03206, "ci": null, "n": 43, "wins": 23, "ties": 0, "clusters": 5, "cluster_wins": 3 } } } ] }, "stock-newcase": { "n": 4, "clusters": 1, "metrics": [ { "key": "xtrack_2s", "value": 0.0511, "median": 0.04587, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.07377, "median": -0.07113, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.01307, "median": 0.006202, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "xtrack_4s", "value": 0.2069, "median": 0.2018, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.03377, "median": 0.02157, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.00145, "median": 0.009579, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "xtrack_7s", "value": 1.076, "median": 1.053, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.5593, "median": -0.4635, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.16, "median": -0.1324, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "fail_1m", "value": 0.5, "median": 0.5, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.125, "median": 0.0, "ci": null, "n": 4, "wins": 1, "ties": 3, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 4, "wins": 0, "ties": 4, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.2444, "median": 0.1822, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.4954, "median": -0.3393, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.01361, "median": -0.01478, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "long_jerk", "value": 0.3757, "median": 0.361, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.01347, "median": -0.05325, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.1277, "median": -0.05075, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "speed_err", "value": 0.6825, "median": 0.7023, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.003678, "median": 0.113, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.2645, "median": -0.1939, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } } ] }, "v13-8mm": { "n": 18, "clusters": 3, "metrics": [ { "key": "xtrack_2s", "value": 0.1721, "median": 0.09447, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.04518, "median": -0.0365, "ci": null, "n": 18, "wins": 12, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.01792, "median": 0.01537, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "xtrack_4s", "value": 0.4819, "median": 0.2709, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.3202, "median": -0.01919, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.06757, "median": 0.05447, "ci": null, "n": 18, "wins": 5, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "xtrack_7s", "value": 1.303, "median": 0.7989, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.322, "median": 0.04322, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.1276, "median": 0.09194, "ci": null, "n": 18, "wins": 4, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "fail_1m", "value": 0.3333, "median": 0.0, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.05556, "median": 0.0, "ci": null, "n": 18, "wins": 2, "ties": 16, "clusters": 3, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 18, "wins": 1, "ties": 16, "clusters": 3, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.2202, "median": 0.15, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1592, "median": -0.1201, "ci": null, "n": 18, "wins": 16, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.02508, "median": 0.008159, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "long_jerk", "value": 0.31, "median": 0.2433, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.005424, "median": -0.03047, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.001618, "median": -0.005004, "ci": null, "n": 18, "wins": 10, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "speed_err", "value": 0.755, "median": 0.6867, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1897, "median": -0.007125, "ci": null, "n": 18, "wins": 10, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.03633, "median": -0.008251, "ci": null, "n": 18, "wins": 9, "ties": 0, "clusters": 3, "cluster_wins": 2 } } } ] }, "v14-8mm": { "n": 19, "clusters": 2, "metrics": [ { "key": "xtrack_2s", "value": 0.127, "median": 0.1108, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02397, "median": -0.0009106, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.001811, "median": -0.003315, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_4s", "value": 0.7681, "median": 0.3345, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2523, "median": 0.04862, "ci": null, "n": 19, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.005756, "median": -0.02704, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_7s", "value": 3.545, "median": 0.7932, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 2.14, "median": -0.02234, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.03387, "median": 0.06589, "ci": null, "n": 19, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "fail_1m", "value": 0.3947, "median": 0.0, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.05263, "median": 0.0, "ci": null, "n": 19, "wins": 3, "ties": 15, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 19, "wins": 1, "ties": 17, "clusters": 2, "cluster_wins": 1 } } }, { "key": "lat_jerk", "value": 0.1902, "median": 0.1786, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.4183, "median": -0.2665, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.01349, "median": -0.01439, "ci": null, "n": 19, "wins": 12, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "long_jerk", "value": 0.3427, "median": 0.3129, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.06775, "median": -0.05455, "ci": null, "n": 19, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.013, "median": -0.0002204, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "speed_err", "value": 1.372, "median": 1.115, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.5771, "median": 0.4169, "ci": null, "n": 19, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.04679, "median": 0.04378, "ci": null, "n": 19, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 } } } ] } }, "simulator": "g55-2026-10-31" }, "truck_recovery": { "status": "ok", "unit_of_evidence": "window", "n": 100, "clusters": 14, "condition": "Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 98 windows (59 day, 39 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 98 windows (59 day, 39 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).", "baselines": [ "lebowski_877m", "870a4823_12864" ], "metrics": [ { "key": "heading_xtrack_2s", "label": "2° off heading: distance from the path at 2 s", "unit": "m", "better": "lower", "value": 0.4916, "median": 0.4102, "ci": [ 0.3994, 0.605 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.03867, "median": 0.0301, "ci": [ 0.00144, 0.07912 ], "n": 98, "wins": 42, "ties": 0, "clusters": 13, "cluster_wins": 5 }, "870a4823_12864": { "delta": -0.02632, "median": -0.02585, "ci": [ -0.03128, -0.02184 ], "n": 98, "wins": 89, "ties": 0, "clusters": 13, "cluster_wins": 13 } } }, { "key": "heading_xtrack_4s", "label": "2° off heading: distance from the path at 4 s", "unit": "m", "better": "lower", "value": 0.7479, "median": 0.4822, "ci": [ 0.55, 0.9572 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.04538, "median": 0.04001, "ci": [ -0.1048, 0.1766 ], "n": 98, "wins": 41, "ties": 0, "clusters": 13, "cluster_wins": 4 }, "870a4823_12864": { "delta": 0.006975, "median": 0.01142, "ci": [ -0.01525, 0.02961 ], "n": 98, "wins": 45, "ties": 0, "clusters": 13, "cluster_wins": 7 } } }, { "key": "heading_lat_jerk", "label": "2° off heading: requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3623, "median": 0.2925, "ci": [ 0.3202, 0.408 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.5246, "median": -0.4886, "ci": [ -0.5932, -0.4527 ], "n": 98, "wins": 96, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "870a4823_12864": { "delta": -0.003317, "median": -0.01123, "ci": [ -0.02983, 0.02119 ], "n": 98, "wins": 64, "ties": 0, "clusters": 13, "cluster_wins": 5 } } }, { "key": "offcentre_xtrack_2s", "label": "0.5 m off centre: distance from the path at 2 s", "unit": "m", "better": "lower", "value": 0.2577, "median": 0.2127, "ci": [ 0.2335, 0.2822 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.009311, "median": 0.001411, "ci": [ -0.03394, 0.01637 ], "n": 98, "wins": 48, "ties": 0, "clusters": 13, "cluster_wins": 6 }, "870a4823_12864": { "delta": -0.02933, "median": -0.02916, "ci": [ -0.03612, -0.02267 ], "n": 98, "wins": 83, "ties": 0, "clusters": 13, "cluster_wins": 13 } } }, { "key": "offcentre_xtrack_4s", "label": "0.5 m off centre: distance from the path at 4 s", "unit": "m", "better": "lower", "value": 0.6593, "median": 0.4332, "ci": [ 0.5357, 0.7827 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.003547, "median": -0.02943, "ci": [ -0.09162, 0.09323 ], "n": 98, "wins": 55, "ties": 0, "clusters": 13, "cluster_wins": 7 }, "870a4823_12864": { "delta": 0.02559, "median": 0.01534, "ci": [ -0.007812, 0.06262 ], "n": 98, "wins": 39, "ties": 0, "clusters": 13, "cluster_wins": 3 } } }, { "key": "offcentre_lat_jerk", "label": "0.5 m off centre: requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.5516, "median": 0.4134, "ci": [ 0.4865, 0.6205 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.5588, "median": -0.5567, "ci": [ -0.6433, -0.4732 ], "n": 98, "wins": 93, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "870a4823_12864": { "delta": 0.01853, "median": 0.00572, "ci": [ -0.01722, 0.05558 ], "n": 98, "wins": 43, "ties": 0, "clusters": 13, "cluster_wins": 5 } } }, { "key": "brake_ahead_4s", "label": "Braking lead: distance ahead of the recorded car at 4 s", "unit": "m", "better": "lower", "value": -0.3168, "median": -0.3168, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 1.021, "median": 1.021, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.6519, "median": 0.6519, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "brake_gap_min", "label": "Braking lead: smallest gap to the lead (derived)", "unit": "m", "better": "higher", "value": 24.89, "median": 24.89, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -2.571, "median": -2.571, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": -2.92, "median": -2.92, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "brake_speed_err", "label": "Braking lead: speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.248, "median": 1.248, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.04457, "median": 0.04457, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.1817, "median": 0.1817, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 14, "clusters": 2, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.3958, "median": 0.3629, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.004021, "median": -0.004532, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.01135, "median": -0.009287, "ci": null, "n": 14, "wins": 12, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "heading_xtrack_4s", "value": 0.694, "median": 0.5212, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.141, "median": 0.02787, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.008609, "median": -0.01102, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "heading_lat_jerk", "value": 0.3958, "median": 0.3027, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3721, "median": -0.3307, "ci": null, "n": 14, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.04156, "median": -0.02048, "ci": null, "n": 14, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2663, "median": 0.2434, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.003401, "median": -0.005949, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.01662, "median": -0.01683, "ci": null, "n": 14, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_4s", "value": 0.669, "median": 0.6389, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.07924, "median": 0.1396, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.03489, "median": 0.03711, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.6915, "median": 0.6046, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3702, "median": -0.4252, "ci": null, "n": 14, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": -0.0006925, "median": 0.00842, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 } } } ] }, "stock-dual": { "n": 45, "clusters": 6, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.4162, "median": 0.367, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.0006042, "median": -0.03118, "ci": null, "n": 43, "wins": 24, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.02592, "median": -0.0254, "ci": null, "n": 43, "wins": 40, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "heading_xtrack_4s", "value": 0.5983, "median": 0.4259, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.03793, "median": -0.005427, "ci": null, "n": 43, "wins": 22, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.01681, "median": -0.0183, "ci": null, "n": 43, "wins": 22, "ties": 0, "clusters": 5, "cluster_wins": 4 } } }, { "key": "heading_lat_jerk", "value": 0.3766, "median": 0.2864, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.5198, "median": -0.4896, "ci": null, "n": 43, "wins": 41, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "870a4823_12864": { "delta": -0.004081, "median": -0.01347, "ci": null, "n": 43, "wins": 30, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2524, "median": 0.2105, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.04688, "median": -0.0393, "ci": null, "n": 43, "wins": 29, "ties": 0, "clusters": 5, "cluster_wins": 4 }, "870a4823_12864": { "delta": -0.03701, "median": -0.03074, "ci": null, "n": 43, "wins": 40, "ties": 0, "clusters": 5, "cluster_wins": 5 } } }, { "key": "offcentre_xtrack_4s", "value": 0.6161, "median": 0.4028, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.0543, "median": -0.06253, "ci": null, "n": 43, "wins": 29, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.005825, "median": -0.001086, "ci": null, "n": 43, "wins": 22, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "offcentre_lat_jerk", "value": 0.5971, "median": 0.4693, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.5795, "median": -0.5535, "ci": null, "n": 43, "wins": 41, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "870a4823_12864": { "delta": 0.01234, "median": 0.004903, "ci": null, "n": 43, "wins": 19, "ties": 0, "clusters": 5, "cluster_wins": 3 } } }, { "key": "brake_ahead_4s", "value": -0.3168, "median": -0.3168, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 1.021, "median": 1.021, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.6519, "median": 0.6519, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "brake_gap_min", "value": 24.89, "median": 24.89, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -2.571, "median": -2.571, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": -2.92, "median": -2.92, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "brake_speed_err", "value": 1.248, "median": 1.248, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.04457, "median": 0.04457, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.1817, "median": 0.1817, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ] }, "stock-newcase": { "n": 4, "clusters": 1, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.3577, "median": 0.384, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.09292, "median": -0.1149, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.03149, "median": -0.03216, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "heading_xtrack_4s", "value": 0.2712, "median": 0.207, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.05156, "median": 0.1046, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.02549, "median": 0.01281, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "heading_lat_jerk", "value": 0.2872, "median": 0.2064, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.685, "median": -0.6591, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.02095, "median": 0.01264, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_2s", "value": 0.3147, "median": 0.2189, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.01484, "median": 0.03616, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.04297, "median": -0.03244, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_4s", "value": 0.6983, "median": 0.4616, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.02936, "median": -0.06336, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.1429, "median": 0.09212, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "offcentre_lat_jerk", "value": 0.6678, "median": 0.4739, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.7295, "median": -0.6287, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.02565, "median": 0.02598, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ] }, "v13-8mm": { "n": 18, "clusters": 3, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.863, "median": 0.8214, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.17, "median": 0.2042, "ci": null, "n": 18, "wins": 3, "ties": 0, "clusters": 3, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.03429, "median": -0.04691, "ci": null, "n": 18, "wins": 16, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "heading_xtrack_4s", "value": 1.175, "median": 0.8219, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.05671, "median": 0.09402, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": 0.04705, "median": 0.02582, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "heading_lat_jerk", "value": 0.3795, "median": 0.374, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.4935, "median": -0.4886, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": -0.006566, "median": -0.003847, "ci": null, "n": 18, "wins": 9, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2792, "median": 0.2111, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.02114, "median": 0.02548, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "870a4823_12864": { "delta": -0.02072, "median": -0.03591, "ci": null, "n": 18, "wins": 13, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "offcentre_xtrack_4s", "value": 0.5837, "median": 0.2461, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1093, "median": 0.01274, "ci": null, "n": 18, "wins": 8, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.08541, "median": 0.03103, "ci": null, "n": 18, "wins": 2, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "offcentre_lat_jerk", "value": 0.4174, "median": 0.3784, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.4644, "median": -0.478, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "870a4823_12864": { "delta": 0.03623, "median": 0.04179, "ci": null, "n": 18, "wins": 3, "ties": 0, "clusters": 3, "cluster_wins": 0 } } } ] }, "v14-8mm": { "n": 19, "clusters": 2, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.4091, "median": 0.3874, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.05364, "median": 0.03878, "ci": null, "n": 19, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.02963, "median": -0.03319, "ci": null, "n": 19, "wins": 17, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "heading_xtrack_4s", "value": 0.8224, "median": 0.4311, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2589, "median": 0.1413, "ci": null, "n": 19, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.03042, "median": 0.0231, "ci": null, "n": 19, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "heading_lat_jerk", "value": 0.3048, "median": 0.2493, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.6436, "median": -0.554, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.02456, "median": -0.01049, "ci": null, "n": 19, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2309, "median": 0.2057, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.03241, "median": 0.05006, "ci": null, "n": 19, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": -0.02663, "median": -0.02738, "ci": null, "n": 19, "wins": 15, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_4s", "value": 0.8134, "median": 0.4719, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.156, "median": 0.002298, "ci": null, "n": 19, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "870a4823_12864": { "delta": 0.0335, "median": 0.02739, "ci": null, "n": 19, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_lat_jerk", "value": 0.448, "median": 0.391, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.7046, "median": -0.6862, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "870a4823_12864": { "delta": 0.02843, "median": -0.0164, "ci": null, "n": 19, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 1 } } } ] } }, "simulator": "g55-2026-10-31" } }, "stills": [], "limits": [ "Open-loop scores compare a model's action with what the recording driver or car did; a model that drives differently but well is scored as wrong. comma's RL residual makes the lateral action smoother, and earlier than its base's by 0.10 to 0.26 s (measured on the G55 and on 16 of comma's cars); both score as error here. Measured on the 2026-10-27 deep dive: open-loop accuracy did not predict closed-loop driving.", "Closed-loop numbers come from a generative world model, not a car. In a separate test on the G55's windows, worldmodel-4B followed an added yaw command through 2 s but showed 0 to 20% of an added lateral offset 3 to 4 s later; past about 4 s the recorded anchor frames also pull every model back toward the recorded path. The front page reports 2 s.", "Intervals are 95% percentile bootstraps that resample whole drives (the drive a public segment was cut from, the recorded drive a truck window comes from), so segments of one drive count once. Below 10 drives there is no interval: the braking-lead test comes from one drive and reports means and win counts only.", "Paired deltas are model minus baseline on the units both have (for lateral gain, the change in its distance from 1); wins count units where the model is better in the metric's direction.", "The lead of the lateral request is the shift, searched from 0 to 1.5 s, at which the request best correlates with the car's later lateral motion; a larger value means the request comes earlier. A value at either edge is censored and left out. Plan speed error at 3 s reads the plan head, which does not separate models built on one frozen base: it is identical on every segment for the four f22e06e5 models, and for Cinque v2 and v3.", "Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.", "Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre. The pooled number mixes camera setups, so read the per-setup numbers too. On the V14 setup the wide frame can be up to 90 ms from the simulated one (its camera recorded 7-9 fps).", "Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test has 2 windows from one drive, whose recorded calibration was carried over from an earlier drive." ] }