{ "schema": 1, "model": { "id": "f78ed37d_12864", "name": "Cinque v3", "sha256": "404a18cfd86d29637d20c697dfde245bb47c666ae016730ab674c65f4d1e1aa4", "source": { "kind": "huggingface", "ref": "81a9ad0139976653c041de50b60f1160a873869e", "url": "https://huggingface.co/commaai/openpilot_driving_models/blob/81a9ad0139976653c041de50b60f1160a873869e/f78ed37d-afad-4dbc-8050-40ea885eedde/12864/big_driving_supercombo.onnx", "date": "2026-09-18" }, "first_public": "2026-09-18", "on_master": "2026-09-18", "params_m": 381.59, "gmac": 46.7, "family": "big-rl", "notes": "On openpilot master 2026-09-18 (PR #38932). Vision base b9facbcc, no residual head." }, "scored": { "date": "2026-10-31T23:59:25Z", "suite": "geval-1", "code": "d8eb40eeba87" }, "baselines": [ "lebowski_877m", "1a421175_12864" ], "sections": { "public_open_loop": { "status": "ok", "unit_of_evidence": "segment", "n": 39, "clusters": 23, "condition": "Measured. comma's public two-camera segments (openpilot CI data), native intrinsics and the device's own calibration, both cameras; every model on the same decoded frames. Truth is what the recording car did next: carState speed and acceleration, yaw rate from the device's cameraOdometry. Segments where the car never moves have no data. 39 of the 45 listed segments have a score, from 23 drives and 20 cars.", "baselines": [ "lebowski_877m", "1a421175_12864" ], "metrics": [ { "key": "lat_mae", "label": "Lateral error", "unit": "m/s²", "better": "lower", "value": 0.1365, "median": 0.1238, "ci": [ 0.1083, 0.1663 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.02775, "median": 0.01789, "ci": [ 0.01668, 0.03857 ], "n": 39, "wins": 7, "ties": 0, "clusters": 23, "cluster_wins": 4 }, "1a421175_12864": { "delta": -0.01084, "median": -0.004197, "ci": [ -0.02013, -0.002714 ], "n": 39, "wins": 24, "ties": 0, "clusters": 23, "cluster_wins": 15 } } }, { "key": "lat_mae_bl", "label": "Lateral error at the model's own best lead", "unit": "m/s²", "better": "lower", "value": 0.1283, "median": 0.1151, "ci": [ 0.1007, 0.1569 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.02688, "median": 0.01938, "ci": [ 0.01472, 0.03842 ], "n": 39, "wins": 8, "ties": 0, "clusters": 23, "cluster_wins": 5 }, "1a421175_12864": { "delta": -0.00947, "median": -0.001997, "ci": [ -0.01946, -0.0006818 ], "n": 39, "wins": 21, "ties": 0, "clusters": 23, "cluster_wins": 13 } } }, { "key": "lat_corr", "label": "Lateral correlation with the car's later motion", "unit": "", "better": "higher", "value": 0.8271, "median": 0.9403, "ci": [ 0.7386, 0.9047 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.03055, "median": -0.01549, "ci": [ -0.08119, 0.01127 ], "n": 39, "wins": 8, "ties": 0, "clusters": 23, "cluster_wins": 7 }, "1a421175_12864": { "delta": 0.03194, "median": 0.004763, "ci": [ 0.005967, 0.07431 ], "n": 39, "wins": 28, "ties": 0, "clusters": 23, "cluster_wins": 17 } } }, { "key": "lat_gain", "label": "Lateral gain (1 = same size as the car's motion)", "unit": "", "better": "near1", "value": 0.7539, "median": 0.789, "ci": [ 0.665, 0.8397 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.06466, "median": 0.06049, "ci": [ -0.01364, 0.1332 ], "n": 39, "wins": 14, "ties": 0, "clusters": 23, "cluster_wins": 7 }, "1a421175_12864": { "delta": -0.05364, "median": -0.0301, "ci": [ -0.09007, -0.02757 ], "n": 39, "wins": 35, "ties": 0, "clusters": 23, "cluster_wins": 20 } } }, { "key": "lat_rate_rms", "label": "Lateral action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.4223, "median": 0.3658, "ci": [ 0.3434, 0.5106 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.2416, "median": -0.2375, "ci": [ -0.293, -0.1866 ], "n": 39, "wins": 38, "ties": 0, "clusters": 23, "cluster_wins": 23 }, "1a421175_12864": { "delta": 0.01019, "median": 0.005522, "ci": [ 0.004123, 0.01735 ], "n": 39, "wins": 12, "ties": 0, "clusters": 23, "cluster_wins": 7 } } }, { "key": "lat_best_lag", "label": "Lead of the lateral request over the car's motion", "unit": "s", "better": "lower", "value": 0.4662, "median": 0.425, "ci": [ 0.3658, 0.5766 ], "n": 34, "clusters": 20, "paired": { "lebowski_877m": { "delta": 0.06515, "median": 0.05, "ci": [ 0.009457, 0.1317 ], "n": 33, "wins": 8, "ties": 5, "clusters": 20, "cluster_wins": 4 }, "1a421175_12864": { "delta": -0.0125, "median": -0.05, "ci": [ -0.032, 0.01212 ], "n": 32, "wins": 17, "ties": 9, "clusters": 20, "cluster_wins": 11 } } }, { "key": "lon_mae", "label": "Longitudinal error", "unit": "m/s²", "better": "lower", "value": 0.3017, "median": 0.271, "ci": [ 0.2596, 0.3489 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.006467, "median": -0.0001497, "ci": [ -0.01975, 0.00781 ], "n": 39, "wins": 20, "ties": 0, "clusters": 23, "cluster_wins": 11 }, "1a421175_12864": { "delta": -0.01531, "median": -0.01114, "ci": [ -0.02157, -0.009417 ], "n": 39, "wins": 32, "ties": 0, "clusters": 23, "cluster_wins": 20 } } }, { "key": "lon_corr", "label": "Longitudinal correlation", "unit": "", "better": "higher", "value": 0.7293, "median": 0.8365, "ci": [ 0.6296, 0.8166 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.01151, "median": 0.003878, "ci": [ -0.007832, 0.03073 ], "n": 39, "wins": 21, "ties": 0, "clusters": 23, "cluster_wins": 13 }, "1a421175_12864": { "delta": 0.01201, "median": 0.003787, "ci": [ 0.002695, 0.02422 ], "n": 39, "wins": 24, "ties": 0, "clusters": 23, "cluster_wins": 15 } } }, { "key": "lon_brake_mae", "label": "Longitudinal error while braking", "unit": "m/s²", "better": "lower", "value": 0.4076, "median": 0.3881, "ci": [ 0.3482, 0.4772 ], "n": 30, "clusters": 21, "paired": { "lebowski_877m": { "delta": -0.02664, "median": -0.004074, "ci": [ -0.05752, 0.01007 ], "n": 30, "wins": 15, "ties": 0, "clusters": 21, "cluster_wins": 11 }, "1a421175_12864": { "delta": -0.03309, "median": -0.02598, "ci": [ -0.05383, -0.01613 ], "n": 30, "wins": 25, "ties": 0, "clusters": 21, "cluster_wins": 17 } } }, { "key": "lon_rate_rms", "label": "Longitudinal action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.8805, "median": 0.8155, "ci": [ 0.7989, 0.9705 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": -0.4525, "median": -0.3671, "ci": [ -0.5945, -0.3214 ], "n": 39, "wins": 37, "ties": 0, "clusters": 23, "cluster_wins": 23 }, "1a421175_12864": { "delta": 0.1127, "median": 0.1142, "ci": [ 0.08569, 0.1387 ], "n": 39, "wins": 3, "ties": 0, "clusters": 23, "cluster_wins": 1 } } }, { "key": "v3_mae", "label": "Plan speed error at 3 s", "unit": "m/s", "better": "lower", "value": 0.7585, "median": 0.7436, "ci": [ 0.6582, 0.8885 ], "n": 39, "clusters": 23, "paired": { "lebowski_877m": { "delta": 0.0427, "median": 0.02799, "ci": [ -0.009648, 0.09442 ], "n": 39, "wins": 14, "ties": 0, "clusters": 23, "cluster_wins": 9 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": [ 0.0, 0.0 ], "n": 39, "wins": 0, "ties": 39, "clusters": 23, "cluster_wins": 0 } } } ], "per_unit_file": "per_unit/f78ed37d_12864/public_open_loop.csv" }, "public_closed_loop": { "status": "ok", "unit_of_evidence": "segment", "n": 28, "clusters": 19, "condition": "Measured. comma's worldmodel-4B generates each next frame at 5 Hz from 9 history frames, 5 recorded anchor frames and the car's pose; the model drives comma's ideal car (the requested curvature and acceleration happen at once) for 7.2 s from the recorded start. Distance is to the recording car's own path. One window per segment, nearest the middle, where the car holds at least 5 m/s for the window and the 4.8 s before it; segments with no such stretch are left out. Seeds 0 and 1, averaged per segment. 28 of the 45 listed segments have a score, from 19 drives and 17 cars.", "baselines": [ "lebowski_877m", "1a421175_12864" ], "metrics": [ { "key": "xtrack_2s", "label": "Distance from the recorded path at 2 s", "unit": "m", "better": "lower", "value": 0.1278, "median": 0.06696, "ci": [ 0.05724, 0.249 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.01993, "median": -0.005552, "ci": [ -0.05577, 0.008714 ], "n": 28, "wins": 15, "ties": 0, "clusters": 19, "cluster_wins": 11 }, "1a421175_12864": { "delta": 0.01404, "median": 0.02152, "ci": [ -0.004167, 0.03204 ], "n": 28, "wins": 11, "ties": 0, "clusters": 19, "cluster_wins": 7 } } }, { "key": "xtrack_4s", "label": "Distance from the recorded path at 4 s", "unit": "m", "better": "lower", "value": 0.2841, "median": 0.1374, "ci": [ 0.1438, 0.5192 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": 0.005819, "median": 0.03103, "ci": [ -0.04845, 0.05084 ], "n": 28, "wins": 11, "ties": 0, "clusters": 19, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.03321, "median": 0.0394, "ci": [ -0.001361, 0.06902 ], "n": 28, "wins": 10, "ties": 0, "clusters": 19, "cluster_wins": 6 } } }, { "key": "xtrack_7s", "label": "Distance from the recorded path at 7.2 s", "unit": "m", "better": "lower", "value": 1.388, "median": 0.4039, "ci": [ 0.6004, 2.38 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": 0.3442, "median": 0.06616, "ci": [ 0.01943, 0.7894 ], "n": 28, "wins": 11, "ties": 0, "clusters": 19, "cluster_wins": 8 }, "1a421175_12864": { "delta": -0.3669, "median": 0.03512, "ci": [ -1.168, 0.0627 ], "n": 28, "wins": 11, "ties": 0, "clusters": 19, "cluster_wins": 8 } } }, { "key": "fail_1m", "label": "Share of episodes ever more than 1 m off the path", "unit": "", "better": "lower", "value": 0.2679, "median": 0.0, "ci": [ 0.1087, 0.45 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": 0.03571, "median": 0.0, "ci": [ 0.0, 0.12 ], "n": 28, "wins": 0, "ties": 27, "clusters": 19, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.01786, "median": 0.0, "ci": [ 0.0, 0.05172 ], "n": 28, "wins": 0, "ties": 27, "clusters": 19, "cluster_wins": 0 } } }, { "key": "lat_jerk", "label": "Requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.171, "median": 0.1015, "ci": [ 0.115, 0.2311 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.1546, "median": -0.09391, "ci": [ -0.217, -0.1013 ], "n": 28, "wins": 26, "ties": 0, "clusters": 19, "cluster_wins": 17 }, "1a421175_12864": { "delta": -0.001562, "median": 0.002036, "ci": [ -0.009568, 0.005966 ], "n": 28, "wins": 13, "ties": 0, "clusters": 19, "cluster_wins": 10 } } }, { "key": "long_jerk", "label": "Requested longitudinal jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3055, "median": 0.2505, "ci": [ 0.2325, 0.4 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": -0.04085, "median": -0.04612, "ci": [ -0.08426, 0.0176 ], "n": 28, "wins": 21, "ties": 0, "clusters": 19, "cluster_wins": 16 }, "1a421175_12864": { "delta": 0.03641, "median": 0.02889, "ci": [ 0.0162, 0.05638 ], "n": 28, "wins": 5, "ties": 0, "clusters": 19, "cluster_wins": 3 } } }, { "key": "speed_err", "label": "Speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.358, "median": 0.8344, "ci": [ 0.9361, 1.848 ], "n": 28, "clusters": 19, "paired": { "lebowski_877m": { "delta": 0.2569, "median": 0.1595, "ci": [ 0.03161, 0.504 ], "n": 28, "wins": 7, "ties": 0, "clusters": 19, "cluster_wins": 5 }, "1a421175_12864": { "delta": -0.07634, "median": -0.08607, "ci": [ -0.2412, 0.07546 ], "n": 28, "wins": 19, "ties": 0, "clusters": 19, "cluster_wins": 14 } } } ], "per_unit_file": "per_unit/f78ed37d_12864/public_closed_loop.csv" }, "truck_open_loop": { "status": "ok", "unit_of_evidence": "window", "n": 119, "clusters": 12, "condition": "Measured. The G55's recorded drives replayed through the model with both of the truck's cameras (each setup's own wide-camera calibration), road-plane warp to 1.45 m. Where the narrow lens does not cover comma's narrow view (the 8 mm lens in the V13 case), those pixels come from the wide camera, brightness matched. Truth is the truck's CAN speed and yaw rate. 30 s windows after a 5 s warm-up; window = unit, drive = cluster; results are also given per camera setup. Corpus windows of 30 s from the truck's recorded drives, scored per camera setup.", "baselines": [ "lebowski_877m", "1a421175_12864" ], "metrics": [ { "key": "lat_mae", "label": "Lateral error", "unit": "m/s²", "better": "lower", "value": 0.1666, "median": 0.1613, "ci": [ 0.1431, 0.1776 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.006144, "median": -0.004989, "ci": [ -0.01235, 0.01474 ], "n": 119, "wins": 68, "ties": 0, "clusters": 12, "cluster_wins": 4 }, "1a421175_12864": { "delta": 0.006101, "median": 0.006297, "ci": [ 0.001554, 0.009402 ], "n": 119, "wins": 39, "ties": 0, "clusters": 12, "cluster_wins": 2 } } }, { "key": "lat_mae_bl", "label": "Lateral error at the model's own best lead", "unit": "m/s²", "better": "lower", "value": 0.1564, "median": 0.1516, "ci": [ 0.1329, 0.1676 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.008065, "median": -0.009821, "ci": [ -0.01444, 0.01048 ], "n": 119, "wins": 74, "ties": 0, "clusters": 12, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.009893, "median": 0.009829, "ci": [ 0.005002, 0.01315 ], "n": 119, "wins": 23, "ties": 0, "clusters": 12, "cluster_wins": 1 } } }, { "key": "lat_corr", "label": "Lateral correlation with the car's later motion", "unit": "", "better": "higher", "value": 0.82, "median": 0.9098, "ci": [ 0.775, 0.8375 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.05318, "median": 0.02282, "ci": [ 0.002168, 0.06385 ], "n": 119, "wins": 98, "ties": 0, "clusters": 12, "cluster_wins": 6 }, "1a421175_12864": { "delta": 0.005176, "median": 0.002172, "ci": [ -0.001287, 0.0128 ], "n": 119, "wins": 76, "ties": 0, "clusters": 12, "cluster_wins": 9 } } }, { "key": "lat_gain", "label": "Lateral gain (1 = same size as the car's motion)", "unit": "", "better": "near1", "value": 0.8068, "median": 0.8192, "ci": [ 0.7875, 0.8532 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.008939, "median": -0.001654, "ci": [ -0.02259, 0.04916 ], "n": 119, "wins": 61, "ties": 0, "clusters": 12, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.01173, "median": 0.01497, "ci": [ -0.001824, 0.02156 ], "n": 119, "wins": 38, "ties": 0, "clusters": 12, "cluster_wins": 5 } } }, { "key": "lat_rate_rms", "label": "Lateral action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 0.9196, "median": 0.7523, "ci": [ 0.591, 1.084 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -1.371, "median": -1.284, "ci": [ -1.55, -0.5454 ], "n": 119, "wins": 118, "ties": 0, "clusters": 12, "cluster_wins": 12 }, "1a421175_12864": { "delta": 0.04015, "median": 0.01503, "ci": [ -0.02122, 0.07428 ], "n": 119, "wins": 45, "ties": 0, "clusters": 12, "cluster_wins": 6 } } }, { "key": "lat_best_lag", "label": "Lead of the lateral request over the car's motion", "unit": "s", "better": "lower", "value": 0.614, "median": 0.55, "ci": [ 0.5792, 0.6629 ], "n": 118, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.002542, "median": 0.0, "ci": [ -0.0407, 0.09706 ], "n": 118, "wins": 41, "ties": 22, "clusters": 12, "cluster_wins": 4 }, "1a421175_12864": { "delta": -0.04195, "median": -0.05, "ci": [ -0.06843, -0.02813 ], "n": 118, "wins": 67, "ties": 45, "clusters": 12, "cluster_wins": 10 } } }, { "key": "lon_mae", "label": "Longitudinal error", "unit": "m/s²", "better": "lower", "value": 0.3207, "median": 0.3153, "ci": [ 0.2625, 0.3563 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.04795, "median": -0.03841, "ci": [ -0.06565, -0.009541 ], "n": 119, "wins": 94, "ties": 0, "clusters": 12, "cluster_wins": 8 }, "1a421175_12864": { "delta": 0.009895, "median": 0.004956, "ci": [ -0.01493, 0.0191 ], "n": 119, "wins": 48, "ties": 0, "clusters": 12, "cluster_wins": 7 } } }, { "key": "lon_corr", "label": "Longitudinal correlation", "unit": "", "better": "higher", "value": 0.3922, "median": 0.3748, "ci": [ 0.3249, 0.6505 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": 0.06534, "median": 0.05782, "ci": [ 0.01725, 0.07889 ], "n": 119, "wins": 85, "ties": 0, "clusters": 12, "cluster_wins": 8 }, "1a421175_12864": { "delta": -0.01098, "median": -0.01272, "ci": [ -0.01958, 0.01557 ], "n": 119, "wins": 49, "ties": 0, "clusters": 12, "cluster_wins": 8 } } }, { "key": "lon_brake_mae", "label": "Longitudinal error while braking", "unit": "m/s²", "better": "lower", "value": 0.4906, "median": 0.4768, "ci": [ 0.4547, 0.5678 ], "n": 81, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.008172, "median": -0.01463, "ci": [ -0.05356, 0.01446 ], "n": 81, "wins": 44, "ties": 0, "clusters": 12, "cluster_wins": 6 }, "1a421175_12864": { "delta": -0.09719, "median": -0.1029, "ci": [ -0.1077, -0.04331 ], "n": 81, "wins": 79, "ties": 0, "clusters": 12, "cluster_wins": 11 } } }, { "key": "lon_rate_rms", "label": "Longitudinal action rate, RMS (smoothness)", "unit": "m/s³", "better": "lower", "value": 2.374, "median": 2.18, "ci": [ 1.625, 2.668 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -3.236, "median": -2.867, "ci": [ -4.077, -1.471 ], "n": 119, "wins": 119, "ties": 0, "clusters": 12, "cluster_wins": 12 }, "1a421175_12864": { "delta": 0.6603, "median": 0.5178, "ci": [ 0.2116, 0.8503 ], "n": 119, "wins": 10, "ties": 0, "clusters": 12, "cluster_wins": 0 } } }, { "key": "v3_mae", "label": "Plan speed error at 3 s", "unit": "m/s", "better": "lower", "value": 1.831, "median": 1.841, "ci": [ 1.271, 2.032 ], "n": 119, "clusters": 12, "paired": { "lebowski_877m": { "delta": -0.1667, "median": -0.1689, "ci": [ -0.2889, 0.07267 ], "n": 119, "wins": 76, "ties": 0, "clusters": 12, "cluster_wins": 7 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": [ 0.0, 0.0 ], "n": 119, "wins": 0, "ties": 119, "clusters": 12, "cluster_wins": 0 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 4, "clusters": 2, "metrics": [ { "key": "lat_mae", "value": 0.1504, "median": 0.157, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.009273, "median": 0.005735, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.02154, "median": 0.02067, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1415, "median": 0.1478, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.001963, "median": -0.004327, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.02115, "median": 0.0172, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.7441, "median": 0.7892, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.05108, "median": -0.04551, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.05075, "median": -0.05103, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_gain", "value": 0.5968, "median": 0.5914, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.06716, "median": 0.1235, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.09821, "median": 0.1238, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_rate_rms", "value": 1.102, "median": 1.111, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.0511, "median": -0.03804, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.198, "median": 0.1871, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "value": 0.7125, "median": 0.675, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2, "median": 0.225, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.075, "median": 0.075, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_mae", "value": 0.5819, "median": 0.536, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1957, "median": -0.2221, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0196, "median": 0.0285, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.5534, "median": 0.6849, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.01797, "median": 0.001919, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.01432, "median": 0.01915, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_brake_mae", "value": 0.2789, "median": 0.2789, "ci": null, "n": 2, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.09097, "median": 0.09097, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.05968, "median": -0.05968, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_rate_rms", "value": 1.558, "median": 1.398, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -1.736, "median": -1.618, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.1767, "median": 0.1945, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 2.269, "median": 2.257, "ci": null, "n": 4, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1204, "median": -0.2188, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 4, "wins": 0, "ties": 4, "clusters": 2, "cluster_wins": 0 } } } ] }, "stock-dual": { "n": 13, "clusters": 4, "metrics": [ { "key": "lat_mae", "value": 0.1549, "median": 0.1509, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.009698, "median": 0.007023, "ci": null, "n": 13, "wins": 4, "ties": 0, "clusters": 4, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.005853, "median": 0.003884, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lat_mae_bl", "value": 0.1456, "median": 0.1512, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.006307, "median": 0.008959, "ci": null, "n": 13, "wins": 4, "ties": 0, "clusters": 4, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.01093, "median": 0.009946, "ci": null, "n": 13, "wins": 2, "ties": 0, "clusters": 4, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.8319, "median": 0.8464, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.01861, "median": 0.00688, "ci": null, "n": 13, "wins": 8, "ties": 0, "clusters": 4, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.01654, "median": 0.01401, "ci": null, "n": 13, "wins": 9, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lat_gain", "value": 0.8871, "median": 0.8756, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.05369, "median": 0.06818, "ci": null, "n": 13, "wins": 3, "ties": 0, "clusters": 4, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.01033, "median": -0.0008347, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "value": 0.5446, "median": 0.5254, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.6313, "median": -0.631, "ci": null, "n": 13, "wins": 13, "ties": 0, "clusters": 4, "cluster_wins": 4 }, "1a421175_12864": { "delta": -0.02115, "median": -0.01705, "ci": null, "n": 13, "wins": 10, "ties": 0, "clusters": 4, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.5962, "median": 0.55, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.08462, "median": 0.0, "ci": null, "n": 13, "wins": 1, "ties": 6, "clusters": 4, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.09615, "median": -0.1, "ci": null, "n": 13, "wins": 11, "ties": 2, "clusters": 4, "cluster_wins": 4 } } }, { "key": "lon_mae", "value": 0.2424, "median": 0.2026, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.01416, "median": 0.002042, "ci": null, "n": 13, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.01347, "median": -0.01141, "ci": null, "n": 13, "wins": 10, "ties": 0, "clusters": 4, "cluster_wins": 4 } } }, { "key": "lon_corr", "value": 0.6622, "median": 0.7332, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": 0.02804, "median": 0.03821, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.01683, "median": 0.002396, "ci": null, "n": 13, "wins": 7, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lon_brake_mae", "value": 0.5948, "median": 0.5958, "ci": null, "n": 6, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.05252, "median": -0.073, "ci": null, "n": 6, "wins": 4, "ties": 0, "clusters": 4, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.03091, "median": -0.03767, "ci": null, "n": 6, "wins": 5, "ties": 0, "clusters": 4, "cluster_wins": 3 } } }, { "key": "lon_rate_rms", "value": 1.171, "median": 1.212, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -1.044, "median": -0.9665, "ci": null, "n": 13, "wins": 13, "ties": 0, "clusters": 4, "cluster_wins": 4 }, "1a421175_12864": { "delta": 0.1234, "median": 0.1164, "ci": null, "n": 13, "wins": 1, "ties": 0, "clusters": 4, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 0.7788, "median": 0.6927, "ci": null, "n": 13, "clusters": 4, "paired": { "lebowski_877m": { "delta": -0.06241, "median": -0.03433, "ci": null, "n": 13, "wins": 8, "ties": 0, "clusters": 4, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 13, "wins": 0, "ties": 13, "clusters": 4, "cluster_wins": 0 } } } ] }, "stock-newcase": { "n": 2, "clusters": 1, "metrics": [ { "key": "lat_mae", "value": 0.1618, "median": 0.1618, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.02746, "median": 0.02746, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.002826, "median": 0.002826, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1601, "median": 0.1601, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.02491, "median": 0.02491, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.005387, "median": 0.005387, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.835, "median": 0.835, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.01261, "median": -0.01261, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.009354, "median": 0.009354, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_gain", "value": 0.7643, "median": 0.7643, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.02563, "median": 0.02563, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.02906, "median": -0.02906, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "value": 0.8164, "median": 0.8164, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.4591, "median": -0.4591, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.005673, "median": 0.005673, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_best_lag", "value": 0.475, "median": 0.475, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.025, "median": -0.025, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.05, "median": -0.05, "ci": null, "n": 2, "wins": 1, "ties": 1, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lon_mae", "value": 0.351, "median": 0.351, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.01148, "median": -0.01148, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.005292, "median": 0.005292, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_corr", "value": 0.5826, "median": 0.5826, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.09595, "median": 0.09595, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.006822, "median": -0.006822, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lon_brake_mae", "value": 0.3579, "median": 0.3579, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.1454, "median": -0.1454, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.05277, "median": -0.05277, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "lon_rate_rms", "value": 1.934, "median": 1.934, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -1.422, "median": -1.422, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.2373, "median": 0.2373, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 2.496, "median": 2.496, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.4021, "median": 0.4021, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 2, "wins": 0, "ties": 2, "clusters": 1, "cluster_wins": 0 } } } ] }, "v13-8mm": { "n": 92, "clusters": 3, "metrics": [ { "key": "lat_mae", "value": 0.169, "median": 0.1635, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.01258, "median": -0.0104, "ci": null, "n": 92, "wins": 62, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.005483, "median": 0.005943, "ci": null, "n": 92, "wins": 31, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lat_mae_bl", "value": 0.1584, "median": 0.1525, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.01375, "median": -0.01332, "ci": null, "n": 92, "wins": 67, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.00927, "median": 0.009063, "ci": null, "n": 92, "wins": 19, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lat_corr", "value": 0.8181, "median": 0.919, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.06964, "median": 0.02814, "ci": null, "n": 92, "wins": 84, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.005606, "median": 0.001547, "ci": null, "n": 92, "wins": 59, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "lat_gain", "value": 0.7985, "median": 0.8222, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.02848, "median": -0.01737, "ci": null, "n": 92, "wins": 55, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.01091, "median": 0.01576, "ci": null, "n": 92, "wins": 25, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lat_rate_rms", "value": 0.9964, "median": 0.8422, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.621, "median": -1.482, "ci": null, "n": 92, "wins": 92, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.05035, "median": 0.01979, "ci": null, "n": 92, "wins": 27, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.6198, "median": 0.6, "ci": null, "n": 91, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.03352, "median": 0.0, "ci": null, "n": 91, "wins": 38, "ties": 14, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": -0.03791, "median": -0.05, "ci": null, "n": 91, "wins": 48, "ties": 41, "clusters": 3, "cluster_wins": 3 } } }, { "key": "lon_mae", "value": 0.3204, "median": 0.3157, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.05565, "median": -0.0493, "ci": null, "n": 92, "wins": 80, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.01514, "median": 0.01194, "ci": null, "n": 92, "wins": 29, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lon_corr", "value": 0.3116, "median": 0.3272, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.0779, "median": 0.06842, "ci": null, "n": 92, "wins": 69, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": -0.01863, "median": -0.02113, "ci": null, "n": 92, "wins": 31, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lon_brake_mae", "value": 0.4803, "median": 0.4768, "ci": null, "n": 65, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.008786, "median": -0.00794, "ci": null, "n": 65, "wins": 35, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.1107, "median": -0.1087, "ci": null, "n": 65, "wins": 64, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "lon_rate_rms", "value": 2.551, "median": 2.373, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -3.782, "median": -3.473, "ci": null, "n": 92, "wins": 92, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.7976, "median": 0.6324, "ci": null, "n": 92, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 2.004, "median": 1.963, "ci": null, "n": 92, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.2299, "median": -0.289, "ci": null, "n": 92, "wins": 65, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 92, "wins": 0, "ties": 92, "clusters": 3, "cluster_wins": 0 } } } ] }, "v14-8mm": { "n": 8, "clusters": 2, "metrics": [ { "key": "lat_mae", "value": 0.1674, "median": 0.1693, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.02598, "median": 0.02568, "ci": null, "n": 8, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.006709, "median": 0.007008, "ci": null, "n": 8, "wins": 2, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_mae_bl", "value": 0.1578, "median": 0.1555, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.02068, "median": 0.02151, "ci": null, "n": 8, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.01088, "median": 0.01285, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_corr", "value": 0.8565, "median": 0.9208, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.01145, "median": -0.004928, "ci": null, "n": 8, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.008691, "median": 0.003823, "ci": null, "n": 8, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_gain", "value": 0.888, "median": 0.8868, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.06734, "median": 0.04736, "ci": null, "n": 8, "wins": 1, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.009533, "median": 0.004052, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_rate_rms", "value": 0.5803, "median": 0.4872, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.5753, "median": -0.5355, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.04787, "median": -0.04292, "ci": null, "n": 8, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lat_best_lag", "value": 0.5625, "median": 0.55, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.1125, "median": 0.1, "ci": null, "n": 8, "wins": 0, "ties": 2, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.05625, "median": -0.05, "ci": null, "n": 8, "wins": 6, "ties": 1, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_mae", "value": 0.3138, "median": 0.2698, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.004396, "median": -0.001195, "ci": null, "n": 8, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.01614, "median": -0.01437, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_corr", "value": 0.7522, "median": 0.8001, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.002471, "median": 0.01167, "ci": null, "n": 8, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.0181, "median": 0.01324, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_brake_mae", "value": 0.6131, "median": 0.6097, "ci": null, "n": 6, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.05554, "median": 0.00942, "ci": null, "n": 6, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.04449, "median": -0.02757, "ci": null, "n": 6, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "lon_rate_rms", "value": 2.808, "median": 2.485, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": -1.725, "median": -1.431, "ci": null, "n": 8, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.3016, "median": 0.2783, "ci": null, "n": 8, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "v3_mae", "value": 1.158, "median": 1.185, "ci": null, "n": 8, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.2253, "median": 0.1725, "ci": null, "n": 8, "wins": 0, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 8, "wins": 0, "ties": 8, "clusters": 2, "cluster_wins": 0 } } } ] } } }, "truck_closed_loop": { "status": "ok", "unit_of_evidence": "window", "n": 98, "clusters": 13, "condition": "Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. The simulation starts from the truck's state (speed, yaw rate, steering, gear) in its CAN recording of the 2 s before the window; one window, whose 2 s before are not all valid CAN, starts steady at its recorded speed. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 98 windows (59 day, 39 night).", "baselines": [ "lebowski_877m", "1a421175_12864" ], "metrics": [ { "key": "xtrack_2s", "label": "Distance from the recorded path at 2 s", "unit": "m", "better": "lower", "value": 0.15, "median": 0.1047, "ci": [ 0.1238, 0.1725 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.01955, "median": -0.01057, "ci": [ -0.04092, -0.003435 ], "n": 98, "wins": 56, "ties": 0, "clusters": 13, "cluster_wins": 9 }, "1a421175_12864": { "delta": 0.017, "median": 0.01216, "ci": [ 0.006637, 0.02735 ], "n": 98, "wins": 34, "ties": 0, "clusters": 13, "cluster_wins": 2 } } }, { "key": "xtrack_4s", "label": "Distance from the recorded path at 4 s", "unit": "m", "better": "lower", "value": 0.4981, "median": 0.3953, "ci": [ 0.4121, 0.5623 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.02188, "median": 0.006395, "ci": [ -0.1799, 0.06727 ], "n": 98, "wins": 45, "ties": 0, "clusters": 13, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.01366, "median": 0.01667, "ci": [ -0.02796, 0.05231 ], "n": 98, "wins": 42, "ties": 0, "clusters": 13, "cluster_wins": 5 } } }, { "key": "xtrack_7s", "label": "Distance from the recorded path at 7.2 s", "unit": "m", "better": "lower", "value": 1.87, "median": 0.8482, "ci": [ 1.435, 2.355 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.1089, "median": 0.001935, "ci": [ -0.748, 0.5025 ], "n": 98, "wins": 48, "ties": 0, "clusters": 13, "cluster_wins": 5 }, "1a421175_12864": { "delta": -0.06409, "median": -0.01336, "ci": [ -0.2168, 0.08087 ], "n": 98, "wins": 50, "ties": 0, "clusters": 13, "cluster_wins": 9 } } }, { "key": "fail_1m", "label": "Share of episodes ever more than 1 m off the path", "unit": "", "better": "lower", "value": 0.4388, "median": 0.5, "ci": [ 0.3564, 0.5354 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.0102, "median": 0.0, "ci": [ -0.03637, 0.07501 ], "n": 98, "wins": 13, "ties": 72, "clusters": 13, "cluster_wins": 4 }, "1a421175_12864": { "delta": 0.02041, "median": 0.0, "ci": [ -0.03306, 0.07865 ], "n": 98, "wins": 5, "ties": 83, "clusters": 13, "cluster_wins": 2 } } }, { "key": "lat_jerk", "label": "Requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.2296, "median": 0.1726, "ci": [ 0.2046, 0.2569 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.3124, "median": -0.1979, "ci": [ -0.3709, -0.2555 ], "n": 98, "wins": 97, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "1a421175_12864": { "delta": 0.007977, "median": 0.005929, "ci": [ -0.001932, 0.0169 ], "n": 98, "wins": 41, "ties": 0, "clusters": 13, "cluster_wins": 3 } } }, { "key": "long_jerk", "label": "Requested longitudinal jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.2524, "median": 0.226, "ci": [ 0.2184, 0.2801 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.1339, "median": -0.1154, "ci": [ -0.1756, -0.09669 ], "n": 98, "wins": 85, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "1a421175_12864": { "delta": 0.04, "median": 0.02904, "ci": [ 0.02566, 0.05211 ], "n": 98, "wins": 19, "ties": 0, "clusters": 13, "cluster_wins": 1 } } }, { "key": "speed_err", "label": "Speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 1.12, "median": 0.9645, "ci": [ 0.8916, 1.36 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.03735, "median": 0.07156, "ci": [ -0.1129, 0.1941 ], "n": 98, "wins": 46, "ties": 0, "clusters": 13, "cluster_wins": 7 }, "1a421175_12864": { "delta": -0.03109, "median": -0.05864, "ci": [ -0.08093, 0.03136 ], "n": 98, "wins": 58, "ties": 0, "clusters": 13, "cluster_wins": 9 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 14, "clusters": 2, "metrics": [ { "key": "xtrack_2s", "value": 0.1384, "median": 0.09879, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.01187, "median": 0.001628, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.02089, "median": 0.01927, "ci": null, "n": 14, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_4s", "value": 0.4977, "median": 0.5159, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.04656, "median": 0.05825, "ci": null, "n": 14, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.02922, "median": 0.0133, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "xtrack_7s", "value": 2.353, "median": 1.508, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.4479, "median": 0.009273, "ci": null, "n": 14, "wins": 7, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.542, "median": -0.2908, "ci": null, "n": 14, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "fail_1m", "value": 0.7143, "median": 1.0, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.03571, "median": 0.0, "ci": null, "n": 14, "wins": 2, "ties": 8, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 14, "wins": 1, "ties": 12, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.304, "median": 0.252, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.2669, "median": -0.1954, "ci": null, "n": 14, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.001741, "median": -0.002669, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "long_jerk", "value": 0.2539, "median": 0.2069, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.2407, "median": -0.2071, "ci": null, "n": 14, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.02472, "median": 0.0175, "ci": null, "n": 14, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 1.653, "median": 1.781, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.1113, "median": -0.2453, "ci": null, "n": 14, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.04385, "median": 0.09046, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 } } } ] }, "stock-dual": { "n": 43, "clusters": 5, "metrics": [ { "key": "xtrack_2s", "value": 0.1653, "median": 0.1386, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.002758, "median": 0.004125, "ci": null, "n": 43, "wins": 21, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.02416, "median": 0.02324, "ci": null, "n": 43, "wins": 13, "ties": 0, "clusters": 5, "cluster_wins": 0 } } }, { "key": "xtrack_4s", "value": 0.5336, "median": 0.4121, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.08137, "median": 0.01353, "ci": null, "n": 43, "wins": 17, "ties": 0, "clusters": 5, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.0674, "median": 0.05421, "ci": null, "n": 43, "wins": 12, "ties": 0, "clusters": 5, "cluster_wins": 0 } } }, { "key": "xtrack_7s", "value": 1.818, "median": 0.6614, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.2008, "median": 0.008656, "ci": null, "n": 43, "wins": 19, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.1386, "median": 0.07168, "ci": null, "n": 43, "wins": 17, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "fail_1m", "value": 0.407, "median": 0.0, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.06977, "median": 0.0, "ci": null, "n": 43, "wins": 4, "ties": 32, "clusters": 5, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.06977, "median": 0.0, "ci": null, "n": 43, "wins": 1, "ties": 35, "clusters": 5, "cluster_wins": 1 } } }, { "key": "lat_jerk", "value": 0.2223, "median": 0.1617, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.3305, "median": -0.1989, "ci": null, "n": 43, "wins": 42, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.00833, "median": 0.004349, "ci": null, "n": 43, "wins": 20, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "long_jerk", "value": 0.2641, "median": 0.2504, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.1101, "median": -0.09093, "ci": null, "n": 43, "wins": 37, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.04146, "median": 0.0304, "ci": null, "n": 43, "wins": 8, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "speed_err", "value": 1.111, "median": 0.951, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.02748, "median": 0.1154, "ci": null, "n": 43, "wins": 19, "ties": 0, "clusters": 5, "cluster_wins": 3 }, "1a421175_12864": { "delta": -0.04986, "median": -0.0766, "ci": null, "n": 43, "wins": 29, "ties": 0, "clusters": 5, "cluster_wins": 3 } } } ] }, "stock-newcase": { "n": 4, "clusters": 1, "metrics": [ { "key": "xtrack_2s", "value": 0.1067, "median": 0.08966, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.01812, "median": -0.005652, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.03985, "median": 0.03431, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "xtrack_4s", "value": 0.2991, "median": 0.3112, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.05852, "median": 0.01388, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.09219, "median": 0.05009, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "xtrack_7s", "value": 1.32, "median": 1.418, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.3154, "median": -0.2218, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.2513, "median": -0.1945, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "fail_1m", "value": 0.625, "median": 0.75, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.0, "median": 0.0, "ci": null, "n": 4, "wins": 1, "ties": 2, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.0, "median": 0.0, "ci": null, "n": 4, "wins": 1, "ties": 2, "clusters": 1, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.2295, "median": 0.1619, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.5102, "median": -0.363, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.05352, "median": -0.05042, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "long_jerk", "value": 0.1785, "median": 0.1881, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.2107, "median": -0.1409, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.02734, "median": 0.02767, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 0.7772, "median": 0.8136, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.09834, "median": 0.03617, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.1857, "median": -0.1623, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 } } } ] }, "v13-8mm": { "n": 18, "clusters": 3, "metrics": [ { "key": "xtrack_2s", "value": 0.1561, "median": 0.07642, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.0612, "median": -0.04196, "ci": null, "n": 18, "wins": 13, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.003801, "median": 0.001718, "ci": null, "n": 18, "wins": 8, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "xtrack_4s", "value": 0.4326, "median": 0.2213, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.3695, "median": -0.06066, "ci": null, "n": 18, "wins": 12, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.04321, "median": -0.01284, "ci": null, "n": 18, "wins": 11, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "xtrack_7s", "value": 1.246, "median": 0.8466, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -1.379, "median": -0.009511, "ci": null, "n": 18, "wins": 9, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.09871, "median": -0.04807, "ci": null, "n": 18, "wins": 13, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "fail_1m", "value": 0.3056, "median": 0.0, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.08333, "median": 0.0, "ci": null, "n": 18, "wins": 3, "ties": 15, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.08333, "median": 0.0, "ci": null, "n": 18, "wins": 2, "ties": 16, "clusters": 3, "cluster_wins": 1 } } }, { "key": "lat_jerk", "value": 0.1828, "median": 0.1483, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1966, "median": -0.1584, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.01369, "median": 0.01239, "ci": null, "n": 18, "wins": 4, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "long_jerk", "value": 0.1768, "median": 0.151, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1278, "median": -0.1106, "ci": null, "n": 18, "wins": 17, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.03447, "median": 0.02699, "ci": null, "n": 18, "wins": 3, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 0.7254, "median": 0.7099, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.2192, "median": -0.1441, "ci": null, "n": 18, "wins": 13, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0684, "median": 0.08668, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 2 } } } ] }, "v14-8mm": { "n": 19, "clusters": 2, "metrics": [ { "key": "xtrack_2s", "value": 0.1269, "median": 0.06424, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.02407, "median": -0.01564, "ci": null, "n": 19, "wins": 13, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.01282, "median": 0.0118, "ci": null, "n": 19, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "xtrack_4s", "value": 0.5222, "median": 0.3717, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.006406, "median": -0.002339, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.03898, "median": -0.02591, "ci": null, "n": 19, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "xtrack_7s", "value": 2.34, "median": 0.6903, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.9354, "median": -0.006728, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.09849, "median": 0.01418, "ci": null, "n": 19, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 2 } } }, { "key": "fail_1m", "value": 0.3947, "median": 0.0, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.05263, "median": 0.0, "ci": null, "n": 19, "wins": 3, "ties": 15, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.02632, "median": 0.0, "ci": null, "n": 19, "wins": 0, "ties": 18, "clusters": 2, "cluster_wins": 0 } } }, { "key": "lat_jerk", "value": 0.2355, "median": 0.2287, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.373, "median": -0.2553, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0193, "median": 0.007885, "ci": null, "n": 19, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "long_jerk", "value": 0.3118, "median": 0.2582, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.09868, "median": -0.06055, "ci": null, "n": 19, "wins": 13, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.05584, "median": 0.0485, "ci": null, "n": 19, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "speed_err", "value": 1.194, "median": 1.01, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.3994, "median": 0.3242, "ci": null, "n": 19, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.0409, "median": -0.04902, "ci": null, "n": 19, "wins": 12, "ties": 0, "clusters": 2, "cluster_wins": 2 } } } ] } }, "simulator": "g55-2026-10-31" }, "truck_recovery": { "status": "ok", "unit_of_evidence": "window", "n": 100, "clusters": 14, "condition": "Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 98 windows (59 day, 39 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 98 windows (59 day, 39 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).", "baselines": [ "lebowski_877m", "1a421175_12864" ], "metrics": [ { "key": "heading_xtrack_2s", "label": "2° off heading: distance from the path at 2 s", "unit": "m", "better": "lower", "value": 0.5096, "median": 0.4215, "ci": [ 0.4138, 0.6267 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.05672, "median": 0.04114, "ci": [ 0.01687, 0.1023 ], "n": 98, "wins": 39, "ties": 0, "clusters": 13, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.008418, "median": 0.009399, "ci": [ 0.0005946, 0.01522 ], "n": 98, "wins": 35, "ties": 0, "clusters": 13, "cluster_wins": 3 } } }, { "key": "heading_xtrack_4s", "label": "2° off heading: distance from the path at 4 s", "unit": "m", "better": "lower", "value": 0.7591, "median": 0.5233, "ci": [ 0.6033, 0.9279 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.05661, "median": 0.07989, "ci": [ -0.05954, 0.1407 ], "n": 98, "wins": 34, "ties": 0, "clusters": 13, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.009421, "median": -0.001567, "ci": [ -0.05311, 0.05553 ], "n": 98, "wins": 49, "ties": 0, "clusters": 13, "cluster_wins": 6 } } }, { "key": "heading_lat_jerk", "label": "2° off heading: requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.3567, "median": 0.3205, "ci": [ 0.3233, 0.386 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.5302, "median": -0.4503, "ci": [ -0.5896, -0.4676 ], "n": 98, "wins": 98, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "1a421175_12864": { "delta": 0.003871, "median": 0.01321, "ci": [ -0.006508, 0.01718 ], "n": 98, "wins": 34, "ties": 0, "clusters": 13, "cluster_wins": 6 } } }, { "key": "offcentre_xtrack_2s", "label": "0.5 m off centre: distance from the path at 2 s", "unit": "m", "better": "lower", "value": 0.2825, "median": 0.2433, "ci": [ 0.2519, 0.3102 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.01553, "median": 0.02913, "ci": [ -0.004051, 0.03667 ], "n": 98, "wins": 38, "ties": 0, "clusters": 13, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.01341, "median": 0.007843, "ci": [ 0.006615, 0.02123 ], "n": 98, "wins": 29, "ties": 0, "clusters": 13, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_4s", "label": "0.5 m off centre: distance from the path at 4 s", "unit": "m", "better": "lower", "value": 0.7022, "median": 0.4651, "ci": [ 0.5732, 0.8258 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": 0.0393, "median": 0.06119, "ci": [ -0.06496, 0.1169 ], "n": 98, "wins": 38, "ties": 0, "clusters": 13, "cluster_wins": 4 }, "1a421175_12864": { "delta": 0.05129, "median": 0.03196, "ci": [ 0.008357, 0.09267 ], "n": 98, "wins": 29, "ties": 0, "clusters": 13, "cluster_wins": 4 } } }, { "key": "offcentre_lat_jerk", "label": "0.5 m off centre: requested lateral jerk, RMS", "unit": "m/s³", "better": "lower", "value": 0.4533, "median": 0.422, "ci": [ 0.4288, 0.4771 ], "n": 98, "clusters": 13, "paired": { "lebowski_877m": { "delta": -0.6571, "median": -0.5979, "ci": [ -0.718, -0.5882 ], "n": 98, "wins": 98, "ties": 0, "clusters": 13, "cluster_wins": 13 }, "1a421175_12864": { "delta": 0.007706, "median": 0.008578, "ci": [ 0.002184, 0.01281 ], "n": 98, "wins": 37, "ties": 0, "clusters": 13, "cluster_wins": 2 } } }, { "key": "brake_ahead_4s", "label": "Braking lead: distance ahead of the recorded car at 4 s", "unit": "m", "better": "lower", "value": -0.03107, "median": -0.03107, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 1.306, "median": 1.306, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.1346, "median": 0.1346, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "brake_gap_min", "label": "Braking lead: smallest gap to the lead (derived)", "unit": "m", "better": "higher", "value": 22.28, "median": 22.28, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -5.176, "median": -5.176, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.3545, "median": 0.3545, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "brake_speed_err", "label": "Braking lead: speed error against the recorded speed, RMS", "unit": "m/s", "better": "lower", "value": 0.9059, "median": 0.9059, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.2976, "median": -0.2976, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.2468, "median": -0.2468, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } } ], "per_unit_file": null, "by_group": { "stock-aimedup": { "n": 14, "clusters": 2, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.3553, "median": 0.3264, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.0365, "median": -0.04475, "ci": null, "n": 14, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0003671, "median": -0.006953, "ci": null, "n": 14, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "heading_xtrack_4s", "value": 0.5561, "median": 0.4303, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.003094, "median": 0.0223, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.0396, "median": 0.00526, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "heading_lat_jerk", "value": 0.3826, "median": 0.3319, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.3854, "median": -0.2987, "ci": null, "n": 14, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.02065, "median": 0.03756, "ci": null, "n": 14, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2481, "median": 0.2258, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.01481, "median": -0.0319, "ci": null, "n": 14, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.01912, "median": -0.007302, "ci": null, "n": 14, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_4s", "value": 0.5742, "median": 0.5166, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.01562, "median": 0.07433, "ci": null, "n": 14, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.03827, "median": 0.02637, "ci": null, "n": 14, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.5274, "median": 0.5248, "ci": null, "n": 14, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.5344, "median": -0.5161, "ci": null, "n": 14, "wins": 14, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.0194, "median": 0.0142, "ci": null, "n": 14, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 } } } ] }, "stock-dual": { "n": 45, "clusters": 6, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.454, "median": 0.4006, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.03847, "median": 0.0131, "ci": null, "n": 43, "wins": 19, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.01776, "median": 0.01833, "ci": null, "n": 43, "wins": 6, "ties": 0, "clusters": 5, "cluster_wins": 0 } } }, { "key": "heading_xtrack_4s", "value": 0.7574, "median": 0.5455, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.1212, "median": 0.07873, "ci": null, "n": 43, "wins": 14, "ties": 0, "clusters": 5, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.05561, "median": 0.05298, "ci": null, "n": 43, "wins": 16, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "heading_lat_jerk", "value": 0.3339, "median": 0.2897, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.5625, "median": -0.4688, "ci": null, "n": 43, "wins": 43, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.004261, "median": 0.01399, "ci": null, "n": 43, "wins": 17, "ties": 0, "clusters": 5, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_2s", "value": 0.3127, "median": 0.2634, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.01333, "median": 0.01742, "ci": null, "n": 43, "wins": 16, "ties": 0, "clusters": 5, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.01873, "median": 0.02029, "ci": null, "n": 43, "wins": 6, "ties": 0, "clusters": 5, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.8119, "median": 0.6184, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": 0.1414, "median": 0.08726, "ci": null, "n": 43, "wins": 14, "ties": 0, "clusters": 5, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.1033, "median": 0.07174, "ci": null, "n": 43, "wins": 8, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.449, "median": 0.4162, "ci": null, "n": 43, "clusters": 5, "paired": { "lebowski_877m": { "delta": -0.7276, "median": -0.6778, "ci": null, "n": 43, "wins": 43, "ties": 0, "clusters": 5, "cluster_wins": 5 }, "1a421175_12864": { "delta": 0.007165, "median": 0.007845, "ci": null, "n": 43, "wins": 17, "ties": 0, "clusters": 5, "cluster_wins": 1 } } }, { "key": "brake_ahead_4s", "value": -0.03107, "median": -0.03107, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": 1.306, "median": 1.306, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.1346, "median": 0.1346, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "brake_gap_min", "value": 22.28, "median": 22.28, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -5.176, "median": -5.176, "ci": null, "n": 2, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.3545, "median": 0.3545, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "brake_speed_err", "value": 0.9059, "median": 0.9059, "ci": null, "n": 2, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.2976, "median": -0.2976, "ci": null, "n": 2, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.2468, "median": -0.2468, "ci": null, "n": 2, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 1 } } } ] }, "stock-newcase": { "n": 4, "clusters": 1, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.3879, "median": 0.4015, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.06264, "median": -0.08433, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.003317, "median": 0.004031, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "heading_xtrack_4s", "value": 0.5026, "median": 0.5024, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.2829, "median": 0.2519, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.1015, "median": 0.05687, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "heading_lat_jerk", "value": 0.3367, "median": 0.2942, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.6355, "median": -0.6521, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.01769, "median": -0.01101, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 } } }, { "key": "offcentre_xtrack_2s", "value": 0.282, "median": 0.2475, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.01779, "median": -0.0177, "ci": null, "n": 4, "wins": 2, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.01015, "median": 0.0104, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.8367, "median": 0.6207, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": 0.1091, "median": 0.1224, "ci": null, "n": 4, "wins": 0, "ties": 0, "clusters": 1, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.1126, "median": 0.06733, "ci": null, "n": 4, "wins": 1, "ties": 0, "clusters": 1, "cluster_wins": 0 } } }, { "key": "offcentre_lat_jerk", "value": 0.5154, "median": 0.5138, "ci": null, "n": 4, "clusters": 1, "paired": { "lebowski_877m": { "delta": -0.882, "median": -0.5638, "ci": null, "n": 4, "wins": 4, "ties": 0, "clusters": 1, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.001173, "median": -0.01479, "ci": null, "n": 4, "wins": 3, "ties": 0, "clusters": 1, "cluster_wins": 0 } } } ] }, "v13-8mm": { "n": 18, "clusters": 3, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.8943, "median": 0.8655, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.2013, "median": 0.234, "ci": null, "n": 18, "wins": 2, "ties": 0, "clusters": 3, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.001231, "median": 0.01109, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "heading_xtrack_4s", "value": 1.086, "median": 0.8043, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1458, "median": 0.0582, "ci": null, "n": 18, "wins": 8, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "1a421175_12864": { "delta": -0.147, "median": -0.04332, "ci": null, "n": 18, "wins": 16, "ties": 0, "clusters": 3, "cluster_wins": 3 } } }, { "key": "heading_lat_jerk", "value": 0.4132, "median": 0.4006, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.4598, "median": -0.411, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": -0.004793, "median": 0.003898, "ci": null, "n": 18, "wins": 7, "ties": 0, "clusters": 3, "cluster_wins": 2 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2857, "median": 0.2425, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": 0.02763, "median": 0.067, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 1 }, "1a421175_12864": { "delta": 0.005963, "median": 0.0086, "ci": null, "n": 18, "wins": 4, "ties": 0, "clusters": 3, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.5085, "median": 0.2829, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.1846, "median": -0.02004, "ci": null, "n": 18, "wins": 10, "ties": 0, "clusters": 3, "cluster_wins": 2 }, "1a421175_12864": { "delta": -0.01656, "median": 0.01464, "ci": null, "n": 18, "wins": 6, "ties": 0, "clusters": 3, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.3975, "median": 0.3784, "ci": null, "n": 18, "clusters": 3, "paired": { "lebowski_877m": { "delta": -0.4843, "median": -0.4795, "ci": null, "n": 18, "wins": 18, "ties": 0, "clusters": 3, "cluster_wins": 3 }, "1a421175_12864": { "delta": 0.001569, "median": 0.001052, "ci": null, "n": 18, "wins": 9, "ties": 0, "clusters": 3, "cluster_wins": 1 } } } ] }, "v14-8mm": { "n": 19, "clusters": 2, "metrics": [ { "key": "heading_xtrack_2s", "value": 0.4104, "median": 0.4002, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.0549, "median": 0.05149, "ci": null, "n": 19, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.00343, "median": -0.002015, "ci": null, "n": 19, "wins": 10, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "heading_xtrack_4s", "value": 0.6575, "median": 0.3783, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.09397, "median": 0.08521, "ci": null, "n": 19, "wins": 6, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.01145, "median": -0.01274, "ci": null, "n": 19, "wins": 11, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "heading_lat_jerk", "value": 0.34, "median": 0.3126, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.6084, "median": -0.5236, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.003366, "median": 0.01834, "ci": null, "n": 19, "wins": 3, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_2s", "value": 0.2369, "median": 0.2109, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.03841, "median": 0.04834, "ci": null, "n": 19, "wins": 5, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": 0.004919, "median": 0.001044, "ci": null, "n": 19, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 0 } } }, { "key": "offcentre_xtrack_4s", "value": 0.7034, "median": 0.432, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": 0.04599, "median": 0.04367, "ci": null, "n": 19, "wins": 8, "ties": 0, "clusters": 2, "cluster_wins": 0 }, "1a421175_12864": { "delta": -0.005533, "median": 0.004959, "ci": null, "n": 19, "wins": 9, "ties": 0, "clusters": 2, "cluster_wins": 1 } } }, { "key": "offcentre_lat_jerk", "value": 0.4485, "median": 0.4434, "ci": null, "n": 19, "clusters": 2, "paired": { "lebowski_877m": { "delta": -0.7041, "median": -0.6306, "ci": null, "n": 19, "wins": 19, "ties": 0, "clusters": 2, "cluster_wins": 2 }, "1a421175_12864": { "delta": 0.007506, "median": 0.0174, "ci": null, "n": 19, "wins": 4, "ties": 0, "clusters": 2, "cluster_wins": 0 } } } ] } }, "simulator": "g55-2026-10-31" } }, "stills": [], "limits": [ "Open-loop scores compare a model's action with what the recording driver or car did; a model that drives differently but well is scored as wrong. comma's RL residual makes the lateral action smoother, and earlier than its base's by 0.10 to 0.26 s (measured on the G55 and on 16 of comma's cars); both score as error here. Measured on the 2026-10-27 deep dive: open-loop accuracy did not predict closed-loop driving.", "Closed-loop numbers come from a generative world model, not a car. In a separate test on the G55's windows, worldmodel-4B followed an added yaw command through 2 s but showed 0 to 20% of an added lateral offset 3 to 4 s later; past about 4 s the recorded anchor frames also pull every model back toward the recorded path. The front page reports 2 s.", "Intervals are 95% percentile bootstraps that resample whole drives (the drive a public segment was cut from, the recorded drive a truck window comes from), so segments of one drive count once. Below 10 drives there is no interval: the braking-lead test comes from one drive and reports means and win counts only.", "Paired deltas are model minus baseline on the units both have (for lateral gain, the change in its distance from 1); wins count units where the model is better in the metric's direction.", "The lead of the lateral request is the shift, searched from 0 to 1.5 s, at which the request best correlates with the car's later lateral motion; a larger value means the request comes earlier. A value at either edge is censored and left out. Plan speed error at 3 s reads the plan head, which does not separate models built on one frozen base: it is identical on every segment for the four f22e06e5 models, and for Cinque v2 and v3.", "Truck open loop comes from few drives, and the camera setup changed between them (lens, housing, aim); the pooled number mixes setups, so read the per-setup numbers too. The 8 mm setups' wide camera recorded about 7-10 fps, so its frame can be up to ~60 ms from the narrow one. On the V14 setup up to 7 of a window's 700 steps are ~100 ms (one camera frame dropped). The camera's aim is the truck's live calibration learned on each drive; drives whose recorded calibration was carried over from another drive are left out.", "Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre. The pooled number mixes camera setups, so read the per-setup numbers too. On the V14 setup the wide frame can be up to 90 ms from the simulated one (its camera recorded 7-9 fps).", "Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test has 2 windows from one drive, whose recorded calibration was carried over from an earlier drive." ] }