Repository navigation
Android / SM8650 (Snapdragon 8 Gen 3): htp-whole meets the frame budget but fails verify_parity (6 slices) #18
Description
Since nothing QNN had run on a Snapdragon when OrtProfile was written, here are parity results from an SM8650 (Snapdragon 8 Gen 3, HTP v75) with the stock processors. Related: #17 / #16, which cover SM8750 only.
Short version: htp-whole is fast enough (32.8 ms mean, 0 frames over 50 ms in 10 minutes) but fails verify_parity on 6 slices. htp (NPU + GPU) is about 115 ms. On this chip, neither stock processor is usable today.
Setup
| Host | Lenovo Legion Tab Gen 3 (2025), Snapdragon 8 Gen 3 (SM8650), 12 GB |
| Jetlink app | 0.8.7 (release APK) |
Server (from hello) |
ort 1.29.0 on htp-whole-SM8650 |
| comma | comma 4, AGNOS 19.8.1, StarPilot with vendored jetlink at d24328d |
| Link | USB-C to USB-C, comma UDC super-speed. The tablet has one USB 3.2 port and one USB 2.0 port; the USB 2.0 port links at high-speed. |
| Model | 404a18cfd86d29637d20c697dfde245bb47c666ae016730ab674c65f4d1e1aa4, 766354845 bytes, checkpoint b9facbcc-4d47-410e-b3ce-dfcbad12ba92/56320/f78ed37d-afad-4dbc-8050-40ea885eedde/12864 |
Benchmarks (in-app, 10 minutes)
| Processor | Frames | Mean | p99 | Max | Over 35 ms | Over 50 ms | Thermal |
|---|---|---|---|---|---|---|---|
htp (NPU + GPU) |
5198 | 115.2 ms | 122.7 ms | 127.0 ms | 5198 | 5198 | serious → serious |
htp-whole (NPU) |
12001 | 32.8 ms | 34.2 ms | 36.3 ms | 12 | 0 | nominal → serious at ~380 s, no change in frame time |
For htp, the accelerator accounts for nearly all of the frame time (accelerator mean 114.5 ms, queues 0.4 ms). That run started already thermally serious, so it is somewhat pessimistic. It is still more than 2x over budget.
Parity: htp-whole
scripts/verify_parity.py at d24328d, default --n 32 --seed 0:
capture --ffson the comma, with openpilot stopped so jetlinkd released the gadget. All 32 frames returned 18452 finite values.reference --onnxon the same ONNX (sha256 matches) with onnxruntime 1.31.0 CPU, Windows x86-64.compare: FAIL.
pooled over 32 frames, per slice and per column:
action corr 0.999829 by error, max abs 0.1582 <= 0.5352 max abs 0.1582 mean abs 0.02413
desire_pred corr 0.999692 (compared whole) max abs 0.2578 mean abs 0.07463
desire_state corr 0.999632 by error, max abs 1.391 <= 2.756 max abs 1.3906 mean abs 0.28295
hidden_state corr 0.999663 (compared whole) max abs 0.8594 mean abs 0.03253
lane_lines corr 0.999591 worst col 0.998911 mu[1] cols: mu[1] 0.998911 max abs 0.6445 mean abs 0.03293 <-- FAIL
lane_lines_prob corr 0.999274 by error, max abs 0.3633 <= 0.7691 max abs 0.3633 mean abs 0.10911
lead corr 0.999832 worst col 0.998609 mu[2] (4 by error) cols: mu[2] 0.998609 max abs 8.8750 mean abs 0.22762 <-- FAIL
lead_prob corr 0.995908 by error, max abs 0.4199 <= 0.5047 max abs 0.4199 mean abs 0.12956
meta corr 0.998456 (compared whole) max abs 0.4443 mean abs 0.12738 <-- FAIL
pad corr 1.000000 by error, max abs 0 <= 0 max abs 0.0000 mean abs 0.00000
plan corr 0.999782 worst col 0.998334 std[5] (11 by error) cols: std[5] 0.998334, std[6] 0.998372, std[8] 0.998817, std[12] 0.998622, std[13] 0.998600 max abs 4.2500 mean abs 0.04139 <-- FAIL
pose corr 0.999812 no column correlated (12 by error) cols: mu[1] by error, max abs 0.004059 > 0.003333 max abs 0.3828 mean abs 0.03753 <-- FAIL
road_edges corr 0.999516 worst col 0.998731 mu[1] cols: mu[1] 0.998731 max abs 0.7344 mean abs 0.05318 <-- FAIL
road_transform corr 0.999990 no column correlated (12 by error) max abs 0.0898 mean abs 0.01746
wide_from_device_euler corr 0.999972 no column correlated (6 by error) max abs 0.0762 mean abs 0.01479
FAIL: 6 slice(s) below corr 0.999 or past their error bound, whole or in a column: meta 0.997357 on one frame, lane_lines 0.998604 on one frame, road_edges 0.998891 on one frame, lead pooled, plan pooled, pose pooled
Error does not accumulate through the recurrent state. Here is mean |link − ref| / mean |ref| per slice, by frame:
| Frame | hidden_state | plan | lead | lane_lines | meta | action | pose |
|---|---|---|---|---|---|---|---|
| 0 | 0.0199 | 0.0090 | 0.0274 | 0.0156 | 0.0425 | 0.0022 | 0.0090 |
| 5 | 0.0249 | 0.0142 | 0.0434 | 0.0157 | 0.0488 | 0.0029 | 0.0125 |
| 10 | 0.0221 | 0.0134 | 0.0116 | 0.0158 | 0.0396 | 0.0047 | 0.0169 |
| 15 | 0.0276 | 0.0286 | 0.0260 | 0.0219 | 0.0459 | 0.0372 | 0.0173 |
| 20 | 0.0248 | 0.0120 | 0.0318 | 0.0156 | 0.0382 | 0.0103 | 0.0127 |
| 25 | 0.0272 | 0.0160 | 0.0210 | 0.0198 | 0.0333 | 0.0225 | 0.0102 |
| 31 | 0.0325 | 0.0314 | 0.0265 | 0.0312 | 0.0432 | 0.0395 | 0.0424 |
The error is about the same size on frame 0, right after the reset, as on frame 31. hidden_state is already about 2% off on frame 0. That reads as per-frame fp16 error on HTP throughout the graph, not only in the heads that headsInFP32 moves to fp32 (which the NPU runs in fp16 anyway, as OrtProfile.htpWhole notes). action passes; the failing slices are the vision heads, the plan's std columns, and pose.
One caveat: the reference used onnxruntime 1.31.0 on the CPU, while the server runs 1.29.0. I haven't measured the CPU-to-CPU noise floor between those two versions.
The full compare.log is below. I can share the captured out/ directory (inputs plus link and reference outputs, 17 MB) if that helps.
Questions
- Is the "range-safe NPU only" preparation from Android: opt-in causal NPU/GPU execution with generic QNN discovery #16 (33.8 ms on SM8750) expected to pass parity? Could it apply to HTP v75?
- Would a split that keeps the precision-sensitive part off the HTP be the right direction for SM8650? For example: the policy, or the heads, in fp32 on the CPU or GPU, with the trunk on HTP.
I'm happy to test builds or preparations on this tablet: benchmark, parity capture over --ffs, and the comma live bench.
Full compare.log
frame 0: corr 0.999807 max abs 1.5625
action corr 0.999997 max abs 0.0117 mean abs 0.00463 (by error, pooled)
desire_pred corr 0.999484 max abs 0.1875 mean abs 0.09339
desire_state corr 0.998518 max abs 1.3906 mean abs 0.45166 (by error, pooled)
hidden_state corr 0.999811 max abs 0.3125 mean abs 0.02495
lane_lines corr 0.999865 max abs 0.0918 mean abs 0.02450
lane_lines_prob corr 0.999709 max abs 0.1445 mean abs 0.07352 (by error, pooled)
lead corr 0.999986 max abs 1.5625 mean abs 0.17256
lead_prob corr 0.994808 max abs 0.2358 mean abs 0.18807 (by error, pooled)
meta corr 0.998533 max abs 0.3301 mean abs 0.12621 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999982 max abs 0.1250 mean abs 0.02292
pose corr 0.999980 max abs 0.0508 mean abs 0.01643 (by error, pooled)
road_edges corr 0.999743 max abs 0.1973 mean abs 0.04430
road_transform corr 0.999999 max abs 0.0469 mean abs 0.01941 (by error, pooled)
wide_from_device_euler corr 0.999999 max abs 0.0430 mean abs 0.01150 (by error, pooled)
frame 1: corr 0.999790 max abs 1.3750
action corr 0.999999 max abs 0.0234 mean abs 0.01208 (by error, pooled)
desire_pred corr 0.999454 max abs 0.2266 mean abs 0.09628
desire_state corr 0.998741 max abs 1.3438 mean abs 0.45557 (by error, pooled)
hidden_state corr 0.999798 max abs 0.3125 mean abs 0.02420
lane_lines corr 0.999856 max abs 0.1074 mean abs 0.02286
lane_lines_prob corr 0.999905 max abs 0.0977 mean abs 0.05914 (by error, pooled)
lead corr 0.999984 max abs 1.3750 mean abs 0.15827
lead_prob corr 0.996461 max abs 0.1978 mean abs 0.16447 (by error, pooled)
meta corr 0.998793 max abs 0.3027 mean abs 0.11605 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999951 max abs 0.4063 mean abs 0.03202
pose corr 0.999987 max abs 0.0625 mean abs 0.02182 (by error, pooled)
road_edges corr 0.999758 max abs 0.1953 mean abs 0.04051
road_transform corr 1.000000 max abs 0.0430 mean abs 0.01358 (by error, pooled)
wide_from_device_euler corr 0.999992 max abs 0.0391 mean abs 0.00978 (by error, pooled)
frame 2: corr 0.999724 max abs 1.6250
action corr 0.999998 max abs 0.0391 mean abs 0.01902 (by error, pooled)
desire_pred corr 0.999403 max abs 0.2578 mean abs 0.10498
desire_state corr 0.998798 max abs 1.3828 mean abs 0.45557 (by error, pooled)
hidden_state corr 0.999752 max abs 0.4375 mean abs 0.02604
lane_lines corr 0.999844 max abs 0.1094 mean abs 0.02294
lane_lines_prob corr 0.999839 max abs 0.1758 mean abs 0.08576 (by error, pooled)
lead corr 0.999988 max abs 1.6250 mean abs 0.17661
lead_prob corr 0.997530 max abs 0.1836 mean abs 0.14640 (by error, pooled)
meta corr 0.998590 max abs 0.3652 mean abs 0.12332 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999838 max abs 0.9063 mean abs 0.04167
pose corr 0.999971 max abs 0.0801 mean abs 0.03518 (by error, pooled)
road_edges corr 0.999799 max abs 0.1621 mean abs 0.03571
road_transform corr 0.999999 max abs 0.0352 mean abs 0.01164 (by error, pooled)
wide_from_device_euler corr 0.999980 max abs 0.0312 mean abs 0.01206 (by error, pooled)
frame 3: corr 0.999478 max abs 3.7188
action corr 0.999981 max abs 0.0664 mean abs 0.03116 (by error, pooled)
desire_pred corr 0.999825 max abs 0.1523 mean abs 0.07584
desire_state corr 0.998920 max abs 1.3125 mean abs 0.37207 (by error, pooled)
hidden_state corr 0.999721 max abs 0.3477 mean abs 0.03063
lane_lines corr 0.999911 max abs 0.0625 mean abs 0.02082
lane_lines_prob corr 0.999039 max abs 0.2891 mean abs 0.10571 (by error, pooled)
lead corr 0.999980 max abs 3.7188 mean abs 0.35846
lead_prob corr 0.997475 max abs 0.2744 mean abs 0.21785 (by error, pooled)
meta corr 0.998958 max abs 0.3115 mean abs 0.09597 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999741 max abs 1.7500 mean abs 0.05742
pose corr 0.999863 max abs 0.0967 mean abs 0.04519 (by error, pooled)
road_edges corr 0.999871 max abs 0.1211 mean abs 0.03435
road_transform corr 0.999998 max abs 0.0391 mean abs 0.01245 (by error, pooled)
wide_from_device_euler corr 0.999997 max abs 0.0312 mean abs 0.00718 (by error, pooled)
frame 4: corr 0.999304 max abs 4.8125
action corr 0.999885 max abs 0.1172 mean abs 0.04885 (by error, pooled)
desire_pred corr 0.999845 max abs 0.1367 mean abs 0.06833
desire_state corr 0.999606 max abs 0.6641 mean abs 0.27158 (by error, pooled)
hidden_state corr 0.999587 max abs 0.7031 mean abs 0.04100
lane_lines corr 0.999528 max abs 0.3672 mean abs 0.03939
lane_lines_prob corr 0.999302 max abs 0.2227 mean abs 0.12020 (by error, pooled)
lead corr 0.999985 max abs 4.8125 mean abs 0.44870
lead_prob corr 0.997381 max abs 0.2651 mean abs 0.23161 (by error, pooled)
meta corr 0.998465 max abs 0.3613 mean abs 0.10691 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999582 max abs 2.6875 mean abs 0.05946
pose corr 0.999659 max abs 0.1875 mean abs 0.05904 (by error, pooled)
road_edges corr 0.999556 max abs 0.3828 mean abs 0.05419
road_transform corr 0.999990 max abs 0.0586 mean abs 0.01609 (by error, pooled)
wide_from_device_euler corr 0.999995 max abs 0.0293 mean abs 0.00954 (by error, pooled)
frame 5: corr 0.999626 max abs 3.0625
action corr 0.999995 max abs 0.0195 mean abs 0.00624 (by error, pooled)
desire_pred corr 0.999765 max abs 0.1270 mean abs 0.05799
desire_state corr 0.999664 max abs 0.4688 mean abs 0.20850 (by error, pooled)
hidden_state corr 0.999703 max abs 0.3125 mean abs 0.03254
lane_lines corr 0.999842 max abs 0.1543 mean abs 0.02600
lane_lines_prob corr 0.999836 max abs 0.1914 mean abs 0.11325 (by error, pooled)
lead corr 0.999992 max abs 3.0625 mean abs 0.30574
lead_prob corr 0.998594 max abs 0.1943 mean abs 0.17969 (by error, pooled)
meta corr 0.998828 max abs 0.3750 mean abs 0.14798 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999844 max abs 1.5938 mean abs 0.03376
pose corr 0.999951 max abs 0.0625 mean abs 0.02352 (by error, pooled)
road_edges corr 0.999691 max abs 0.1914 mean abs 0.04245
road_transform corr 0.999996 max abs 0.0625 mean abs 0.02284 (by error, pooled)
wide_from_device_euler corr 0.999994 max abs 0.0508 mean abs 0.02138 (by error, pooled)
frame 6: corr 0.999532 max abs 8.8750
action corr 0.999992 max abs 0.0723 mean abs 0.02897 (by error, pooled)
desire_pred corr 0.999479 max abs 0.2090 mean abs 0.09460
desire_state corr 0.999688 max abs 0.4121 mean abs 0.22192 (by error, pooled)
hidden_state corr 0.999669 max abs 0.6445 mean abs 0.03535
lane_lines corr 0.999768 max abs 0.1914 mean abs 0.03391
lane_lines_prob corr 0.999758 max abs 0.1805 mean abs 0.08341 (by error, pooled)
lead corr 0.999986 max abs 8.8750 mean abs 0.83468
lead_prob corr 0.977282 max abs 0.3770 mean abs 0.33203 (by error, pooled)
meta corr 0.997410 max abs 0.4160 mean abs 0.14886 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999815 max abs 4.2500 mean abs 0.08178
pose corr 0.999892 max abs 0.3359 mean abs 0.04508 (by error, pooled)
road_edges corr 0.999414 max abs 0.3281 mean abs 0.06402
road_transform corr 0.999990 max abs 0.0859 mean abs 0.02958 (by error, pooled)
wide_from_device_euler corr 0.999980 max abs 0.0762 mean abs 0.03147 (by error, pooled)
frame 7: corr 0.999776 max abs 8.1250
action corr 0.999989 max abs 0.0430 mean abs 0.01198 (by error, pooled)
desire_pred corr 0.999613 max abs 0.2187 mean abs 0.07919
desire_state corr 0.999629 max abs 0.5273 mean abs 0.26270 (by error, pooled)
hidden_state corr 0.999645 max abs 0.6875 mean abs 0.03684
lane_lines corr 0.999813 max abs 0.1338 mean abs 0.03001
lane_lines_prob corr 0.999532 max abs 0.2227 mean abs 0.12592 (by error, pooled)
lead corr 0.999997 max abs 8.1250 mean abs 0.79229
lead_prob corr 0.970594 max abs 0.4199 mean abs 0.36849 (by error, pooled)
meta corr 0.998731 max abs 0.2821 mean abs 0.10533 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999920 max abs 3.8750 mean abs 0.06348
pose corr 0.999979 max abs 0.1328 mean abs 0.02578 (by error, pooled)
road_edges corr 0.999596 max abs 0.1992 mean abs 0.06030
road_transform corr 0.999993 max abs 0.0859 mean abs 0.03332 (by error, pooled)
wide_from_device_euler corr 0.999992 max abs 0.0703 mean abs 0.02136 (by error, pooled)
frame 8: corr 0.999886 max abs 2.1875
action corr 0.999970 max abs 0.0273 mean abs 0.00832 (by error, pooled)
desire_pred corr 0.999801 max abs 0.1230 mean abs 0.04537
desire_state corr 0.999894 max abs 0.5937 mean abs 0.21094 (by error, pooled)
hidden_state corr 0.999771 max abs 0.3164 mean abs 0.02961
lane_lines corr 0.999842 max abs 0.0974 mean abs 0.02578
lane_lines_prob corr 0.999946 max abs 0.2461 mean abs 0.11386 (by error, pooled)
lead corr 0.999991 max abs 2.1875 mean abs 0.17007
lead_prob corr 0.930878 max abs 0.1274 mean abs 0.07536 (by error, pooled)
meta corr 0.998173 max abs 0.3369 mean abs 0.14547 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999969 max abs 1.6875 mean abs 0.03012
pose corr 0.999922 max abs 0.1641 mean abs 0.02882 (by error, pooled)
road_edges corr 0.999570 max abs 0.2129 mean abs 0.04895
road_transform corr 0.999995 max abs 0.0508 mean abs 0.02126 (by error, pooled)
wide_from_device_euler corr 0.999998 max abs 0.0469 mean abs 0.01186 (by error, pooled)
frame 9: corr 0.999861 max abs 1.3750
action corr 0.999999 max abs 0.0078 mean abs 0.00230 (by error, pooled)
desire_pred corr 0.999820 max abs 0.1768 mean abs 0.04774
desire_state corr 0.999791 max abs 0.7812 mean abs 0.38574 (by error, pooled)
hidden_state corr 0.999773 max abs 0.4297 mean abs 0.02577
lane_lines corr 0.999846 max abs 0.1094 mean abs 0.02377
lane_lines_prob corr 0.999594 max abs 0.3047 mean abs 0.09747 (by error, pooled)
lead corr 0.999999 max abs 1.3750 mean abs 0.12986
lead_prob corr 0.981962 max abs 0.1582 mean abs 0.07425 (by error, pooled)
meta corr 0.998524 max abs 0.3652 mean abs 0.14773 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999959 max abs 0.5938 mean abs 0.02857
pose corr 0.999987 max abs 0.0273 mean abs 0.01202 (by error, pooled)
road_edges corr 0.999407 max abs 0.2813 mean abs 0.06400
road_transform corr 0.999994 max abs 0.0430 mean abs 0.01388 (by error, pooled)
wide_from_device_euler corr 0.999996 max abs 0.0273 mean abs 0.00533 (by error, pooled)
frame 10: corr 0.999821 max abs 1.4688
action corr 0.999992 max abs 0.0293 mean abs 0.01048 (by error, pooled)
desire_pred corr 0.999735 max abs 0.1660 mean abs 0.08437
desire_state corr 0.999921 max abs 0.5078 mean abs 0.22729 (by error, pooled)
hidden_state corr 0.999761 max abs 0.3750 mean abs 0.02427
lane_lines corr 0.999842 max abs 0.1074 mean abs 0.02594
lane_lines_prob corr 0.999805 max abs 0.2148 mean abs 0.09348 (by error, pooled)
lead corr 0.999983 max abs 1.0000 mean abs 0.08077
lead_prob corr 0.998575 max abs 0.0947 mean abs 0.04671 (by error, pooled)
meta corr 0.999025 max abs 0.3418 mean abs 0.12560
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999711 max abs 1.4688 mean abs 0.02988
pose corr 0.999950 max abs 0.1009 mean abs 0.04125 (by error, pooled)
road_edges corr 0.999752 max abs 0.2109 mean abs 0.04100
road_transform corr 0.999998 max abs 0.0273 mean abs 0.00924 (by error, pooled)
wide_from_device_euler corr 0.999951 max abs 0.0547 mean abs 0.01961 (by error, pooled)
frame 11: corr 0.999702 max abs 2.1250
action corr 0.999973 max abs 0.0605 mean abs 0.02378 (by error, pooled)
desire_pred corr 0.999793 max abs 0.1528 mean abs 0.07362
desire_state corr 0.999902 max abs 0.5547 mean abs 0.26654 (by error, pooled)
hidden_state corr 0.999646 max abs 0.4219 mean abs 0.02950
lane_lines corr 0.999834 max abs 0.1465 mean abs 0.02596
lane_lines_prob corr 0.998580 max abs 0.3633 mean abs 0.12680 (by error, pooled)
lead corr 0.999949 max abs 2.1250 mean abs 0.16247
lead_prob corr 0.998580 max abs 0.0791 mean abs 0.04373 (by error, pooled)
meta corr 0.999093 max abs 0.3311 mean abs 0.11911
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999533 max abs 2.1094 mean abs 0.03767
pose corr 0.999976 max abs 0.0820 mean abs 0.03799 (by error, pooled)
road_edges corr 0.999675 max abs 0.2461 mean abs 0.04368
road_transform corr 0.999999 max abs 0.0273 mean abs 0.01106 (by error, pooled)
wide_from_device_euler corr 0.999969 max abs 0.0352 mean abs 0.01532 (by error, pooled)
frame 12: corr 0.999658 max abs 3.2500
action corr 0.999803 max abs 0.1152 mean abs 0.05089 (by error, pooled)
desire_pred corr 0.999863 max abs 0.1660 mean abs 0.07404
desire_state corr 0.999876 max abs 0.5781 mean abs 0.17187 (by error, pooled)
hidden_state corr 0.999627 max abs 0.7266 mean abs 0.03577
lane_lines corr 0.999243 max abs 0.4297 mean abs 0.04876
lane_lines_prob corr 0.999428 max abs 0.2783 mean abs 0.11304 (by error, pooled)
lead corr 0.999979 max abs 3.2500 mean abs 0.25526
lead_prob corr 0.984468 max abs 0.1584 mean abs 0.10734 (by error, pooled)
meta corr 0.997647 max abs 0.4238 mean abs 0.12199 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999802 max abs 2.5938 mean abs 0.05649
pose corr 0.999831 max abs 0.1143 mean abs 0.05289 (by error, pooled)
road_edges corr 0.999379 max abs 0.5234 mean abs 0.05827
road_transform corr 0.999998 max abs 0.0430 mean abs 0.01474 (by error, pooled)
wide_from_device_euler corr 0.999988 max abs 0.0391 mean abs 0.00947 (by error, pooled)
frame 13: corr 0.999510 max abs 3.6875
action corr 0.999529 max abs 0.1582 mean abs 0.06515 (by error, pooled)
desire_pred corr 0.999780 max abs 0.1445 mean abs 0.08328
desire_state corr 0.999858 max abs 0.5078 mean abs 0.22705 (by error, pooled)
hidden_state corr 0.999514 max abs 0.7188 mean abs 0.04419
lane_lines corr 0.999332 max abs 0.4688 mean abs 0.04365
lane_lines_prob corr 0.999450 max abs 0.2461 mean abs 0.10209 (by error, pooled)
lead corr 0.999979 max abs 3.6875 mean abs 0.30491
lead_prob corr 0.971306 max abs 0.1738 mean abs 0.11979 (by error, pooled)
meta corr 0.998040 max abs 0.3965 mean abs 0.11439 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999678 max abs 2.7813 mean abs 0.06902
pose corr 0.999361 max abs 0.3125 mean abs 0.07226 (by error, pooled)
road_edges corr 0.999433 max abs 0.5000 mean abs 0.05548
road_transform corr 0.999994 max abs 0.0547 mean abs 0.01545 (by error, pooled)
wide_from_device_euler corr 0.999995 max abs 0.0332 mean abs 0.01115 (by error, pooled)
frame 14: corr 0.999698 max abs 2.3125
action corr 0.999961 max abs 0.0566 mean abs 0.02528 (by error, pooled)
desire_pred corr 0.999744 max abs 0.1353 mean abs 0.06373
desire_state corr 0.999919 max abs 0.3281 mean abs 0.14307 (by error, pooled)
hidden_state corr 0.999695 max abs 0.2871 mean abs 0.03280
lane_lines corr 0.999702 max abs 0.2422 mean abs 0.03271
lane_lines_prob corr 0.999816 max abs 0.2187 mean abs 0.11813 (by error, pooled)
lead corr 0.999976 max abs 2.3125 mean abs 0.18198
lead_prob corr 0.994836 max abs 0.0732 mean abs 0.04053 (by error, pooled)
meta corr 0.998700 max abs 0.3965 mean abs 0.15353 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999784 max abs 2.0000 mean abs 0.03853
pose corr 0.999914 max abs 0.0762 mean abs 0.03094 (by error, pooled)
road_edges corr 0.999541 max abs 0.3027 mean abs 0.05065
road_transform corr 0.999998 max abs 0.0508 mean abs 0.01892 (by error, pooled)
wide_from_device_euler corr 0.999995 max abs 0.0430 mean abs 0.01907 (by error, pooled)
frame 15: corr 0.999724 max abs 3.5625
action corr 0.999750 max abs 0.1074 mean abs 0.04381 (by error, pooled)
desire_pred corr 0.999395 max abs 0.2119 mean abs 0.10202
desire_state corr 0.999893 max abs 0.9219 mean abs 0.34961 (by error, pooled)
hidden_state corr 0.999605 max abs 0.8477 mean abs 0.03833
lane_lines corr 0.999760 max abs 0.1836 mean abs 0.03507
lane_lines_prob corr 0.999359 max abs 0.1914 mean abs 0.12387 (by error, pooled)
lead corr 0.999931 max abs 3.5625 mean abs 0.29390
lead_prob corr 0.999688 max abs 0.1377 mean abs 0.12537 (by error, pooled)
meta corr 0.997540 max abs 0.3901 mean abs 0.14428 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999841 max abs 2.5938 mean abs 0.07044
pose corr 0.999910 max abs 0.3828 mean abs 0.04697 (by error, pooled)
road_edges corr 0.999449 max abs 0.3047 mean abs 0.06328
road_transform corr 0.999987 max abs 0.0859 mean abs 0.02715 (by error, pooled)
wide_from_device_euler corr 0.999984 max abs 0.0703 mean abs 0.02755 (by error, pooled)
frame 16: corr 0.999866 max abs 2.0000
action corr 0.999840 max abs 0.0361 mean abs 0.01204 (by error, pooled)
desire_pred corr 0.999744 max abs 0.1758 mean abs 0.06519
desire_state corr 0.999623 max abs 0.8047 mean abs 0.25989 (by error, pooled)
hidden_state corr 0.999655 max abs 0.5820 mean abs 0.03665
lane_lines corr 0.999821 max abs 0.1299 mean abs 0.03112
lane_lines_prob corr 0.999219 max abs 0.3047 mean abs 0.16312 (by error, pooled)
lead corr 0.999995 max abs 2.0000 mean abs 0.14950
lead_prob corr 0.999857 max abs 0.0488 mean abs 0.02930 (by error, pooled)
meta corr 0.998390 max abs 0.3034 mean abs 0.11686 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999968 max abs 1.7188 mean abs 0.03907
pose corr 0.999984 max abs 0.0781 mean abs 0.01979 (by error, pooled)
road_edges corr 0.999633 max abs 0.1719 mean abs 0.05436
road_transform corr 0.999995 max abs 0.0781 mean abs 0.03081 (by error, pooled)
wide_from_device_euler corr 0.999988 max abs 0.0703 mean abs 0.01875 (by error, pooled)
frame 17: corr 0.999783 max abs 2.3125
action corr 0.999958 max abs 0.0273 mean abs 0.01268 (by error, pooled)
desire_pred corr 0.999747 max abs 0.1543 mean abs 0.04861
desire_state corr 0.999761 max abs 0.5703 mean abs 0.20215 (by error, pooled)
hidden_state corr 0.999764 max abs 0.3555 mean abs 0.02952
lane_lines corr 0.999827 max abs 0.0924 mean abs 0.02692
lane_lines_prob corr 0.999949 max abs 0.2227 mean abs 0.11336 (by error, pooled)
lead corr 0.999975 max abs 2.3125 mean abs 0.16377
lead_prob corr 0.996948 max abs 0.0693 mean abs 0.03548 (by error, pooled)
meta corr 0.998046 max abs 0.3418 mean abs 0.15659 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999852 max abs 1.7188 mean abs 0.03611
pose corr 0.999901 max abs 0.1543 mean abs 0.02607 (by error, pooled)
road_edges corr 0.999535 max abs 0.2129 mean abs 0.05152
road_transform corr 0.999994 max abs 0.0508 mean abs 0.02061 (by error, pooled)
wide_from_device_euler corr 0.999997 max abs 0.0469 mean abs 0.01151 (by error, pooled)
frame 18: corr 0.999848 max abs 0.7031
action corr 0.999997 max abs 0.0078 mean abs 0.00351 (by error, pooled)
desire_pred corr 0.999725 max abs 0.2070 mean abs 0.06620
desire_state corr 0.999719 max abs 0.7031 mean abs 0.41357 (by error, pooled)
hidden_state corr 0.999765 max abs 0.4326 mean abs 0.02540
lane_lines corr 0.999840 max abs 0.1001 mean abs 0.02448
lane_lines_prob corr 0.999901 max abs 0.2852 mean abs 0.10498 (by error, pooled)
lead corr 0.999999 max abs 0.6250 mean abs 0.05278
lead_prob corr 0.995969 max abs 0.1787 mean abs 0.09928 (by error, pooled)
meta corr 0.998658 max abs 0.3633 mean abs 0.15510 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999984 max abs 0.4688 mean abs 0.01439
pose corr 0.999987 max abs 0.0391 mean abs 0.01291 (by error, pooled)
road_edges corr 0.999388 max abs 0.2891 mean abs 0.06262
road_transform corr 0.999997 max abs 0.0352 mean abs 0.01013 (by error, pooled)
wide_from_device_euler corr 0.999995 max abs 0.0195 mean abs 0.00633 (by error, pooled)
frame 19: corr 0.999808 max abs 1.3438
action corr 0.999989 max abs 0.0273 mean abs 0.01361 (by error, pooled)
desire_pred corr 0.999710 max abs 0.1790 mean abs 0.08781
desire_state corr 0.999903 max abs 0.5859 mean abs 0.28741 (by error, pooled)
hidden_state corr 0.999775 max abs 0.3574 mean abs 0.02342
lane_lines corr 0.999841 max abs 0.1113 mean abs 0.02599
lane_lines_prob corr 0.999905 max abs 0.1914 mean abs 0.08066 (by error, pooled)
lead corr 0.999992 max abs 1.3438 mean abs 0.11758
lead_prob corr 0.999681 max abs 0.0576 mean abs 0.02197 (by error, pooled)
meta corr 0.999032 max abs 0.3398 mean abs 0.12845
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999923 max abs 0.9063 mean abs 0.01982
pose corr 0.999949 max abs 0.1017 mean abs 0.03991 (by error, pooled)
road_edges corr 0.999820 max abs 0.1563 mean abs 0.03573
road_transform corr 0.999998 max abs 0.0273 mean abs 0.00745 (by error, pooled)
wide_from_device_euler corr 0.999952 max abs 0.0547 mean abs 0.01962 (by error, pooled)
frame 20: corr 0.999708 max abs 2.2813
action corr 0.999980 max abs 0.0312 mean abs 0.01756 (by error, pooled)
desire_pred corr 0.999814 max abs 0.1709 mean abs 0.06601
desire_state corr 0.999896 max abs 0.6094 mean abs 0.39844 (by error, pooled)
hidden_state corr 0.999695 max abs 0.4141 mean abs 0.02774
lane_lines corr 0.999848 max abs 0.1250 mean abs 0.02548
lane_lines_prob corr 0.998681 max abs 0.3477 mean abs 0.12753 (by error, pooled)
lead corr 0.999992 max abs 2.2813 mean abs 0.18908
lead_prob corr 0.999597 max abs 0.0920 mean abs 0.05263 (by error, pooled)
meta corr 0.999260 max abs 0.2979 mean abs 0.11654
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999909 max abs 1.1875 mean abs 0.02544
pose corr 0.999982 max abs 0.0664 mean abs 0.02887 (by error, pooled)
road_edges corr 0.999678 max abs 0.1992 mean abs 0.04746
road_transform corr 0.999999 max abs 0.0312 mean abs 0.01023 (by error, pooled)
wide_from_device_euler corr 0.999975 max abs 0.0332 mean abs 0.01368 (by error, pooled)
frame 21: corr 0.999666 max abs 2.1875
action corr 0.999597 max abs 0.1172 mean abs 0.06538 (by error, pooled)
desire_pred corr 0.999893 max abs 0.1602 mean abs 0.07142
desire_state corr 0.999695 max abs 0.6406 mean abs 0.22241 (by error, pooled)
hidden_state corr 0.999502 max abs 0.8594 mean abs 0.04148
lane_lines corr 0.998604 max abs 0.6445 mean abs 0.06580 <-- FAIL
lane_lines_prob corr 0.999427 max abs 0.2393 mean abs 0.09894 (by error, pooled)
lead corr 0.999993 max abs 2.1875 mean abs 0.18941
lead_prob corr 0.999921 max abs 0.2661 mean abs 0.18669 (by error, pooled)
meta corr 0.997579 max abs 0.4238 mean abs 0.11203 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999903 max abs 1.4375 mean abs 0.04818
pose corr 0.999648 max abs 0.1660 mean abs 0.07027 (by error, pooled)
road_edges corr 0.998891 max abs 0.7344 mean abs 0.07769 <-- FAIL
road_transform corr 0.999997 max abs 0.0430 mean abs 0.01264 (by error, pooled)
wide_from_device_euler corr 0.999988 max abs 0.0312 mean abs 0.00947 (by error, pooled)
frame 22: corr 0.999730 max abs 0.9688
action corr 0.999818 max abs 0.0859 mean abs 0.04308 (by error, pooled)
desire_pred corr 0.999743 max abs 0.1699 mean abs 0.06920
desire_state corr 0.999755 max abs 0.6484 mean abs 0.25293 (by error, pooled)
hidden_state corr 0.999578 max abs 0.7734 mean abs 0.04318
lane_lines corr 0.999460 max abs 0.5234 mean abs 0.06310
lane_lines_prob corr 0.998969 max abs 0.2344 mean abs 0.15248 (by error, pooled)
lead corr 0.999997 max abs 0.7500 mean abs 0.08373
lead_prob corr 0.998906 max abs 0.2559 mean abs 0.16699 (by error, pooled)
meta corr 0.998557 max abs 0.3164 mean abs 0.08858 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999900 max abs 0.9688 mean abs 0.04323
pose corr 0.999838 max abs 0.1797 mean abs 0.04309 (by error, pooled)
road_edges corr 0.999552 max abs 0.5391 mean abs 0.06540
road_transform corr 0.999989 max abs 0.0625 mean abs 0.01803 (by error, pooled)
wide_from_device_euler corr 0.999991 max abs 0.0293 mean abs 0.01118 (by error, pooled)
frame 23: corr 0.999754 max abs 1.8125
action corr 0.999946 max abs 0.0566 mean abs 0.03091 (by error, pooled)
desire_pred corr 0.999797 max abs 0.1094 mean abs 0.06693
desire_state corr 0.999854 max abs 0.4141 mean abs 0.18726 (by error, pooled)
hidden_state corr 0.999689 max abs 0.3203 mean abs 0.03380
lane_lines corr 0.999647 max abs 0.2305 mean abs 0.03639
lane_lines_prob corr 0.999812 max abs 0.2578 mean abs 0.13620 (by error, pooled)
lead corr 0.999996 max abs 1.8125 mean abs 0.15519
lead_prob corr 0.998586 max abs 0.1162 mean abs 0.07145 (by error, pooled)
meta corr 0.998837 max abs 0.3789 mean abs 0.14272 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999919 max abs 1.4375 mean abs 0.03212
pose corr 0.999884 max abs 0.0957 mean abs 0.03571 (by error, pooled)
road_edges corr 0.999518 max abs 0.2852 mean abs 0.05398
road_transform corr 0.999995 max abs 0.0625 mean abs 0.02266 (by error, pooled)
wide_from_device_euler corr 0.999993 max abs 0.0508 mean abs 0.02232 (by error, pooled)
frame 24: corr 0.999663 max abs 4.2500
action corr 0.999419 max abs 0.0977 mean abs 0.03265 (by error, pooled)
desire_pred corr 0.999471 max abs 0.1943 mean abs 0.09872
desire_state corr 0.999775 max abs 0.8828 mean abs 0.28491 (by error, pooled)
hidden_state corr 0.999624 max abs 0.7793 mean abs 0.03748
lane_lines corr 0.999770 max abs 0.1523 mean abs 0.03479
lane_lines_prob corr 0.999445 max abs 0.2224 mean abs 0.12391 (by error, pooled)
lead corr 0.999963 max abs 4.2500 mean abs 0.32501
lead_prob corr 0.935773 max abs 0.0977 mean abs 0.06380 (by error, pooled)
meta corr 0.997676 max abs 0.3789 mean abs 0.13816 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999838 max abs 2.5625 mean abs 0.06005
pose corr 0.999927 max abs 0.3438 mean abs 0.04296 (by error, pooled)
road_edges corr 0.999467 max abs 0.2617 mean abs 0.06262
road_transform corr 0.999990 max abs 0.0820 mean abs 0.02707 (by error, pooled)
wide_from_device_euler corr 0.999989 max abs 0.0645 mean abs 0.02626 (by error, pooled)
frame 25: corr 0.999808 max abs 3.1250
action corr 0.999653 max abs 0.0664 mean abs 0.01833 (by error, pooled)
desire_pred corr 0.999687 max abs 0.2070 mean abs 0.07175
desire_state corr 0.999755 max abs 0.6094 mean abs 0.16528 (by error, pooled)
hidden_state corr 0.999628 max abs 0.6484 mean abs 0.03783
lane_lines corr 0.999819 max abs 0.1348 mean abs 0.03033
lane_lines_prob corr 0.999582 max abs 0.2324 mean abs 0.12898 (by error, pooled)
lead corr 0.999993 max abs 3.1250 mean abs 0.24022
lead_prob corr 0.924085 max abs 0.0723 mean abs 0.03646 (by error, pooled)
meta corr 0.998607 max abs 0.2879 mean abs 0.10612 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999934 max abs 2.0313 mean abs 0.04017
pose corr 0.999973 max abs 0.1719 mean abs 0.03019 (by error, pooled)
road_edges corr 0.999625 max abs 0.2109 mean abs 0.05730
road_transform corr 0.999993 max abs 0.0898 mean abs 0.03440 (by error, pooled)
wide_from_device_euler corr 0.999991 max abs 0.0742 mean abs 0.02070 (by error, pooled)
frame 26: corr 0.999859 max abs 0.8125
action corr 0.999989 max abs 0.0156 mean abs 0.00751 (by error, pooled)
desire_pred corr 0.999796 max abs 0.1348 mean abs 0.04940
desire_state corr 0.999849 max abs 0.4766 mean abs 0.16553 (by error, pooled)
hidden_state corr 0.999762 max abs 0.3292 mean abs 0.02990
lane_lines corr 0.999832 max abs 0.0932 mean abs 0.02609
lane_lines_prob corr 0.999951 max abs 0.1992 mean abs 0.10305 (by error, pooled)
lead corr 0.999999 max abs 0.8125 mean abs 0.06857
lead_prob corr 0.993880 max abs 0.0977 mean abs 0.05078 (by error, pooled)
meta corr 0.998106 max abs 0.3428 mean abs 0.14360 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999976 max abs 0.7500 mean abs 0.01680
pose corr 0.999965 max abs 0.0664 mean abs 0.02186 (by error, pooled)
road_edges corr 0.999589 max abs 0.2051 mean abs 0.04899
road_transform corr 0.999996 max abs 0.0410 mean abs 0.01718 (by error, pooled)
wide_from_device_euler corr 0.999999 max abs 0.0430 mean abs 0.01247 (by error, pooled)
frame 27: corr 0.999812 max abs 1.3750
action corr 0.999999 max abs 0.0098 mean abs 0.00638 (by error, pooled)
desire_pred corr 0.999810 max abs 0.1748 mean abs 0.05507
desire_state corr 0.999825 max abs 0.6484 mean abs 0.37073 (by error, pooled)
hidden_state corr 0.999774 max abs 0.3740 mean abs 0.02504
lane_lines corr 0.999852 max abs 0.1052 mean abs 0.02324
lane_lines_prob corr 0.999538 max abs 0.3555 mean abs 0.11041 (by error, pooled)
lead corr 0.999995 max abs 1.3750 mean abs 0.10300
lead_prob corr 0.989304 max abs 0.2083 mean abs 0.13688 (by error, pooled)
meta corr 0.998791 max abs 0.3535 mean abs 0.13906 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999869 max abs 0.9688 mean abs 0.02216
pose corr 0.999993 max abs 0.0430 mean abs 0.01334 (by error, pooled)
road_edges corr 0.999497 max abs 0.2754 mean abs 0.06200
road_transform corr 0.999998 max abs 0.0273 mean abs 0.00877 (by error, pooled)
wide_from_device_euler corr 0.999996 max abs 0.0156 mean abs 0.00536 (by error, pooled)
frame 28: corr 0.999783 max abs 1.5313
action corr 0.999996 max abs 0.0098 mean abs 0.00696 (by error, pooled)
desire_pred corr 0.999699 max abs 0.1741 mean abs 0.08971
desire_state corr 0.999917 max abs 0.4922 mean abs 0.26221 (by error, pooled)
hidden_state corr 0.999765 max abs 0.3613 mean abs 0.02414
lane_lines corr 0.999848 max abs 0.1021 mean abs 0.02579
lane_lines_prob corr 0.999867 max abs 0.1680 mean abs 0.08319 (by error, pooled)
lead corr 0.999988 max abs 1.5313 mean abs 0.11691
lead_prob corr 0.956972 max abs 0.2461 mean abs 0.16732 (by error, pooled)
meta corr 0.999044 max abs 0.3418 mean abs 0.12666
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999812 max abs 1.1563 mean abs 0.02474
pose corr 0.999957 max abs 0.0934 mean abs 0.03850 (by error, pooled)
road_edges corr 0.999779 max abs 0.1836 mean abs 0.04042
road_transform corr 0.999998 max abs 0.0273 mean abs 0.00712 (by error, pooled)
wide_from_device_euler corr 0.999959 max abs 0.0488 mean abs 0.01798 (by error, pooled)
frame 29: corr 0.999693 max abs 2.0000
action corr 0.999983 max abs 0.0176 mean abs 0.01233 (by error, pooled)
desire_pred corr 0.999821 max abs 0.1563 mean abs 0.07727
desire_state corr 0.999855 max abs 0.5859 mean abs 0.35730 (by error, pooled)
hidden_state corr 0.999646 max abs 0.4375 mean abs 0.02923
lane_lines corr 0.999854 max abs 0.1289 mean abs 0.02482
lane_lines_prob corr 0.999032 max abs 0.3164 mean abs 0.10803 (by error, pooled)
lead corr 0.999984 max abs 2.0000 mean abs 0.15293
lead_prob corr 0.990910 max abs 0.2988 mean abs 0.21615 (by error, pooled)
meta corr 0.999208 max abs 0.3242 mean abs 0.11801
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999776 max abs 1.2188 mean abs 0.03181
pose corr 0.999963 max abs 0.0937 mean abs 0.04216 (by error, pooled)
road_edges corr 0.999693 max abs 0.2227 mean abs 0.04173
road_transform corr 0.999999 max abs 0.0312 mean abs 0.01121 (by error, pooled)
wide_from_device_euler corr 0.999977 max abs 0.0313 mean abs 0.01336 (by error, pooled)
frame 30: corr 0.999605 max abs 2.0625
action corr 0.999815 max abs 0.0723 mean abs 0.04001 (by error, pooled)
desire_pred corr 0.999872 max abs 0.1680 mean abs 0.07307
desire_state corr 0.999837 max abs 0.7891 mean abs 0.32837 (by error, pooled)
hidden_state corr 0.999548 max abs 0.7969 mean abs 0.03899
lane_lines corr 0.999035 max abs 0.4922 mean abs 0.05368
lane_lines_prob corr 0.999311 max abs 0.2910 mean abs 0.11401 (by error, pooled)
lead corr 0.999993 max abs 2.0625 mean abs 0.18208
lead_prob corr 0.997241 max abs 0.2879 mean abs 0.21696 (by error, pooled)
meta corr 0.997357 max abs 0.4443 mean abs 0.12528 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999632 max abs 1.6875 mean abs 0.05149
pose corr 0.999748 max abs 0.1436 mean abs 0.06083 (by error, pooled)
road_edges corr 0.999208 max abs 0.6016 mean abs 0.06727
road_transform corr 0.999998 max abs 0.0391 mean abs 0.01377 (by error, pooled)
wide_from_device_euler corr 0.999995 max abs 0.0352 mean abs 0.00818 (by error, pooled)
frame 31: corr 0.999541 max abs 1.9375
action corr 0.999693 max abs 0.1016 mean abs 0.05630 (by error, pooled)
desire_pred corr 0.999761 max abs 0.1445 mean abs 0.08717
desire_state corr 0.999801 max abs 0.6016 mean abs 0.21435 (by error, pooled)
hidden_state corr 0.999481 max abs 0.7969 mean abs 0.04540
lane_lines corr 0.998952 max abs 0.6016 mean abs 0.05358 <-- FAIL
lane_lines_prob corr 0.999583 max abs 0.2109 mean abs 0.08696 (by error, pooled)
lead corr 0.999993 max abs 1.6875 mean abs 0.16762
lead_prob corr 0.990846 max abs 0.1953 mean abs 0.13208 (by error, pooled)
meta corr 0.997982 max abs 0.4082 mean abs 0.11955 <-- FAIL
pad corr 1.000000 max abs 0.0000 mean abs 0.00000 (by error, pooled)
plan corr 0.999547 max abs 1.9375 mean abs 0.06553
pose corr 0.999258 max abs 0.3398 mean abs 0.07945 (by error, pooled)
road_edges corr 0.999064 max abs 0.6484 mean abs 0.07150
road_transform corr 0.999995 max abs 0.0547 mean abs 0.01610 (by error, pooled)
wide_from_device_euler corr 0.999996 max abs 0.0332 mean abs 0.01246 (by error, pooled)
pooled over 32 frames, per slice and per column:
action corr 0.999829 by error, max abs 0.1582 <= 0.5352 max abs 0.1582 mean abs 0.02413
desire_pred corr 0.999692 (compared whole) max abs 0.2578 mean abs 0.07463
desire_state corr 0.999632 by error, max abs 1.391 <= 2.756 max abs 1.3906 mean abs 0.28295
hidden_state corr 0.999663 (compared whole) max abs 0.8594 mean abs 0.03253
lane_lines corr 0.999591 worst col 0.998911 mu[1] cols: mu[1] 0.998911 max abs 0.6445 mean abs 0.03293 <-- FAIL
lane_lines_prob corr 0.999274 by error, max abs 0.3633 <= 0.7691 max abs 0.3633 mean abs 0.10911
lead corr 0.999832 worst col 0.998609 mu[2] (4 by error) cols: mu[2] 0.998609 max abs 8.8750 mean abs 0.22762 <-- FAIL
lead_prob corr 0.995908 by error, max abs 0.4199 <= 0.5047 max abs 0.4199 mean abs 0.12956
meta corr 0.998456 (compared whole) max abs 0.4443 mean abs 0.12738 <-- FAIL
pad corr 1.000000 by error, max abs 0 <= 0 max abs 0.0000 mean abs 0.00000
plan corr 0.999782 worst col 0.998334 std[5] (11 by error) cols: std[5] 0.998334, std[6] 0.998372, std[8] 0.998817, std[12] 0.998622, std[13] 0.998600 max abs 4.2500 mean abs 0.04139 <-- FAIL
pose corr 0.999812 no column correlated (12 by error) cols: mu[1] by error, max abs 0.004059 > 0.003333 max abs 0.3828 mean abs 0.03753 <-- FAIL
road_edges corr 0.999516 worst col 0.998731 mu[1] cols: mu[1] 0.998731 max abs 0.7344 mean abs 0.05318 <-- FAIL
road_transform corr 0.999990 no column correlated (12 by error) max abs 0.0898 mean abs 0.01746
wide_from_device_euler corr 0.999972 no column correlated (6 by error) max abs 0.0762 mean abs 0.01479
FAIL: 6 slice(s) below corr 0.999 or past their error bound, whole or in a column: meta 0.997357 on one frame, lane_lines 0.998604 on one frame, road_edges 0.998891 on one frame, lead pooled, plan pooled, pose pooled
exit=1
Activity