Reasoning Faithfulness on US driving scenarios (held-out)
64.8Overall Consistency ScorePinocchio (Ours)
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Pinocchio (Ours)Model=Ours2026.07 | 64.8 | 74 | 88.8 | 97.7 | 83.5 | 73.2 | |
| VLM-JudgeModel=VLM-Judge2026.07 | 62.4 | 81.9 | 92.1 | 98.4 | 81 | 68 | |
| ADE-SwapModel=ADE-Swap2026.07 | 48.9 | 60 | 78.3 | 96.7 | 76.7 | 62 | |
| ADEModel=ADE2026.07 | 47.4 | 56.5 | 75.5 | 96.1 | 75.8 | 62.5 | |
| SFTModel=SFT2026.07 | 31.8 | 50.5 | 71.1 | 96 | 64.2 | 46.6 | |
| ADE-ReasonModel=ADE-Reason2026.07 | 24.7 | 51.9 | 72.1 | 96.3 | 46.5 | 36 |