Fine-grained traffic reasoning on FGTR-Bench blind (test)
97.4Holistic Sign AccuracyTSR-MLLM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TSR-MLLMParams=4B, Training=Fine-tuned (ours)2026.07 | 97.4 | 99.3 | 86.9 | 29.4 | 52.3 | 74.1 | |
| Qwen3-VL-4BParams=4B, Training=w/ FGTR-Bench training2026.07 | 94.1 | 97.4 | 83.1 | 28.4 | 51.4 | 72 | |
| Qwen2.5-VL-7BParams=7B, Training=w/ FGTR-Bench training2026.07 | 90 | 98.2 | 80 | 35.1 | 52.8 | 71.8 | |
| Qwen3-VL-4BParams=4B, Training=w/o FGTR-Bench training2026.07 | 86.8 | 96.3 | 83.2 | 28.9 | 52.4 | 70.1 | |
| LLaVA-OneVisionParams=7B, Training=w/ FGTR-Bench training2026.07 | 86.8 | 87.9 | 78.4 | 29.6 | 54.1 | 68.5 | |
| Qwen2.5-VL-7BParams=7B, Training=w/o FGTR-Bench training2026.07 | 86.7 | 97.9 | 77.7 | 34 | 51.3 | 69.9 | |
| Qwen2.5-VL-3BParams=3B, Training=w/ FGTR-Bench training2026.07 | 85.4 | 79.1 | 67.9 | 30.7 | 39.4 | 61.8 | |
| LLaVA-OneVisionParams=7B, Training=w/o FGTR-Bench training2026.07 | 83 | 80.8 | 75.6 | 27.5 | 53.3 | 65.4 | |
| Qwen2.5-VL-3BParams=3B, Training=w/o FGTR-Bench training2026.07 | 82 | 73.2 | 64.7 | 29.1 | 36.6 | 58.5 | |
| LLaVA-1.5Params=7B, Training=w/ FGTR-Bench training2026.07 | 73 | 79.7 | 48.2 | 14.2 | 30.4 | 50.5 | |
| LLaVA-1.5Params=7B, Training=w/o FGTR-Bench training2026.07 | 41.9 | 77.5 | 29.3 | 14.4 | 42 | 41 |