Anomaly Recognition on ARFBench 1.0 (test)
87.2Overall AccuracyModel-Expert Oracle
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Model-Expert OracleModel Category=Baselines2026.04 | 87.2 | 96.4 | 80.3 | 90.5 | 82.8 | 89 | 77.1 | 86.3 | |
| Domain Experts (n=2)Model Category=Baselines2026.04 | 72.7 | 89.3 | 67.7 | 71.4 | 64.6 | 76.1 | 64.5 | 60.9 | |
| Non-domain Experts (n=2)Model Category=Baselines2026.04 | 69.7 | 80.4 | 63.2 | 72 | 60.7 | 68 | 59.9 | 59 | |
| Toto-1.0-QA-Experimental 32B (TSFM-VLM)Model Category=Post-Trained Foundation Models, Size=32B, Architecture=TSFM-VLM2026.04 | 63.9 | 84.7 | 55.6 | 64.6 | 48.9 | 66.3 | 48.4 | 43.5 | |
| GPT-5Model Category=Vision-Language Models (VLMs)2026.04 | 62.7 | 82 | 55.9 | 62.5 | 51.9 | 66.9 | 51.2 | 47.5 | |
| GPT-5.4Model Category=Vision-Language Models (VLMs)2026.04 | 61.3 | 81.1 | 54.2 | 61.3 | 51.4 | 62.6 | 50.4 | 48.4 | |
| Gemini 3 ProModel Category=Vision-Language Models (VLMs)2026.04 | 58.1 | 82.9 | 51 | 56.5 | 49.6 | 67.8 | 49.7 | 43.4 | |
| Qwen3-VL 32B (post-trained)Model Category=Post-Trained Foundation Models, Size=32B, Mode=post-trained2026.04 | 56.9 | 84.7 | 50.3 | 53.8 | 46.6 | 69.8 | 44.9 | 40.5 | |
| GPT-5 (text)Model Category=Large Language Models (LLMs), Modality=text2026.04 | 56.4 | 82.6 | 45.2 | 57.9 | 43.9 | 66.1 | 39.6 | 40.3 | |
| Claude Opus 4.6Model Category=Vision-Language Models (VLMs)2026.04 | 54.8 | 88.3 | 52.3 | 45.9 | 46.7 | 65.8 | 49.1 | 38.2 | |
| Qwen3-VL 32BModel Category=Vision-Language Models (VLMs), Size=32B2026.04 | 52.8 | 80.2 | 46.7 | 49.2 | 45.1 | 65.1 | 41.9 | 41.3 | |
| Toto-1.0-Qwen3 32B (TSFM-LLM)Model Category=Post-Trained Foundation Models, Size=32B, Architecture=TSFM-LLM2026.04 | 48.8 | 82.9 | 47.4 | 38.7 | 33.9 | 60 | 43.6 | 16.4 | |
| Qwen3 32BModel Category=Large Language Models (LLMs), Size=32B2026.04 | 47.9 | 80.9 | 35.1 | 48.6 | 36.1 | 55.7 | 31.5 | 33.8 | |
| GPT-4.1Model Category=Vision-Language Models (VLMs)2026.04 | 47.9 | 80.2 | 50.3 | 34.8 | 44 | 65.1 | 48 | 33.1 | |
| Claude Sonnet 4.5Model Category=Vision-Language Models (VLMs)2026.04 | 47.2 | 83.8 | 43.5 | 38.4 | 37.9 | 63.2 | 40.6 | 26.9 | |
| GPT-4oModel Category=Vision-Language Models (VLMs)2026.04 | 47.2 | 79.3 | 49 | 34.8 | 42.4 | 64.2 | 43.8 | 33.8 | |
| Qwen3-VL 8BModel Category=Vision-Language Models (VLMs), Size=8B2026.04 | 45.3 | 80.2 | 40.8 | 37.8 | 34.7 | 63.5 | 36.1 | 23.6 | |
| Per-category Frequent ChoiceModel Category=Baselines2026.04 | 45.1 | 84.7 | 30.1 | 45.6 | 17.3 | 45.9 | 12.3 | 12.5 | |
| ChatTS 8B (TS-LLM)Model Category=Post-Trained Foundation Models, Size=8B, Architecture=TS-LLM2026.04 | 31.1 | 60.4 | 26.5 | 25.5 | 22.1 | 48.1 | 20 | 15.4 | |
| Random ChoiceModel Category=Baselines2026.04 | 24.5 | 50 | 20 | 20 | 22.5 | 45.6 | 20.5 | 16.8 | |
| OpenTSLM 1B (TS-LLM)Model Category=Post-Trained Foundation Models, Size=1B, Architecture=TS-LLM2026.04 | 0.8 | 0 | 2 | 0 | 1.2 | 0 | 3 | 0 |