Financial Auditing on FinMR
82.09Joint AccuracyAUDITFLOW
Evaluation results
| Method | Links | |
|---|---|---|
| AUDITFLOWBackbone=GPT-5.52026.06 | 82.09 | |
| AUDITFLOWBackbone=GPT-4o2026.06 | 80.6 | |
| AUDITFLOWBackbone=Sonnet 4.62026.06 | 80.6 | |
| AUDITFLOWBackbone=Qwen-397B2026.06 | 80.6 | |
| AUDITFLOWBackbone=Qwen-27B2026.06 | 73.13 | |
| Single AgentBackbone=GPT-5.52026.06 | 67.16 | |
| Single AgentBackbone=GPT-4o2026.06 | 67.16 | |
| Single AgentBackbone=Sonnet 4.62026.06 | 67.16 | |
| Single AgentBackbone=Qwen-397B2026.06 | 67.16 | |
| HerculeanBackbone=GPT-5.52026.06 | 65.67 | |
| HerculeanBackbone=GPT-4o2026.06 | 65.67 | |
| HerculeanBackbone=Sonnet 4.62026.06 | 65.67 | |
| Single AgentBackbone=Qwen-27B2026.06 | 64.18 | |
| GraphRAGBackbone=GPT-5.52026.06 | 48.39 | |
| GraphRAGBackbone=Sonnet 4.62026.06 | 46.77 | |
| TreeRAGBackbone=GPT-5.52026.06 | 43.28 | |
| HerculeanBackbone=Qwen-397B2026.06 | 43.28 | |
| TreeRAGBackbone=Sonnet 4.62026.06 | 41.79 | |
| GraphRAGBackbone=GPT-4o2026.06 | 38.1 | |
| HerculeanBackbone=Qwen-27B2026.06 | 37.31 | |
| TreeRAGBackbone=GPT-4o2026.06 | 35.82 | |
| TreeRAGBackbone=Qwen-397B2026.06 | 32.84 | |
| AUDITFLOWBackbone=Fino1-14B2026.06 | 31.34 | |
| Single AgentBackbone=Fino1-14B2026.06 | 29.85 | |
| HerculeanBackbone=Fino1-14B2026.06 | 28.35 | |
| GraphRAGBackbone=Qwen-397B2026.06 | 27.42 | |
| TreeRAGBackbone=Qwen-27B2026.06 | 23.88 | |
| GraphRAGBackbone=Qwen-27B2026.06 | 22.39 | |
| TreeRAGBackbone=Fino1-14B2026.06 | 20.9 | |
| GraphRAGBackbone=Fino1-14B2026.06 | 17.91 | |
| FinAuditingBackbone=GPT-5.52026.06 | 14.93 | |
| FinAuditingBackbone=Fino1-14B2026.06 | 14.93 | |
| Direct LLMBackbone=Qwen-397B2026.06 | 8.96 | |
| FinAuditingBackbone=Sonnet 4.62026.06 | 7.46 | |
| Direct LLMBackbone=GPT-5.52026.06 | 7.46 | |
| Direct LLMBackbone=Fino1-14B2026.06 | 7.46 | |
| Vanilla RAGBackbone=GPT-5.52026.06 | 7.46 | |
| Vanilla RAGBackbone=Qwen-27B2026.06 | 7.46 | |
| FinAuditingBackbone=GPT-4o2026.06 | 5.97 | |
| FinAuditingBackbone=Qwen-397B2026.06 | 5.97 | |
| Direct LLMBackbone=GPT-4o2026.06 | 5.97 | |
| FinAuditingBackbone=Qwen-27B2026.06 | 4.48 | |
| Vanilla RAGBackbone=Qwen-397B2026.06 | 4.48 | |
| Vanilla RAGBackbone=Sonnet 4.62026.06 | 2.99 | |
| Direct LLMBackbone=Sonnet 4.62026.06 | 1.49 | |
| Direct LLMBackbone=Qwen-27B2026.06 | 1.49 | |
| Vanilla RAGBackbone=GPT-4o2026.06 | 1.49 | |
| Vanilla RAGBackbone=Fino1-14B2026.06 | 1.49 |