Financial Auditing on Financial Audit Evaluation Set 67 cases (test)
0SERFinAuditing
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FinAuditingBackbone=GPT-5.52026.06 | 0 | 16.42 | 68.66 | |
| FinAuditingBackbone=GPT-4o2026.06 | 0 | 19.4 | 74.63 | |
| FinAuditingBackbone=Sonnet 4.62026.06 | 0 | 22.39 | 70.15 | |
| HerculeanBackbone=GPT-5.52026.06 | 0 | 5.97 | 28.36 | |
| HerculeanBackbone=Sonnet 4.62026.06 | 0 | 0 | 34.33 | |
| HerculeanBackbone=Qwen-397B2026.06 | 0 | 11.94 | 33.78 | |
| AUDITFLOWBackbone=Sonnet 4.62026.06 | 0 | 0 | 19.4 | |
| FinAuditingBackbone=Qwen-397B2026.06 | 1.49 | 20.89 | 71.64 | |
| HerculeanBackbone=GPT-4o2026.06 | 1.49 | 8.96 | 23.88 | |
| AUDITFLOWBackbone=GPT-5.52026.06 | 1.49 | 1.49 | 14.93 | |
| AUDITFLOWBackbone=GPT-4o2026.06 | 1.49 | 0 | 17.91 | |
| HerculeanBackbone=Qwen-27B2026.06 | 2.99 | 11.94 | 47.76 | |
| AUDITFLOWBackbone=Qwen-397B2026.06 | 2.99 | 0 | 16.42 | |
| GraphRAGBackbone=GPT-5.52026.06 | 4.84 | 1.61 | 45.16 | |
| AUDITFLOWBackbone=Qwen-27B2026.06 | 7.46 | 2.99 | 16.42 | |
| GraphRAGBackbone=Sonnet 4.62026.06 | 11.29 | 0 | 41.94 | |
| Single AgentBackbone=GPT-5.52026.06 | 13.43 | 1.49 | 17.91 | |
| Single AgentBackbone=GPT-4o2026.06 | 13.43 | 1.49 | 17.91 | |
| Single AgentBackbone=Sonnet 4.62026.06 | 13.43 | 1.49 | 17.91 | |
| Single AgentBackbone=Qwen-397B2026.06 | 13.43 | 1.49 | 17.91 | |
| FinAuditingBackbone=Qwen-27B2026.06 | 14.93 | 29.85 | 50.75 | |
| Single AgentBackbone=Qwen-27B2026.06 | 16.42 | 1.49 | 17.91 | |
| GraphRAGBackbone=GPT-4o2026.06 | 17.46 | 6.35 | 37.38 | |
| TreeRAGBackbone=GPT-4o2026.06 | 20.89 | 1.49 | 41.79 | |
| TreeRAGBackbone=GPT-5.52026.06 | 32.84 | 2.99 | 20.89 | |
| TreeRAGBackbone=Sonnet 4.62026.06 | 34.33 | 1.49 | 22.39 | |
| TreeRAGBackbone=Qwen-27B2026.06 | 37.31 | 2.99 | 35.82 | |
| Single AgentBackbone=Fino1-14B2026.06 | 37.31 | 1.49 | 31.34 | |
| AUDITFLOWBackbone=Fino1-14B2026.06 | 37.31 | 1.49 | 29.85 | |
| GraphRAGBackbone=Qwen-397B2026.06 | 38.71 | 1.61 | 32.26 | |
| TreeRAGBackbone=Qwen-397B2026.06 | 38.81 | 7.46 | 20.89 | |
| HerculeanBackbone=Fino1-14B2026.06 | 43.28 | 22.39 | 5.97 | |
| GraphRAGBackbone=Qwen-27B2026.06 | 44.78 | 4.48 | 28.36 | |
| TreeRAGBackbone=Fino1-14B2026.06 | 44.78 | 7.46 | 26.87 | |
| Direct LLMBackbone=Qwen-397B2026.06 | 53.73 | 1.49 | 35.82 | |
| Direct LLMBackbone=Fino1-14B2026.06 | 56.72 | 8.96 | 26.87 | |
| Direct LLMBackbone=GPT-5.52026.06 | 59.7 | 2.99 | 29.85 | |
| Direct LLMBackbone=GPT-4o2026.06 | 59.7 | 2.99 | 31.34 | |
| GraphRAGBackbone=Fino1-14B2026.06 | 62.69 | 14.93 | 4.48 | |
| Direct LLMBackbone=Qwen-27B2026.06 | 64.18 | 1.49 | 32.84 | |
| FinAuditingBackbone=Fino1-14B2026.06 | 70.15 | 11.94 | 2.99 | |
| Direct LLMBackbone=Sonnet 4.62026.06 | 88.06 | 0 | 10.45 | |
| Vanilla RAGBackbone=GPT-4o2026.06 | 89.55 | 7.46 | 1.49 | |
| Vanilla RAGBackbone=GPT-5.52026.06 | 92.54 | 0 | 0 | |
| Vanilla RAGBackbone=Qwen-27B2026.06 | 92.54 | 0 | 0 | |
| Vanilla RAGBackbone=Fino1-14B2026.06 | 94.03 | 1.49 | 2.99 | |
| Vanilla RAGBackbone=Qwen-397B2026.06 | 95.52 | 0 | 0 | |
| Vanilla RAGBackbone=Sonnet 4.62026.06 | 97.01 | 0 | 0 |