Chartered Accountancy Subject Examination on CA-Ben Final level
78.57FN1 (Fin. Reporting) AccuracyClaude 3.5 Sonnet
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Claude 3.5 Sonnet2026.04 | 78.57 | 46.67 | 64.29 | 53.33 | 20 | 62.5 | |
| GPT-4o2026.04 | 71.43 | 53.33 | 78.57 | 53.33 | 33.33 | 41.67 | |
| LLAMA 3.3 70B Inst.Parameters=70B, Model Type=Instruct2026.04 | 64.29 | 33.33 | 71.43 | 53.33 | 6.67 | 20.83 | |
| LLAMA 3.1 405B Inst.Parameters=405B, Model Type=Instruct2026.04 | 64.29 | 46.67 | 71.43 | 13.33 | 26.67 | 41.67 | |
| Microsoft Phi 42026.04 | 64.29 | 53.33 | 57.14 | 26.67 | 6.67 | 41.67 | |
| CA-ThinkFlow2026.04 | 57.14 | 73.33 | 57.14 | 40 | 26.67 | 37.5 | |
| 14B-Deepseek-R1Parameters=14B2026.04 | 42.86 | 60 | 50 | 13.33 | 6.67 | 25 | |
| MISTRAL Large2026.04 | 42.86 | 41.67 | 57.14 | 46.67 | 13.33 | 29.17 |