Chartered Accountancy Reasoning on CA-Ben official ICAI Study Material (test)
2Foundation PassesCA-ThinkFlow
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CA-ThinkFlowPrecision=4-bit quantized, Base Model=14B-DeepSeek-R1, Setup=Zero-shot2026.04 | 2 | 4 | 4 | 22 | 68.75 | |
| 14B-Deepseek-R1Setup=Zero-shot2026.04 | 2 | 2 | 3 | 15 | 46.87 | |
| GPT-4oPrecision=Full, Setup=Zero-shot2026.04 | 2 | 4 | 4 | 22 | 68.75 | |
| LLaMA-3.3-70B-InstructSetup=Zero-shot2026.04 | 2 | 4 | 3 | 19 | 59.38 | |
| LLaMA-3.1-405B-InstructSetup=Zero-shot2026.04 | 2 | 4 | 3 | 19 | 59.38 | |
| Mistral-LargePrecision=Full, Setup=Zero-shot2026.04 | 2 | 3 | 4 | 20 | 62.5 | |
| Claude 3.5 SonnetPrecision=Full, Setup=Zero-shot2026.04 | 2 | 4 | 4 | 22 | 68.75 | |
| Microsoft-Phi-4Setup=Zero-shot2026.04 | 2 | 4 | 3 | 19 | 59.38 |