Logical Reasoning on CounterBench (test)
88.9AccuracyFLEx
Evaluation results
| Method | Links | |
|---|---|---|
| FLExBase Model=Qwen-72B, Model Family=Qwen-2.5 Instruct2026.01 | 88.9 | |
| RAGBase Model=Qwen-72B, Model Family=Qwen-2.5 Instruct2026.01 | 86.5 | |
| RAGBase Model=Qwen-32B, Model Family=Qwen-2.5 Instruct2026.01 | 85.9 | |
| SCBase Model=Qwen-72B, Model Family=Qwen-2.5 Instruct2026.01 | 85.2 | |
| FLExBase Model=Gemma-12B, Model Family=Gemma-3 Instruct2026.01 | 84.9 | |
| FLExBase Model=Qwen-32B, Model Family=Qwen-2.5 Instruct2026.01 | 84.6 | |
| CoTBase Model=Qwen-72B, Model Family=Qwen-2.5 Instruct2026.01 | 82.8 | |
| SRBase Model=Qwen-72B, Model Family=Qwen-2.5 Instruct2026.01 | 81.3 | |
| FLExBase Model=Gemma-27B, Model Family=Gemma-3 Instruct2026.01 | 80.7 | |
| FLExBase Model=Qwen-14B, Model Family=Qwen-2.5 Instruct2026.01 | 80.6 | |
| SRBase Model=Qwen-32B, Model Family=Qwen-2.5 Instruct2026.01 | 80.3 | |
| SCBase Model=Qwen-32B, Model Family=Qwen-2.5 Instruct2026.01 | 80.2 | |
| CoTBase Model=Qwen-32B, Model Family=Qwen-2.5 Instruct2026.01 | 79.7 | |
| RAGBase Model=Qwen-14B, Model Family=Qwen-2.5 Instruct2026.01 | 78.3 | |
| SCBase Model=Gemma-27B, Model Family=Gemma-3 Instruct2026.01 | 78.2 | |
| FLExBase Model=Qwen-7B, Model Family=Qwen-2.5 Instruct2026.01 | 78 | |
| CoTBase Model=Gemma-27B, Model Family=Gemma-3 Instruct2026.01 | 76 | |
| RAGBase Model=Gemma-27B, Model Family=Gemma-3 Instruct2026.01 | 75.9 | |
| SCBase Model=Qwen-14B, Model Family=Qwen-2.5 Instruct2026.01 | 75.8 | |
| SRBase Model=Qwen-14B, Model Family=Qwen-2.5 Instruct2026.01 | 74.7 | |
| SRBase Model=Gemma-27B, Model Family=Gemma-3 Instruct2026.01 | 74.6 | |
| FLExBase Model=Gemma-4B, Model Family=Gemma-3 Instruct2026.01 | 74.3 | |
| CoTBase Model=Qwen-14B, Model Family=Qwen-2.5 Instruct2026.01 | 74 | |
| SCBase Model=Gemma-12B, Model Family=Gemma-3 Instruct2026.01 | 72.1 | |
| RAGBase Model=Gemma-12B, Model Family=Gemma-3 Instruct2026.01 | 70.4 | |
| CoTBase Model=Qwen-7B, Model Family=Qwen-2.5 Instruct2026.01 | 69.9 | |
| RAGBase Model=Gemma-4B, Model Family=Gemma-3 Instruct2026.01 | 69.7 | |
| SCBase Model=Qwen-7B, Model Family=Qwen-2.5 Instruct2026.01 | 69 | |
| SRBase Model=Qwen-7B, Model Family=Qwen-2.5 Instruct2026.01 | 68.8 | |
| CoTBase Model=Gemma-12B, Model Family=Gemma-3 Instruct2026.01 | 68.7 | |
| SCBase Model=Gemma-4B, Model Family=Gemma-3 Instruct2026.01 | 68.5 | |
| RAGBase Model=Qwen-7B, Model Family=Qwen-2.5 Instruct2026.01 | 68.5 | |
| SRBase Model=Gemma-12B, Model Family=Gemma-3 Instruct2026.01 | 66.7 | |
| SRBase Model=Gemma-4B, Model Family=Gemma-3 Instruct2026.01 | 65.4 | |
| CoTBase Model=Gemma-4B, Model Family=Gemma-3 Instruct2026.01 | 64.9 | |
| RAGBase Model=Qwen-3B, Model Family=Qwen-2.5 Instruct2026.01 | 61.3 | |
| FLExBase Model=Qwen-3B, Model Family=Qwen-2.5 Instruct2026.01 | 58.1 | |
| SCBase Model=Qwen-3B, Model Family=Qwen-2.5 Instruct2026.01 | 55.2 | |
| SRBase Model=Qwen-3B, Model Family=Qwen-2.5 Instruct2026.01 | 53.3 | |
| FLExBase Model=Gemma-1B, Model Family=Gemma-3 Instruct2026.01 | 51.6 | |
| SCBase Model=Gemma-1B, Model Family=Gemma-3 Instruct2026.01 | 51.1 | |
| CoTBase Model=Qwen-3B, Model Family=Qwen-2.5 Instruct2026.01 | 50.5 | |
| CoTBase Model=Gemma-1B, Model Family=Gemma-3 Instruct2026.01 | 49.8 | |
| SRBase Model=Gemma-1B, Model Family=Gemma-3 Instruct2026.01 | 49.7 | |
| RAGBase Model=Qwen-1.5B, Model Family=Qwen-2.5 Instruct2026.01 | 49.7 | |
| RAGBase Model=Gemma-1B, Model Family=Gemma-3 Instruct2026.01 | 48.5 | |
| SCBase Model=Qwen-1.5B, Model Family=Qwen-2.5 Instruct2026.01 | 47.8 | |
| FLExBase Model=Qwen-1.5B, Model Family=Qwen-2.5 Instruct2026.01 | 47.8 | |
| SRBase Model=Qwen-1.5B, Model Family=Qwen-2.5 Instruct2026.01 | 47 | |
| RAGBase Model=Qwen-0.5B, Model Family=Qwen-2.5 Instruct2026.01 | 43.4 | |
| FLExBase Model=Qwen-0.5B, Model Family=Qwen-2.5 Instruct2026.01 | 42.6 | |
| CoTBase Model=Qwen-1.5B, Model Family=Qwen-2.5 Instruct2026.01 | 39.3 | |
| SCBase Model=Qwen-0.5B, Model Family=Qwen-2.5 Instruct2026.01 | 38 | |
| SRBase Model=Qwen-0.5B, Model Family=Qwen-2.5 Instruct2026.01 | 21.8 | |
| CoTBase Model=Qwen-0.5B, Model Family=Qwen-2.5 Instruct2026.01 | 21.3 |