Writing flaw detection on BenchMarker Writing - In Domain, NLP
81.5AccuracyGemini 2.5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 2.5Model Variant=Pro2026.02 | 81.5 | 65.8 | 53.1 | |
| GPT-52026.02 | 80.8 | 63.1 | 50.1 | |
| Claude 4.5Model Variant=Sonnet2026.02 | 79.4 | 62.5 | 48.3 | |
| Gemini 2.5Model Variant=Flash2026.02 | 78.8 | 61.9 | 47.3 | |
| SAQUET2026.02 | 77.4 | 40.4 | 28.5 | |
| Command R2026.02 | 76.7 | 53.2 | 37.6 | |
| Qwen-3Parameters=1.7B2026.02 | 76 | 24.9 | 16.4 | |
| Command R+2026.02 | 75.9 | 52.5 | 36.4 | |
| GPT-5Model Variant=Mini2026.02 | 75 | 55.3 | 38.1 | |
| Gemma-3Parameters=12B2026.02 | 74.7 | 4 | 2.6 | |
| Gemma-3Parameters=27B2026.02 | 74.7 | 2.9 | 2 | |
| Always Not Flawed2026.02 | 74.4 | 0 | 0 | |
| GPT-5Model Variant=Nano2026.02 | 73.5 | 38.6 | 22.4 | |
| Qwen-3Parameters=0.6B2026.02 | 73.4 | 4.6 | 0.4 | |
| Qwen-3Parameters=4B2026.02 | 73.4 | 49.4 | 31.5 | |
| Claude 4.5Model Variant=Haiku2026.02 | 71.8 | 57.6 | 38.1 | |
| LLaMA-3.1Parameters=8B2026.02 | 71.2 | 37.5 | 19.1 | |
| Qwen-3Parameters=32B2026.02 | 70.6 | 55.3 | 34.9 | |
| Qwen-3Parameters=14B2026.02 | 70.5 | 54.5 | 34 | |
| Qwen-3Parameters=8B2026.02 | 69.8 | 54.3 | 33.4 | |
| Gemini 2.5Model Variant=Lite2026.02 | 68.3 | 56.3 | 34.6 | |
| LLaMA-3.2Parameters=3B2026.02 | 68.3 | 36.6 | 15.5 | |
| LLaMA-3.1Parameters=70B2026.02 | 63.9 | 46.5 | 21.6 | |
| Gemma-3Parameters=4B2026.02 | 63.1 | 45.6 | 20 | |
| Random (50/50)2026.02 | 50 | 33.9 | 0 | |
| LLaMA-3.2Parameters=1B2026.02 | 46.6 | 37.5 | 2.5 | |
| Always Flawed2026.02 | 25.6 | 40.8 | 0 |