Error Recognition on WHO&WHEN Algorithm-Generated
38.9Accuracy@0CORRECT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CORRECTModel=Gemini-2.5-flash2025.09 | 38.9 | 55.2 | |
| CORRECTModel=GPT-52025.09 | 38.1 | 58.8 | |
| LLM-as-a-JudgeModel=Gemini-2.5-flash2025.09 | 31.8 | 56 | |
| LLM-as-a-JudgeModel=Gemini-2.5-pro2025.09 | 25.4 | 50 | |
| CORRECTModel=Gemini-2.5-pro2025.09 | 24.6 | 52.4 | |
| CORRECTModel=GPT-5-nano2025.09 | 24.6 | 44.4 | |
| LLM-as-a-JudgeModel=DeepSeek-R12025.09 | 23.8 | 54 | |
| LLM-as-a-JudgeModel=Qwen-3-80b2025.09 | 21.4 | 47.6 | |
| CORRECTModel=Qwen-2.5-7b2025.09 | 19.8 | 46.8 | |
| LLM-as-a-JudgeModel=Qwen-2.5-7b2025.09 | 19.1 | 42.9 | |
| LLM-as-a-JudgeModel=GPT-5-nano2025.09 | 19.1 | 41.3 | |
| Fine-tuned LLMModel=Qwen-2.5-7b2025.09 | 18.9 | 42.9 | |
| LLM-as-a-JudgeModel=GPT-4o2025.09 | 18.3 | 50 | |
| LLM-as-a-JudgeModel=GPT-52025.09 | 18.3 | 56.4 | |
| Naive ICLModel=Qwen-2.5-7b2025.09 | 15.9 | 40.5 | |
| LLM-as-a-JudgeModel=Qwen-3-30b2025.09 | 15.1 | 42.9 | |
| MIPROModel=Qwen-2.5-7b2025.09 | 15.1 | 41.3 | |
| LLM-as-a-JudgeModel=GPT-4o-mini2025.09 | 12.7 | 39.7 | |
| LLM-as-a-JudgeModel=Llama-8b2025.09 | 3.2 | 15.9 |