Algorithmic Program Execution on FHP Algorithmic Task n=100 (test)
100AccuracyGemini-3.1-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3.1-ProProtocol=LLM-PV, Tools=with tools2025.10 | 100 | |
| GPT-5-ThinkingProtocol=LLM-PV, Tools=with tools2025.10 | 100 | |
| GPT-5-ThinkingProtocol=ICL, Tools=with tools2025.10 | 88 | |
| Genetic Algorithm2025.10 | 52.5 | |
| Llama-3.2-1BProtocol=FT2025.10 | 51.3 | |
| Qwen3-1.7BProtocol=FT2025.10 | 50.7 | |
| Qwen3-1.7BProtocol=SGD2025.10 | 50.6 | |
| TabPFN2025.10 | 50.5 | |
| DeepSeek-Coder-1.3BProtocol=SGD2025.10 | 50.5 | |
| DeepSeek-Coder-1.3BProtocol=FT2025.10 | 50.2 | |
| XGBoost2025.10 | 50 | |
| SVM2025.10 | 49.9 | |
| Random Forest2025.10 | 49.7 | |
| Llama-3.2-1BProtocol=SGD2025.10 | 48.6 |