Reasoning on Big-Bench Hard (test)
73.9AccuracyAgent-GWO
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Agent-GWOBackbone=GPT-4o-mini2026.04 | 73.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AoTBackbone=GPT-4o-mini2026.04 | 72.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agent-GWOBackbone=Gemma-3-12b-it2026.04 | 70.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GoTBackbone=GPT-4o-mini2026.04 | 70.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ToTBackbone=GPT-4o-mini2026.04 | 69.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SC/n=5Backbone=GPT-4o-mini2026.04 | 69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AoTBackbone=Gemma-3-12b-it2026.04 | 68.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ToTBackbone=Gemma-3-12b-it2026.04 | 68.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFlowBackbone=GPT-4o-mini2026.04 | 68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GoTBackbone=Gemma-3-12b-it2026.04 | 68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-RefineBackbone=GPT-4o-mini2026.04 | 67.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=GPT-4o-mini2026.04 | 66.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SC/n=5Backbone=Gemma-3-12b-it2026.04 | 66.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFlowBackbone=Gemma-3-12b-it2026.04 | 66 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-RefineBackbone=Gemma-3-12b-it2026.04 | 65.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=Gemma-3-12b-it2026.04 | 64.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dynamic Persona RoutingTotal=6511, Sampled=6512026.04 | 62.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 404 | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=-2025.06 | 60.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeekMath-BaseSize=7B, prompting=few-shot chain-of-thought2024.02 | 59.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Agent-GWOBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 58.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dense baselineModel Type=Dense, Compute=5.45e212025.06 | 58.02 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 57 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MoE w/ optimal ARModel Type=MoE, Activation rate=20.07, Compute=2.86e21, Data reuse=strict2025.06 | 56.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MistralSize=7B, prompting=few-shot chain-of-thought2024.02 | 55.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-Coder-Base-v1.5Size=7B, prompting=few-shot chain-of-thought2024.02 | 55.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ToTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-GMBackbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 54.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 53.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFlowBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 52.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-WS (-log p)Backbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-WS (1/p)Backbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 50.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-SC/n=5Backbone=Qwen2.5-Coder-7B-Instruct2026.04 | 49.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-RefineBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 48.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 47.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AlchemistCoder-DSBase Model=DeepSeek-Coder, Parameters=6.7B2024.05 | 45.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-GMBackbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 44.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MagicoderS-DSBase Model=DeepSeek-Coder, Parameters=6.7B2024.05 | 43.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-WS (1/p)Backbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 43.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Verifree*Backbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 43.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-Coder-Base-v1.5†Size=7B, prompting=few-shot chain-of-thought, note=checkpoint before learning rate decay2024.02 | 42.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeLlamaParameters=7B2024.05 | 42.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AlchemistCoder-LBase Model=Llama 2, Parameters=7B2024.05 | 42.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-WS (-log p)Backbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 42.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MagicoderS-CLBase Model=CodeLlama, Parameters=7B2024.05 | 41.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RLPR*Backbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 41.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Alpaca-GPT4 + NAIT (BBH)System ID=11, Capability Features=Individual, Data sampling ratio=10%2026.03 | 40.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.47 | — | — | — | — | — | 1.23 | — | — | |
| RLPR*Backbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 40.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Alpaca-GPT4 + Q2QSystem ID=04, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 40.34 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35.68 | — | — | — | — | — | -0.98 | — | — | |
| Alpaca-GPT4 + SelectITSystem ID=05, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 40.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.16 | — | — | — | — | — | 3.15 | — | — | |
| Alpaca-GPT4 + NAIT (GSM)System ID=08, Capability Features=Individual, Data sampling ratio=10%2026.03 | 40.28 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.7 | — | — | — | — | — | 4.65 | — | — | |
| JLB*Backbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 40.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| JLB*Backbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 40.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Alpaca-GPT4 + NAIT (7-11)System ID=12, Capability Features=All, Data sampling ratio=10%2026.03 | 40.02 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.2 | — | — | — | — | — | 3.24 | — | — | |
| Alpaca-GPT4 + NAIT (TydiQA)System ID=10, Capability Features=Individual, Data sampling ratio=10%2026.03 | 40 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.71 | — | — | — | — | — | 1.89 | — | — | |
| Alpaca-GPT4System ID=01, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 39.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.03 | — | — | — | — | — | — | — | — | |
| LIMASystem ID=02, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 39.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.17 | — | — | — | — | — | 0.39 | — | — | |
| Alpaca-GPT4 + AlpaGasusSystem ID=03, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 39.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35.18 | — | — | — | — | — | -2.34 | — | — | |
| Alpaca-GPT4 + NAIT (CodeX)System ID=09, Capability Features=Individual, Data sampling ratio=10%2026.03 | 39.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.06 | — | — | — | — | — | 2.88 | — | — | |
| AlchemistCoder-CLBase Model=CodeLlama, Parameters=7B2024.05 | 39.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Alpaca-GPT4 + Random BaselineSystem ID=06, Fine-tuning protocol=Full Fine-tuning, Data sampling ratio=10%2026.03 | 39.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35.69 | — | — | — | — | — | -0.94 | — | — | |
| NRT-WS (-log p)Backbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 39.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Alpaca-GPT4 + NAIT (MMLU)System ID=07, Capability Features=Individual, Data sampling ratio=10%2026.03 | 38.52 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.18 | — | — | — | — | — | 3.2 | — | — | |
| SFTBackbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SFTBackbone=Mistral-7B-v0.3, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 36.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Verifree*Backbone=Llama-3.1-8B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 35.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-WS (1/p)Backbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 2Parameters=7B2024.05 | 34.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AdaGCSteps (tokens)=60k (1T), AdamW eps=1e-152025.02 | 31.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GlobalGCSteps (tokens)=60k (1T), AdamW eps=1e-82025.02 | 30.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GlobalGCSteps (tokens)=60k (1T), AdamW eps=1e-152025.02 | 30.59 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SFTBackbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 30.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NRT-GMBackbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 29.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GlobalGCSteps (tokens)=21k (350B), AdamW eps=1e-152025.02 | 29.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GlobalGCSteps (tokens)=21k (350B), AdamW eps=1e-82025.02 | 28.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RLPR*Backbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 28.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| JLB*Backbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 28.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AdaGCSteps (tokens)=21k (350B), AdamW eps=1e-152025.02 | 27.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Verifree*Backbone=Llama-3.2-3B-Base, Fine-tuning mixture=200k subset of tulu-3-sft-mixture2026.02 | 26.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CodeLlama-PythonParameters=7B2024.05 | 26.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-Coder-BaseParameters=6.7B2024.05 | 12.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Cog-DRIFTBase Model=Qwen3-4B-Instruct-25072026.04 | 10.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Cog-DRIFTBase Model=Llama3.2-3B-Instruct2026.04 | 8.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NuRL (Abstract)Base Model=Qwen3-4B-Instruct-25072026.04 | 7.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NuRL (Prefix)Base Model=Qwen3-4B-Instruct-25072026.04 | 6.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBase Model=Qwen3-4B-Instruct-25072026.04 | 6.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RFTBase Model=Qwen3-4B-Instruct-25072026.04 | 4.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RFTBase Model=Llama3.2-3B-Instruct2026.04 | 2.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NuRL (Abstract)Base Model=Llama3.2-3B-Instruct2026.04 | 1.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NuRL (Prefix)Base Model=Llama3.2-3B-Instruct2026.04 | 1.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shotBase Model=Qwen3-4B-Instruct-25072026.04 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Few-shotBase Model=Qwen3-4B-Instruct-25072026.04 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-shotBase Model=Llama3.2-3B-Instruct2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Few-shotBase Model=Llama3.2-3B-Instruct2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBase Model=Llama3.2-3B-Instruct2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| APETBackbone=Llama-3-8B-Instruct2025.03 | — | 67.5 | 0 | 3 | 22.5 | 0 | 0 | 24.5 | 3.5 | 3.5 | 3 | 20.5 | 0 | 52 | 38.5 | 27.5 | 8.33 | 6.5 | 65.5 | 6.5 | 0 | 0 | 6 | 18 | 19.5 | 80.5 | 42 | 41 | 20.73 | — | — | — | — | — | — | — | — | |
| Best Static BaselineTotal=6511, Sampled=6512026.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 63.4 | — | |
| EvoPrompt(DE)Task-solving LLM=GPT-4o mini2025.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.17 | — | — | — | — | — | — | — | — | |
| EvoPrompt(DE)-OPTS(TS)Task-solving LLM=GPT-4o mini2025.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.43 | — | — | — | — | — | — | — | — | |
| EvoPrompt(GA)Backbone=Llama-3-8B-Instruct2025.03 | — | 72.83 | 37.71 | 13.67 | 32.17 | 6.67 | 42.5 | 29.83 | 54.83 | 12.33 | 6.83 | 58.17 | 48.67 | 50.33 | 79.67 | 85.5 | 25 | 50.17 | 51 | 31.67 | 45.83 | 62.63 | 63.67 | 67.67 | 53.33 | 81 | 96.33 | 46.83 | 48.4 | — | — | — | — | — | — | — | — | |
| EvoPrompt(GA)-OPTS(APET)Backbone=Llama-3-8B-Instruct2025.03 | — | 84.33 | 40.88 | 14.5 | 37 | 6.5 | 45.17 | 37.33 | 61.67 | 20 | 7.83 | 64 | 49.67 | 49.83 | 80.17 | 82.67 | 29.86 | 47.5 | 63.17 | 49 | 52.6 | 62.63 | 59.33 | 67.83 | 51.5 | 81.67 | 95.83 | 43.67 | 51.34 | — | — | — | — | — | — | — | — |