General Reasoning on BIG-bench
81.6Accuracy (General)POES
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| POESOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 81.6 | — | — | — | |
| IPOMPOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 75.8 | — | — | — | |
| RandomOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 74.8 | — | — | — | |
| PredictionOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 74.6 | — | — | — | |
| SESSOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 71.9 | — | — | — | |
| POESOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 71.3 | — | — | — | |
| AnchorOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 70.9 | — | — | — | |
| POESOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 63.3 | — | — | — | |
| POESOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 61.2 | — | — | — | |
| POESOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 59.3 | — | — | — | |
| POESOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 56.6 | — | — | — | |
| IPOMPOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 55.8 | — | — | — | |
| PredictionOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 55 | — | — | — | |
| AnchorOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 54.6 | — | — | — | |
| SESSOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 53.8 | — | — | — | |
| AnchorOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 53.3 | — | — | — | |
| AnchorOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 52.1 | — | — | — | |
| IPOMPOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 52 | — | — | — | |
| SESSOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 51.7 | — | — | — | |
| RandomOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 51.2 | — | — | — | |
| PredictionOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 51.2 | — | — | — | |
| IPOMPOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 50.8 | — | — | — | |
| RandomOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 50.1 | — | — | — | |
| PredictionOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 49.6 | — | — | — | |
| IPOMPOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 49.6 | — | — | — | |
| RandomOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 49.5 | — | — | — | |
| SESSOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 48.6 | — | — | — | |
| RandomOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 48.2 | — | — | — | |
| PredictionOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 46.7 | — | — | — | |
| IPOMPOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 46.7 | — | — | — | |
| SESSOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 46.5 | — | — | — | |
| AnchorOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 46.5 | — | — | — | |
| SESSOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 46.4 | — | — | — | |
| RandomOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 46.4 | — | — | — | |
| PredictionOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 44.6 | — | — | — | |
| AnchorOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 42.5 | — | — | — | |
| Prompt basedBackbone=Gemma-2-9B-it, Configuration=w/o reflection2025.05 | — | 61.2 | 75.6 | 14.4 | |
| Prompt basedBackbone=Gemma-2-9B-it, Configuration=w/ reflection2025.05 | — | 61.2 | 74.4 | 13.2 | |
| Prompt basedBackbone=Gemma-2-27B-it, Configuration=w/o reflection2025.05 | — | 63.4 | 72 | 8.6 | |
| Prompt basedBackbone=Gemma-2-27B-it, Configuration=w/ reflection2025.05 | — | 63.4 | 75.2 | 11.8 | |
| Prompt basedBackbone=Meta-Llama-3-8B-Instruct, Variant=w/o reflection2025.05 | — | 38.2 | — | — | |
| Prompt basedBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ reflection2025.05 | — | 38.2 | 52.4 | 14.2 | |
| Prompt basedBackbone=Meta-Llama-3-70B-Instruct, Variant=w/o reflection2025.05 | — | 48 | — | — | |
| Prompt basedBackbone=Meta-Llama-3-70B-Instruct, Variant=w/ reflection (variant 1)2025.05 | — | 48 | 67 | 19 | |
| Prompt basedBackbone=Meta-Llama-3-70B-Instruct, Variant=w/ reflection (variant 2)2025.05 | — | 48 | 64.4 | 16.4 | |
| Prompt basedBackbone=Mistral-7B-Instruct-v0.2, Variant=w/o reflection2025.05 | — | 36.6 | — | — | |
| Prompt basedBackbone=Mistral-7B-Instruct-v0.2, Variant=w/ reflection2025.05 | — | 36.6 | 43.8 | 7.2 | |
| Prompt basedBackbone=Mistral-22B-Small-Instruct, Variant=w/o reflection2025.05 | — | 54.4 | — | — | |
| Prompt basedBackbone=Mistral-22B-Small-Instruct, Variant=w/ reflection (variant 1)2025.05 | — | 54.4 | 67.2 | 12.8 | |
| Prompt basedBackbone=Mistral-22B-Small-Instruct, Variant=w/ reflection (variant 2)2025.05 | — | 54.4 | 68 | 13.6 | |
| ReflectEvoBackbone=Gemma-2-9B-it, Configuration=one-stage w/ D+2025.05 | — | 61.2 | 78.4 | 17.2 | |
| ReflectEvoBackbone=Gemma-2-9B-it, Configuration=two-stage w/ D+2025.05 | — | 61.2 | 67 | 5.8 | |
| ReflectEvoBackbone=Gemma-2-9B-it, Configuration=w/ D-2025.05 | — | 61.2 | 74.8 | 13.6 | |
| ReflectEvoBackbone=Gemma-2-9B-it, Configuration=w/ Dpref2025.05 | — | 61.2 | 75 | 13.8 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=one-stage w/ D+2025.05 | — | 38.2 | 71.2 | 33 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=two-stage w/ D+2025.05 | — | 38.2 | 45.4 | 7.2 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ D+/-2025.05 | — | 38.2 | 63 | 24.8 | |
| ReflectEvoBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ Dpref2025.05 | — | 38.2 | 59.6 | 21.4 | |
| ReflectEvoBackbone=Mistral-7B-Instruct-v0.2, Variant=one-stage w/ D+2025.05 | — | 36.6 | 51.6 | 15 | |
| ReflectEvoBackbone=Mistral-7B-Instruct-v0.2, Variant=two-stage w/ D+2025.05 | — | 36.6 | 71.1 | 34.5 | |
| ReflectEvoBackbone=Mistral-7B-Instruct-v0.2, Variant=w/ D+/-2025.05 | — | 36.6 | 50.2 | 13.6 | |
| ReflectEvoBackbone=Mistral-7B-Instruct-v0.2, Variant=w/ Dpref2025.05 | — | 36.6 | 48.4 | 11.8 | |
| SFT basedBackbone=Gemma-2-9B-it, Configuration=w/ SFT qa pairs2025.05 | — | 74.6 | — | — | |
| SFT basedBackbone=Meta-Llama-3-8B-Instruct, Variant=w/ SFT qa pairs2025.05 | — | 61.6 | — | — | |
| SFT basedBackbone=Mistral-7B-Instruct-v0.2, Variant=w/ SFT qa pairs2025.05 | — | 37.8 | — | — |