Logical Reasoning on GeoShape BBEH
43AccuracyMemAPO
Evaluation results
| Method | Links | |
|---|---|---|
| MemAPOBackbone=GPT-4o-mini2026.03 | 43 | |
| RaRBackbone=GPT-4o-mini2026.03 | 30 | |
| SPOBackbone=Qwen3-8B2026.03 | 25 | |
| Step-BackBackbone=GPT-4o-mini2026.03 | 25 | |
| MemAPOBackbone=Qwen3-8B2026.03 | 24 | |
| SPOBackbone=GPT-4o-mini2026.03 | 24 | |
| TextGradBackbone=GPT-4o-mini2026.03 | 23 | |
| OPROBackbone=GPT-4o-mini2026.03 | 18 | |
| CoTBackbone=GPT-4o-mini2026.03 | 17 | |
| TextGradBackbone=Qwen3-8B2026.03 | 14 | |
| ProTeGiBackbone=GPT-4o-mini2026.03 | 14 | |
| IOBackbone=GPT-4o-mini2026.03 | 13 | |
| PromptBreederBackbone=GPT-4o-mini2026.03 | 13 | |
| PromptBreederBackbone=Qwen3-8B2026.03 | 10 | |
| OPROBackbone=Qwen3-8B2026.03 | 9 | |
| Step-BackBackbone=Qwen3-8B2026.03 | 8 | |
| ProTeGiBackbone=Qwen3-8B2026.03 | 7 | |
| IOBackbone=Qwen3-8B2026.03 | 6 | |
| CoTBackbone=Qwen3-8B2026.03 | 6 | |
| RaRBackbone=Qwen3-8B2026.03 | 3 |