Logical Reasoning on Boolean BBEH
100AccuracyPSAO_LLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PSAO_LLMModel=GPT-4o2026.05 | 100 | — | |
| PromptAgentBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PromptAgent2026.05 | 100 | — | |
| PSAO_LLMBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=PSAO_LLM2026.05 | 100 | — | |
| PSAO_LLMLLM=GPT-4o2026.05 | 100 | — | |
| ProTeGiBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=ProTeGi2026.05 | 99.73 | — | |
| BaselineBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=Baseline2026.05 | 99.67 | — | |
| COPROModel=GPT-4o2026.05 | 99.33 | — | |
| BaselineLLM=GPT-4o2026.05 | 99.33 | — | |
| COPROLLM=GPT-4o2026.05 | 99.33 | — | |
| GEPABackbone Model=Gemini-2-Flash, Prompt Optimization Technique=GEPA2026.05 | 99.3 | — | |
| MIPROModel=GPT-4o2026.05 | 99.2 | — | |
| ProTeGiLLM=GPT-4o2026.05 | 99.2 | — | |
| PromptAgentLLM=GPT-4o2026.05 | 99.2 | — | |
| MIPROLLM=GPT-4o2026.05 | 99.2 | — | |
| GEPALLM=GPT-4o2026.05 | 98.9 | — | |
| COPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=COPRO2026.05 | 98.4 | — | |
| MIPROBackbone Model=Gemini-2-Flash, Prompt Optimization Technique=MIPRO2026.05 | 98.27 | — | |
| CoTModel=Gemma32026.03 | 25.5 | 6,010 | |
| CoTModel=Qwen32026.03 | 24 | 4,977 | |
| CoTModel=Llama3.12026.03 | 18 | 5,055 | |
| ToTModel=Llama3.12026.03 | 6 | 12,450 | |
| DPTSModel=Llama3.12026.03 | 5 | 9,452 | |
| DSTModel=Llama3.12026.03 | 4.5 | 5,752 | |
| ToTModel=Gemma32026.03 | 4.5 | 13,817 | |
| ToTModel=Qwen32026.03 | 4 | 11,751 | |
| DSTModel=Gemma32026.03 | 3.5 | 6,874 | |
| DPTSModel=Qwen32026.03 | 3 | 9,955 | |
| DSTModel=Qwen32026.03 | 3 | 5,953 | |
| DPTSModel=Gemma32026.03 | 2 | 11,501 |