Question Answering on DROP (F1 score)
87.5F1 ScoreOneFlow
Evaluation Results
| Method | Links | |
|---|---|---|
| OneFlowModel=Claude 3.5 Haiku2026.01 | 87.5 | |
| AFlowModel=Claude 3.5 Haiku2026.01 | 86.8 | |
| AFlowMethod category=Automatically designed multi-agent frameworks, Executor LLM=GPT-4o-mini, Execution protocol=multi-agent2026.01 | 83.1 | |
| ReConcileBackbone=GPT-4o-mini2026.05 | 82.1 | |
| OneFlowMethod category=Single-LLM implementation, Executor LLM=GPT-4o-mini, Execution protocol=single-agent execution2026.01 | 81.7 | |
| LLM-DebateBackbone=GPT-4o-mini2026.05 | 81.4 | |
| OneFlowMethod category=Automatically designed multi-agent frameworks, Executor LLM=GPT-4o-mini, Execution protocol=multi-agent2026.01 | 81.1 | |
| AFlowMethod category=Single-LLM implementation, Executor LLM=GPT-4o-mini, Execution protocol=single-agent execution2026.01 | 81.1 | |
| AFlow*Backbone=Mixed2026.05 | 80.6 | |
| CoT SCMethod category=Manual baselines, Executor LLM=GPT-4o-mini, Execution protocol=standard, shots=5-shot2026.01 | 79.4 | |
| CoTMethod category=Manual baselines, Executor LLM=GPT-4o-mini, Execution protocol=standard2026.01 | 78.9 | |
| MultiPersonaMethod category=Manual baselines, Executor LLM=GPT-4o-mini, Execution protocol=standard2026.01 | 78.9 | |
| CoT SCBackbone=GPT-4o-mini, Prompting Setting=5-shot2026.05 | 78.8 | |
| CoTBackbone=GPT-4o-mini2026.05 | 78.5 | |
| MedPromptBackbone=GPT-4o-mini2026.05 | 78 | |
| AGENTCO-OPBackbone=GPT-4o-mini2026.05 | 77.2 | |
| ADASBackbone=GPT-4o-mini2026.05 | 76.6 | |
| MultiPersonaBackbone=GPT-4o-mini2026.05 | 74.4 | |
| Self RefineBackbone=GPT-4o-mini2026.05 | 70.2 | |
| AFlowBackbone=GPT-4o-mini2026.05 | 68.9 | |
| IOBackbone=GPT-4o-mini2026.05 | 68.3 | |
| IOMethod category=Manual baselines, Executor LLM=GPT-4o-mini, Execution protocol=standard2026.01 | 66.2 | |
| IOModel=Claude 3.5 Haiku2026.01 | 64 | |
| MeZOModel Scale=OPT-6.7B, Privacy Level=non-private, Few-shot shots (k)=10002025.06 | 28.8 | |
| DPZeroModel Scale=OPT-6.7B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 28.4 | |
| DP-GRAPEModel Scale=OPT-6.7B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 28.2 | |
| DP-GRAPEModel Scale=OPT-6.7B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 27.8 | |
| DPZeroModel Scale=OPT-6.7B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 27.6 | |
| DP-AdamModel Scale=OPT-2.7B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 26.3 | |
| DP-GRAPEModel Scale=OPT-1.3B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 26.1 | |
| DP-AdamModel Scale=OPT-1.3B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 25.9 | |
| MeZOModel Scale=OPT-2.7B, Privacy Level=non-private, Few-shot shots (k)=10002025.06 | 25.5 | |
| DP-GRAPEModel Scale=OPT-2.7B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 25.5 | |
| DP-AdamModel Scale=OPT-1.3B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 25.2 | |
| DP-GRAPEModel Scale=OPT-1.3B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 25 | |
| DP-AdamModel Scale=OPT-2.7B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 24.9 | |
| DPZeroModel Scale=OPT-1.3B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 24.7 | |
| DPZeroModel Scale=OPT-2.7B, Privacy Budget (epsilon)=6, Few-shot shots (k)=10002025.06 | 24.6 | |
| MeZOModel Scale=OPT-1.3B, Privacy Level=non-private, Few-shot shots (k)=10002025.06 | 24.4 | |
| DP-GRAPEModel Scale=OPT-2.7B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 24.1 | |
| DPZeroModel Scale=OPT-1.3B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 23.9 | |
| DPZeroModel Scale=OPT-2.7B, Privacy Budget (epsilon)=2, Few-shot shots (k)=10002025.06 | 23.1 | |
| Zero-ShotModel Scale=OPT-6.7B, Evaluation Protocol=zero-shot2025.06 | 17.8 | |
| Zero-ShotModel Scale=OPT-1.3B, Evaluation Protocol=zero-shot2025.06 | 11.1 | |
| Zero-ShotModel Scale=OPT-2.7B, Evaluation Protocol=zero-shot2025.06 | 9.7 |