Question Answering on DROP (Acc, F1, Avg. Tokens (K))
81.08AccuracySmolagent
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Smolagent2025.10 | 81.08 | 81.08 | 12.01 | |
| OWL2025.10 | 79.85 | 79.85 | 11.47 | |
| SUPERVISORAGENTmodule=SMAS, base_agent=Smolagent2025.10 | 79.8 | 79.8 | 11.34 | |
| MetaAgent2025.10 | 78.16 | 78.16 | 1.43 | |
| CoT SCshots=3-shot2025.10 | 77.72 | 77.72 | 2.73 | |
| Vanilla2025.10 | 76.36 | 76.36 | 0.46 | |
| GRZOModel=Llama3-8B, Optimization Type=ZO2026.06 | 65 | — | — | |
| AdamModel=Llama3-8B, Optimization Type=FO2026.06 | 59.4 | — | — | |
| LoRAModel=Llama3-8B, Optimization Type=FO2026.06 | 58.2 | — | — | |
| FZOOModel=Llama3-8B, Optimization Type=ZO2026.06 | 57.4 | — | — | |
| MeZOModel=Llama3-8B, Optimization Type=ZO2026.06 | 57.1 | — | — | |
| GRZOModel=OPT-13B, Optimization Type=ZO2026.06 | 32.8 | — | — | |
| AdamModel=OPT-13B, Optimization Type=FO2026.06 | 31.3 | — | — | |
| LoRAModel=OPT-13B, Optimization Type=FO2026.06 | 30.9 | — | — | |
| MeZOModel=OPT-13B, Optimization Type=ZO2026.06 | 30.9 | — | — | |
| FZOOModel=OPT-13B, Optimization Type=ZO2026.06 | 28.7 | — | — |