STEM Question Answering on TheoremQA
76.2CorrectnessMOCHA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MOCHAEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 76.2 | — | |
| ProTeGiEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 69 | — | |
| TextGradEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 67.2 | — | |
| GEPAEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 65.6 | — | |
| Seed SkillEvaluation Backbone=Claude Haiku 4.5, Rollout Budget=1000, Number of Seeds=52026.05 | 53.4 | — | |
| Full TextBackbone Model=Qwen3-8B2026.06 | 0.6881 | 935 | |
| SKIM (Adaptive)Backbone Model=Qwen3-8B2026.06 | 0.6787 | 727 | |
| LLMLingua-2 (L)Backbone Model=Qwen3-8B2026.06 | 0.6613 | 710 | |
| SKIM (Fix-512)Backbone Model=Qwen3-8B2026.06 | 0.6479 | 509 | |
| LLMLingua-2 (M)Backbone Model=Qwen3-8B2026.06 | 0.6292 | 535 | |
| SKIM (Fix-256)Backbone Model=Qwen3-8B2026.06 | 0.6024 | 256 | |
| LLMLingua-2 (S)Backbone Model=Qwen3-8B2026.06 | 0.5649 | 281 | |
| NaiveBackbone Model=Qwen3-8B2026.06 | 0.5207 | 0 | |
| Full TextBackbone Model=Phi-42026.06 | 0.5114 | 921 | |
| LLMLingua-2 (L)Backbone Model=Phi-42026.06 | 0.4873 | 693 | |
| SKIM (Adaptive)Backbone Model=Phi-42026.06 | 0.4873 | 682 | |
| SKIM (Fix-512)Backbone Model=Phi-42026.06 | 0.4833 | 509 | |
| LLMLingua-2 (S)Backbone Model=Phi-42026.06 | 0.4739 | 272 | |
| LLMLingua-2 (M)Backbone Model=Phi-42026.06 | 0.4565 | 521 | |
| NaiveBackbone Model=Phi-42026.06 | 0.4538 | 0 | |
| SKIM (Fix-256)Backbone Model=Phi-42026.06 | 0.4378 | 256 | |
| ICAEBackbone Model=Qwen3-8B2026.06 | 0.2838 | 256 | |
| ICAEBackbone Model=Phi-42026.06 | 0.1861 | 256 | |
| 500xCompressorBackbone Model=Phi-42026.06 | 0.1513 | 256 | |
| 500xCompressorBackbone Model=Qwen3-8B2026.06 | 0.1432 | 256 |