Mathematical Reasoning on AQuA-RAT (test)
83AccuracyaPSF
Evaluation Results
| Method | Links | |
|---|---|---|
| aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 83 | |
| aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 82.5 | |
| GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 82 | |
| Zero-shot CoTWorker Model=Qwen2.5-7B-Instruct, Optimization Protocol=No optimization (p_init)2026.04 | 81.5 | |
| APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 81.5 | |
| CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 81.48 | |
| ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 81.36 | |
| CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 80.99 | |
| GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 80.5 | |
| ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 80.5 | |
| ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 80.31 | |
| APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 80 | |
| ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 79.5 | |
| OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 78.5 | |
| OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 77 | |
| aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 72.5 | |
| aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 71.5 | |
| GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 69 | |
| ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 68.43 | |
| Zero-shot CoTWorker Model=Llama-3.1-8B-Instruct, Optimization Protocol=No optimization (p_init)2026.04 | 68 | |
| GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 68 | |
| OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 67.5 | |
| APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 66 | |
| ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 65.95 | |
| CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 65.86 | |
| CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 65.13 | |
| OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 64.5 | |
| APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 64 | |
| ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 62.5 | |
| ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 53 | |
| MLP ProbeEvaluation Protocol=Probe2026.02 | 50.3 | |
| Teacher 5-shotEvaluation Protocol=5-shot2026.02 | 44.7 | |
| PROBE-KD (MLP)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=MLP2026.02 | 29.4 | |
| SupervisedBackbone=DeBERTa-v3-base, Parameters=86M2026.02 | 29.3 | |
| PROBE-KD (Logistic)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=Logistic2026.02 | 29.3 | |
| PROBE-KD (CCS)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=CCS2026.02 | 28.5 | |
| Label SmoothingBackbone=DeBERTa-v3-base, Parameters=86M2026.02 | 27.6 | |
| Patient-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Patient-KD2026.02 | 27.6 | |
| Feature-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Feature-KD2026.02 | 27.4 | |
| Logit-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Logit-KD2026.02 | 26.6 |