List your company
ResearchFormerly WizWand

Mathematical Reasoning on AQuA-RAT (test)

83AccuracyaPSF
24.34439.57254.870.028Feb 18, 2026Feb 26, 2026Mar 6, 2026Mar 14, 2026Mar 22, 2026Mar 30, 2026Apr 8, 2026
Updated 6mo ago

Evaluation results

MethodLinks
aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0483
aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0482.5
GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0482
Zero-shot CoTWorker Model=Qwen2.5-7B-Instruct, Optimization Protocol=No optimization (p_init)2026.0481.5
APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0481.5
CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0481.48
ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0481.36
CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0480.99
GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0480.5
ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0480.5
ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0480.31
APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0480
ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0479.5
OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.0478.5
OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.0477
aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0472.5
aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0471.5
GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0469
ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0468.43
Zero-shot CoTWorker Model=Llama-3.1-8B-Instruct, Optimization Protocol=No optimization (p_init)2026.0468
GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0468
OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0467.5
APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0466
ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0465.95
CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0465.86
CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0465.13
OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0464.5
APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0464
ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.0462.5
ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.0453
MLP ProbeEvaluation Protocol=Probe2026.0250.3
Teacher 5-shotEvaluation Protocol=5-shot2026.0244.7
PROBE-KD (MLP)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=MLP2026.0229.4
SupervisedBackbone=DeBERTa-v3-base, Parameters=86M2026.0229.3
PROBE-KD (Logistic)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=Logistic2026.0229.3
PROBE-KD (CCS)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=CCS2026.0228.5
Label SmoothingBackbone=DeBERTa-v3-base, Parameters=86M2026.0227.6
Patient-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Patient-KD2026.0227.6
Feature-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Feature-KD2026.0227.4
Logit-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Logit-KD2026.0226.6