Question Answering on ARC-Easy (test)
97.2AccuracyMLP Probe
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MLP ProbeEvaluation Protocol=Probe2026.02 | 97.2 | — | |
| Teacher 5-shotEvaluation Protocol=5-shot2026.02 | 96.6 | — | |
| PROBE-KD (MLP)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=MLP2026.02 | 75.1 | — | |
| Patient-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Patient-KD2026.02 | 74.6 | — | |
| Logit-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Logit-KD2026.02 | 74.4 | — | |
| PROBE-KD (Logistic)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=Logistic2026.02 | 74.3 | — | |
| PROBE-KD (CCS)Backbone=DeBERTa-v3-base, Parameters=86M, Probe Type=CCS2026.02 | 74.1 | — | |
| Label SmoothingBackbone=DeBERTa-v3-base, Parameters=86M2026.02 | 73.9 | — | |
| SupervisedBackbone=DeBERTa-v3-base, Parameters=86M2026.02 | 73.5 | — | |
| Feature-KDBackbone=DeBERTa-v3-base, Parameters=86M, Distillation Strategy=Feature-KD2026.02 | 69.7 | — | |
| AMOOptimizer=AMO, Model=Llama3.1-1.4B, Few-shot=02026.05 | 66.46 | — | |
| Muon-PEOptimizer=Muon-PE, Model=Llama3.1-1.4B, Few-shot=02026.05 | 66.41 | — | |
| AdamWOptimizer=AdamW, Model=Llama3.1-1.4B, Few-shot=02026.05 | 65.84 | — | |
| ROOTOptimizer=ROOT, Model=Llama3.1-1.4B, Few-shot=02026.05 | 65.74 | — | |
| Muon-CANSOptimizer=Muon-CANS, Model=Llama3.1-1.4B, Few-shot=02026.05 | 65.57 | — | |
| Muon-KJOptimizer=Muon-KJ, Model=Llama3.1-1.4B, Few-shot=02026.05 | 65.32 | — | |
| Muon-YouOptimizer=Muon-You, Model=Llama3.1-1.4B, Few-shot=02026.05 | 64.81 | — | |
| Muon-TurboOptimizer=Muon-Turbo, Model=Llama3.1-1.4B, Few-shot=02026.05 | 64.77 | — | |
| AdaMuonOptimizer=AdaMuon, Model=Llama3.1-1.4B, Few-shot=02026.05 | 63.17 | — | |
| Muon-YouOptimizer=Muon-You, Model=Llama3.1-760M, Few-shot=02026.05 | 60.27 | — | |
| AdamWOptimizer=AdamW, Model=Llama3.1-760M, Few-shot=02026.05 | 59.89 | — | |
| Muon-PEOptimizer=Muon-PE, Model=Llama3.1-760M, Few-shot=02026.05 | 59.89 | — | |
| ROOTOptimizer=ROOT, Model=Llama3.1-760M, Few-shot=02026.05 | 59.85 | — | |
| Muon-TurboOptimizer=Muon-Turbo, Model=Llama3.1-760M, Few-shot=02026.05 | 59.81 | — | |
| Muon-KJOptimizer=Muon-KJ, Model=Llama3.1-760M, Few-shot=02026.05 | 59.68 | — | |
| Muon-CANSOptimizer=Muon-CANS, Model=Llama3.1-760M, Few-shot=02026.05 | 59.51 | — | |
| AMOOptimizer=AMO, Model=Llama3.1-760M, Few-shot=02026.05 | 59.43 | — | |
| AdaMuonOptimizer=AdaMuon, Model=Llama3.1-760M, Few-shot=02026.05 | 58.29 | — | |
| GQA (Original)Rank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 55.89 | — | |
| CARE-U (OURS)Rank=512, KV Save=50.00, Model=Llama-3.1-8B-Instruct2026.03 | 52.73 | — | |
| SVD-LLM V2Rank=512, KV Save=50.00, Model=Llama-3.1-8B-Instruct2026.03 | 52.39 | — | |
| GQA (Original)Rank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 50.34 | — | |
| MuSeModel Scale=1B, Training Data=scientific PDFs2025.09 | 48.7 | 43.6 | |
| CUDNNModel Scale=1B, Training Data=scientific PDFs2025.09 | 47.9 | 43.1 | |
| ASVDRank=512, KV Save=50.00, Model=Llama-3.1-8B-Instruct2026.03 | 46.33 | — | |
| CARE-E (OURS)Rank=512, KV Save=50.00, Model=Llama-3.1-8B-Instruct2026.03 | 42.41 | — | |
| PGM 8 / 8distillation=Before2025.05 | 40.4 | — | |
| PGM 6 / 6 (1024)distillation=Before2025.05 | 39.98 | — | |
| Baseline GPT-2shot=0-shot, protocol=standalone evaluation2026.04 | 38.93 | 35.61 | |
| CARE-E (OURS)Rank=256, KV Save=75.00, Model=Llama-3.1-8B-Instruct2026.03 | 38.48 | — | |
| MDLMdistillation=Before2025.05 | 37.88 | — | |
| PGM 8 / 8distillation=After Distillation (SDTT)2025.05 | 37.42 | — | |
| GMT v7shot=0-shot, protocol=standalone evaluation2026.04 | 37.04 | 33.84 | |
| PGM 6 / 6 (1024)distillation=After Distillation (SDTT)2025.05 | 36.7 | — | |
| SVD-LLM V2Rank=256, KV Save=75.00, Model=Llama-3.1-8B-Instruct2026.03 | 36.26 | — | |
| CARE-U (OURS)Rank=256, KV Save=75.00, Model=Llama-3.1-8B-Instruct2026.03 | 36.09 | — | |
| MDLMdistillation=After Distillation (SDTT)2025.05 | 33.8 | — | |
| CARE-E (OURS)Rank=128, KV Save=87.50, Model=Qwen3-4B-Instruct-25072026.03 | 30.29 | — | |
| Palu(SVD)Rank=512, KV Save=50.00, Model=Llama-3.1-8B-Instruct2026.03 | 28.16 | — | |
| SVD-LLM V2Rank=128, KV Save=87.50, Model=Llama-3.1-8B-Instruct2026.03 | 27.56 | — | |
| MHA2MLARank=256, KV Save=75.00, Model=Llama-3.1-8B-Instruct2026.03 | 27.47 | — | |
| CARE-U (OURS)Rank=128, KV Save=87.50, Model=Qwen3-4B-Instruct-25072026.03 | 27.47 | — | |
| ASVDRank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 27.39 | — | |
| CARE-E (OURS)Rank=128, KV Save=87.50, Model=Llama-3.1-8B-Instruct2026.03 | 27.3 | — | |
| CARE-U (OURS)Rank=128, KV Save=87.50, Model=Llama-3.1-8B-Instruct2026.03 | 26.71 | — | |
| Palu(SVD)Rank=128, KV Save=87.50, Model=Qwen3-4B-Instruct-25072026.03 | 26.02 | — | |
| MHA2MLARank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 25.94 | — | |
| MHA2MLARank=512, KV Save=50.00, Model=Llama-3.1-8B-Instruct2026.03 | 25.94 | — | |
| Palu(SVD)Rank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 25.77 | — | |
| MHA2MLARank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 25.68 | — | |
| Palu(SVD)Rank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 25.34 | — | |
| MHA2MLARank=128, KV Save=87.50, Model=Llama-3.1-8B-Instruct2026.03 | 25 | — | |
| ASVDRank=128, KV Save=87.50, Model=Llama-3.1-8B-Instruct2026.03 | 25 | — | |
| CARE-U (OURS)Rank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 23.89 | — | |
| Palu(SVD)Rank=128, KV Save=87.50, Model=Llama-3.1-8B-Instruct2026.03 | 23.89 | — | |
| ASVDRank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 23.81 | — | |
| MHA2MLARank=128, KV Save=87.50, Model=Qwen3-4B-Instruct-25072026.03 | 23.81 | — | |
| CARE-E (OURS)Rank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 23.63 | — | |
| SVD-LLM V2Rank=64, KV Save=93.75, Model=Llama-3.1-8B-Instruct2026.03 | 23.63 | — | |
| CARE-U (OURS)Rank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 23.46 | — | |
| CARE-E (OURS)Rank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 23.29 | — | |
| SVD-LLM V2Rank=64, KV Save=93.75, Model=Qwen3-4B-Instruct-25072026.03 | 23.12 | — | |
| Palu(SVD)Rank=256, KV Save=75.00, Model=Llama-3.1-8B-Instruct2026.03 | 22.61 | — | |
| ASVDRank=256, KV Save=75.00, Model=Llama-3.1-8B-Instruct2026.03 | 21.76 | — |