Question Answering on OpenBookQA (OBQA)
94.95AccuracyDirect Fine-tuning
Evaluation results
| Method | Links | |
|---|---|---|
| Direct Fine-tuningrank=1, additional_steps=02026.02 | 94.95 | |
| Direct Fine-tuningrank=1, additional_steps=2002026.02 | 94.95 | |
| Direct Fine-tuningrank=1, additional_steps=7002026.02 | 94.83 | |
| Cont-Squeezerank=128 to 1, additional_steps=02026.02 | 94.71 | |
| Cont-Squeezerank=128 to 1, additional_steps=7002026.02 | 94.71 | |
| In-Squeezerank=128 to 1, schedule=Min steps2026.02 | 94.71 | |
| In-Squeezerank=128 to 1, schedule=Standard2026.02 | 94.59 | |
| Cont-Squeezerank=128 to 1, additional_steps=2002026.02 | 94.47 | |
| UD+-XXLMode=Fully-supervised, Backbone=T5-XXL encoder, Model Parameters=11B2022.11 | 89.2 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+700 steps2026.02 | 89.06 | |
| In-Squeezereduction=128 ... -> 1, strategy=Min steps2026.02 | 88.7 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+200 steps2026.02 | 88.22 | |
| Direct Fine-tuningrank=1, training_steps=+700 steps2026.02 | 88.1 | |
| LogicGazeBackbone=Qwen2.5-7B, Retrieval Status=With Retrieval, Evaluation Protocol=RAG2026.01 | 87.9 | |
| Direct Fine-tuningrank=1, training_steps=+0 steps2026.02 | 87.74 | |
| Cont-Squeezereduction=128 -> 1, training_steps=+0 steps2026.02 | 87.74 | |
| Direct Fine-tuningrank=1, training_steps=+200 steps2026.02 | 87.5 | |
| RankRAGBackbone=Qwen2.5-7B, Retrieval Status=With Retrieval, Evaluation Protocol=RAG2026.01 | 87.5 | |
| In-Squeezereduction=128 ... -> 1, strategy=Standard2026.02 | 87.26 | |
| Tay et al. (2022)Mode=Fully-supervised, Model Parameters=4x of UD (~44B)2022.11 | 87.2 | |
| IterDRAGBackbone=Qwen2.5-7B, Retrieval Status=With Retrieval, Evaluation Protocol=RAG2026.01 | 86.6 | |
| H2OKV cache budget=2562025.12 | 86 | |
| H2OKV cache budget=3842025.12 | 86 | |
| H2OKV cache budget=5122025.12 | 86 | |
| AutoRAGBackbone=Qwen2.5-7B, Retrieval Status=With Retrieval, Evaluation Protocol=RAG2026.01 | 85.1 | |
| FullBackbone=DeepSeek-R1-Distill-Llama-8B2025.12 | 84 | |
| Full2025.12 | 84 | |
| KnormKV cache budget=3842025.12 | 84 | |
| RKVKV cache budget=5122025.12 | 84 | |
| StreamingLLMKV cache budget=3842025.12 | 84 | |
| StreamingLLMKV cache budget=5122025.12 | 84 | |
| Stable-LoRAModel=1.5B2026.03 | 84 | |
| Stable-LoRAModel=3B2026.03 | 84 | |
| AdamWModel=1.5B2026.03 | 83.6 | |
| RQ-RAGBackbone=Qwen2.5-7B, Retrieval Status=With Retrieval, Evaluation Protocol=RAG2026.01 | 83.5 | |
| LoRA-RITEModel=3B2026.03 | 83.4 | |
| LoRA+Model=1.5B2026.03 | 83.2 | |
| AdamWModel=3B2026.03 | 83.2 | |
| SnapKVBackbone=DeepSeek-R1-Distill-Llama-8B, Cache Budget=2562025.12 | 83 | |
| H2OKV cache budget=1282025.12 | 83 | |
| LoRA-RITEModel=1.5B2026.03 | 83 | |
| LoRA+Model=3B2026.03 | 83 | |
| ShapLoRATunable Params=22.8M, Backbone=LLaMA-3 8B2026.01 | 82.9 | |
| RiemannModel=3B2026.03 | 82.6 | |
| RiemannModel=1.5B2026.03 | 82.4 | |
| LogicGazeBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 82.1 | |
| KnormKV cache budget=5122025.12 | 82 | |
| RankRAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 81.8 | |
| AutoLoRATunable Params=23.1M, Backbone=LLaMA-3 8B2026.01 | 81.7 | |
| MOELoRATunable Params=29.9M, Backbone=LLaMA-3 8B2026.01 | 81.7 | |
| DoRATunable Params=22.6M, Backbone=LLaMA-3 8B2026.01 | 81.6 | |
| AdaLoRATunable Params=23.2M, Backbone=LLaMA-3 8B2026.01 | 81.4 | |
| Self-RAGBackbone=Qwen2.5-7B, Retrieval Status=With Retrieval, Evaluation Protocol=RAG2026.01 | 81.2 | |
| IterDRAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 81.2 | |
| LoRATunable Params=22.5M, Backbone=LLaMA-3 8B2026.01 | 81.1 | |
| AutoRAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 80.2 | |
| SSPTunable Params=80.6M, Backbone=LLaMA-3 8B2026.01 | 79.5 | |
| IAPTTunable Params=20.9M, Backbone=LLaMA-3 8B2026.01 | 79.2 | |
| KnormKV cache budget=2562025.12 | 79 | |
| RQ-RAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 79 | |
| Learned-AdapterTunable Params=21.1M, Backbone=LLaMA-3 8B2026.01 | 78.4 | |
| (IA)3Tunable Params=23.1M, Backbone=LLaMA-3 8B2026.01 | 78.1 | |
| FairSteerBackbone=Qwen-3-4B2026.01 | 77.81 | |
| BaseBackbone=Qwen-3-4B2026.01 | 77.79 | |
| LFTFBackbone=Qwen-3-4B2026.01 | 77.77 | |
| BitFitTunable Params=25.2M, Backbone=LLaMA-3 8B2026.01 | 77.2 | |
| RKVKV cache budget=3842025.12 | 77 | |
| SnapKVKV cache budget=2562025.12 | 77 | |
| Self-RAGBackbone=LLaMA3-8B, Retrieval Setting=Yes, Evaluation Protocol=RAG2026.01 | 76.5 | |
| Parallel-AdapterTunable Params=20.9M, Backbone=LLaMA-3 8B2026.01 | 76.3 | |
| P-tuning v2Tunable Params=21.0M, Backbone=LLaMA-3 8B2026.01 | 76.1 | |
| SnapKVKV cache budget=5122025.12 | 76 | |
| MixLoRA# Params (%)=4.3%, ST/MT=ST, Base Model=Gemma-2B2024.04 | 75.8 | |
| MixDoRA# Params (%)=4.3%, ST/MT=ST, Base Model=Gemma-2B2024.04 | 75.4 | |
| PEFTBackbone=Qwen-3-4B, Debias target=religion2026.01 | 74.49 | |
| KnowBiasBackbone=Qwen-3-4B2026.01 | 74.01 | |
| Stable-LoRAModel=1B2026.03 | 73.8 | |
| AdamWModel=1B2026.03 | 73.2 | |
| LoRA+Model=1B2026.03 | 73.2 | |
| SnapKVKV cache budget=1282025.12 | 73 | |
| ReGiFTBackbone=Qwen-3-4B2026.01 | 72.29 | |
| SnapKVKV cache budget=3842025.12 | 72 | |
| StreamingLLMKV cache budget=2562025.12 | 72 | |
| CRISPRBackbone=Qwen-3-4B, Debias target=gender2026.01 | 71.96 | |
| CRISPRBackbone=Qwen-3-4B, Debias target=race2026.01 | 71.47 | |
| LoRA# Params (%)=3.2%, ST/MT=ST, Base Model=Gemma-2B2024.04 | 71.4 | |
| RiemannModel=1B2026.03 | 71.2 | |
| LoRA-RITEModel=1B2026.03 | 71 | |
| MixDoRA# Params (%)=4.3%, ST/MT=MT, Base Model=Gemma-2B2024.04 | 70.6 | |
| DoRA# Params (%)=3.2%, ST/MT=ST, Base Model=Gemma-2B2024.04 | 70.4 | |
| SDBackbone=Qwen-3-4B2026.01 | 70.01 | |
| MixLoRA# Params (%)=4.3%, ST/MT=MT, Base Model=Gemma-2B2024.04 | 70 | |
| BiasEditBackbone=Qwen-3-4B, Debias target=religion2026.01 | 70 | |
| Stable-LoRAModel=0.5B2026.03 | 67 | |
| CRISPRBackbone=Qwen-3-4B, Debias target=religion2026.01 | 66.29 | |
| RKVKV cache budget=2562025.12 | 66 | |
| LoRA-RITEModel=0.5B2026.03 | 66 | |
| RiemannModel=0.5B2026.03 | 65 | |
| LoRA# Params (%)=3.2%, ST/MT=MT, Base Model=Gemma-2B2024.04 | 64.8 | |
| LoRA+Model=0.5B2026.03 | 64.4 |