Question Answering on ARC (test)
90.5AccuracyUNIPROMPT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UNIPROMPTSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot, Prompt Initialization=Task Description, Search Strategy=Greedy2024.06 | 90.5 | — | — | — | |
| EvoPromptSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 89.9 | — | — | — | |
| Llama PromptSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot, Reference=Section 3.22024.06 | 89.7 | — | — | — | |
| EvokeSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 89 | — | — | — | |
| DSPy (BootstrapFewShotWithRandomSearch)Solver Model=GPT-3.5-Turbo, Evaluation Setting=few-shot2024.06 | 87.5 | — | — | — | |
| UNIPROMPTSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot, Prompt Initialization=Task Description, Search Strategy=Beam2024.06 | 86 | — | — | — | |
| DSPy (MIPRO v2)Solver Model=GPT-3.5-Turbo, Evaluation Setting=few-shot2024.06 | 86 | — | — | — | |
| DSPy (MIPRO v2)Solver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 82.8 | — | — | — | |
| Task DescriptionSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 79.7 | — | — | — | |
| CoTSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 79.4 | — | — | — | |
| OPROSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 79.1 | — | — | — | |
| ProTeGiSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 78.8 | — | — | — | |
| Expert PromptSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 78.4 | — | — | — | |
| TextGradSolver Model=GPT-3.5-Turbo, Evaluation Setting=zero-shot2024.06 | 76.5 | — | — | — | |
| Qwen2.5-14B-InstructModel Series=Qwen Series, Number of Parameters=14B, Training Method=Instruct2026.02 | 71.84 | — | — | — | |
| SafeNeuronModel Series=Qwen Series, Number of Parameters=14B, Base Model=Qwen2.5-14B-Instruct2026.02 | 71.5 | — | — | — | |
| JoBSData selection method=JoBS2026.02 | 70.4 | — | — | — | |
| SafeNeuronModel Series=Other Models, Base Model=Phi-42026.02 | 67.66 | — | — | — | |
| Phi-4Model Series=Other Models, Training Method=Instruct2026.02 | 66.47 | — | — | — | |
| Queryable LoRABackbone=Qwen0.5B, Type=Test Accuracy2026.05 | 65.6 | — | — | — | |
| Instruction-Queryable LoRABackbone=Qwen0.5B, Type=Test Accuracy2026.05 | 65.1 | — | — | — | |
| OracleModel Size=Large, Expertise Distribution=Dist. 32026.04 | 64.81 | — | — | — | |
| OracleModel Size=Large2026.04 | 64.81 | — | — | — | |
| OracleModel Size=Large, Noise Type=Uniform2026.04 | 64.81 | — | — | — | |
| BOData selection method=Diversity2026.02 | 63.4 | — | — | — | |
| AutoLoRAData selection method=LESS2026.02 | 63.2 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 62.85 | — | — | — | |
| DARTSData selection method=DoReMi2026.02 | 62.8 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Systematic2026.04 | 62.4 | — | — | — | |
| DefaultData selection method=BO2026.02 | 62.3 | — | — | — | |
| AutoLoRAData selection method=Diversity2026.02 | 62.1 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 61.51 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 61.51 | — | — | — | |
| DefaultData selection method=DoReMi2026.02 | 61.4 | — | — | — | |
| RoBoTData selection method=BO2026.02 | 61.4 | — | — | — | |
| RoBoTData selection method=DoReMi2026.02 | 61.1 | — | — | — | |
| DARTSData selection method=LESS2026.02 | 61 | — | — | — | |
| DefaultData selection method=Diversity2026.02 | 60.6 | — | — | — | |
| DARTSData selection method=BO2026.02 | 60.5 | — | — | — | |
| AutoLoRAData selection method=Default2026.02 | 60.4 | — | — | — | |
| RoBoTData selection method=IF2026.02 | 60.3 | — | — | — | |
| SafeNeuronModel Series=LLaMA Series, Number of Parameters=8B, Base Model=LLaMA-3.2-8B-Instruct2026.02 | 59.98 | — | — | — | |
| AutoLoRAData selection method=BO2026.02 | 59.8 | — | — | — | |
| BOData selection method=DoReMi2026.02 | 59.7 | — | — | — | |
| REALMModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. 32026.04 | 59.64 | — | — | — | |
| noisyModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Systematic2026.04 | 59.64 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 59.64 | — | — | — | |
| REALMModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. 32026.04 | 59.59 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 59.59 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 59.59 | — | — | — | |
| LoRABackbone=Qwen0.5B, Type=Test Accuracy2026.05 | 59.5 | — | — | — | |
| Qwen2.5-7B-InstructModel Series=Qwen Series, Number of Parameters=7B, Training Method=Instruct2026.02 | 59.22 | — | — | — | |
| DefaultData selection method=LESS2026.02 | 59.2 | — | — | — | |
| noisyModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 59.09 | — | — | — | |
| RoBoTData selection method=LESS2026.02 | 58.7 | — | — | — | |
| AutoLoRAData selection method=DoReMi2026.02 | 58.6 | — | — | — | |
| BOData selection method=IF2026.02 | 58.5 | — | — | — | |
| RepLoRABackbone=Qwen0.5B, Type=Test Accuracy2026.05 | 58.5 | — | — | — | |
| REALMModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. 32026.04 | 58.49 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Systematic2026.04 | 58.49 | — | — | — | |
| SafeNeuronModel Series=Qwen Series, Number of Parameters=7B, Base Model=Qwen2.5-7B-Instruct2026.02 | 58.36 | — | — | — | |
| DARTSData selection method=Default2026.02 | 58.1 | — | — | — | |
| noisyModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 57.96 | — | — | — | |
| NOISYModel Size=Large, Expertise Distribution=Dist. 2, Noise Type=Uniform2026.04 | 57.96 | — | — | — | |
| DARTSData selection method=Diversity2026.02 | 57.9 | — | — | — | |
| LLaMA-3.2-8B-InstructModel Series=LLaMA Series, Number of Parameters=8B, Training Method=Instruct2026.02 | 57.59 | — | — | — | |
| REALMModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 57.56 | — | — | — | |
| BOData selection method=BO2026.02 | 57.4 | — | — | — | |
| RoBoTData selection method=Default2026.02 | 56.8 | — | — | — | |
| NOISYModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. 32026.04 | 56.74 | — | — | — | |
| noisyModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Systematic2026.04 | 56.74 | — | — | — | |
| NOISYModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. 32026.04 | 56.14 | — | — | — | |
| noisyModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 56.14 | — | — | — | |
| NOISYModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Uniform2026.04 | 56.14 | — | — | — | |
| NOISYModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. 32026.04 | 55.93 | — | — | — | |
| noisyModel Size=Large, Expertise Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 55.93 | — | — | — | |
| BOData selection method=LESS2026.02 | 55.9 | — | — | — | |
| RoBoTData selection method=Diversity2026.02 | 55.7 | — | — | — | |
| AutoLoRAData selection method=IF2026.02 | 55.1 | — | — | — | |
| DefaultData selection method=Default2026.02 | 54.7 | — | — | — | |
| DoRABackbone=Qwen0.5B, Type=Test Accuracy2026.05 | 54.5 | — | — | — | |
| DARTSData selection method=IF2026.02 | 54.3 | — | — | — | |
| NOISYModel Size=Large, Expertise Distribution=Dist. 1, Noise Type=Uniform2026.04 | 53.68 | — | — | — | |
| Qwen2.5-3B-InstructModel Series=Qwen Series, Number of Parameters=3B, Training Method=Instruct2026.02 | 52.99 | — | — | — | |
| DefaultData selection method=IF2026.02 | 52.8 | — | — | — | |
| BOData selection method=Default2026.02 | 52.6 | — | — | — | |
| SafeNeuronModel Series=Qwen Series, Number of Parameters=3B, Base Model=Qwen2.5-3B-Instruct2026.02 | 52.13 | — | — | — | |
| SafeNeuronModel Series=Other Models, Number of Parameters=7B, Base Model=Gemma-7B-Instruct2026.02 | 51.96 | — | — | — | |
| SafeNeuronModel Series=LLaMA Series, Number of Parameters=3B, Base Model=LLaMA-3.2-3B-Instruct2026.02 | 49.74 | — | — | — | |
| RLHF-SafetyModel Series=Qwen Series, Number of Parameters=1.5B, Base Model=Qwen2.5-1.5B-Instruct2026.02 | 49.57 | — | — | — | |
| SafeNeuronModel Series=Qwen Series, Number of Parameters=1.5B, Base Model=Qwen2.5-1.5B-Instruct2026.02 | 49.57 | — | — | — | |
| SN-TuneModel Series=Qwen Series, Number of Parameters=1.5B, Base Model=Qwen2.5-1.5B-Instruct2026.02 | 49.4 | — | — | — | |
| Qwen2.5-1.5B-InstructModel Series=Qwen Series, Number of Parameters=1.5B, Training Method=Instruct2026.02 | 49.23 | — | — | — | |
| Gemma-7B-InstructModel Series=Other Models, Number of Parameters=7B, Training Method=Instruct2026.02 | 48.55 | — | — | — | |
| LLaMA-3.2-3B-InstructModel Series=LLaMA Series, Number of Parameters=3B, Training Method=Instruct2026.02 | 47.87 | — | — | — | |
| OracleModel Size=Base, Expertise Distribution=Dist. 32026.04 | 47.47 | — | — | — | |
| OracleModel Size=Base2026.04 | 47.47 | — | — | — | |
| OracleModel Size=Base, Noise Type=Uniform2026.04 | 47.47 | — | — | — | |
| REALMModel Size=Base, Expertise Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 46.11 | — | — | — | |
| REALMModel Size=Base, Expertise Distribution=Dist. 2, Noise Type=Systematic2026.04 | 45.85 | — | — | — |