Arithmetic Reasoning on SingleEq (test)
0.971AccuracyDoRA + MoDEK
Evaluation Results
| Method | Links | |
|---|---|---|
| DoRA + MoDEKBackbone=LLaMA3-8B2024.10 | 0.971 | |
| DoRABackbone=LLaMA3-8B2024.10 | 0.969 | |
| LoRAALL + MoDEKBackbone=LLaMA3-8B2024.10 | 0.965 | |
| LoRAALLBackbone=LLaMA3-8B2024.10 | 0.963 | |
| LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA3-8B2024.10 | 0.96 | |
| LoRA¬KBackbone=LLaMA3-8B2024.10 | 0.957 | |
| REFTBackbone=LLaMA3-8B2024.10 | 0.94 | |
| CoReBackbone=GPT-J 12B, Evaluation Protocol=Zero-shot2022.10 | 0.795 | |
| UNITDEPEvaluation Protocol=Fine-tune2022.10 | 0.723 | |
| DoRA + MoDEKBackbone=LLaMA2-7B2024.10 | 0.676 | |
| DoRABackbone=LLaMA2-7B2024.10 | 0.674 | |
| LoRAALL + MoDEKBackbone=LLaMA2-7B2024.10 | 0.673 | |
| LoRAALLBackbone=LLaMA2-7B2024.10 | 0.669 | |
| LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA2-7B2024.10 | 0.668 | |
| LoRA¬KBackbone=LLaMA2-7B2024.10 | 0.661 | |
| REFTBackbone=LLaMA2-7B2024.10 | 0.638 | |
| Self-ConsistencyBackbone=GPT-J 6B, Evaluation Protocol=Zero-shot2022.10 | 0.602 | |
| Generator onlyBackbone=GPT-J 6B, Evaluation Protocol=Zero-shot2022.10 | 0.492 |