List your company
ResearchFormerly WizWand

Arithmetic Reasoning on SingleEq (test)

0.971AccuracyDoRA + MoDEK
0.472840.602170.73150.86083Oct 28, 2022Feb 24, 2023Jun 24, 2023Oct 22, 2023Feb 19, 2024Jun 18, 2024Oct 16, 2024
Updated 4mo ago

Evaluation results

MethodLinks
DoRA + MoDEKBackbone=LLaMA3-8B2024.100.971
DoRABackbone=LLaMA3-8B2024.100.969
LoRAALL + MoDEKBackbone=LLaMA3-8B2024.100.965
LoRAALLBackbone=LLaMA3-8B2024.100.963
LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA3-8B2024.100.96
LoRA¬KBackbone=LLaMA3-8B2024.100.957
REFTBackbone=LLaMA3-8B2024.100.94
CoReBackbone=GPT-J 12B, Evaluation Protocol=Zero-shot2022.100.795
UNITDEPEvaluation Protocol=Fine-tune2022.100.723
DoRA + MoDEKBackbone=LLaMA2-7B2024.100.676
DoRABackbone=LLaMA2-7B2024.100.674
LoRAALL + MoDEKBackbone=LLaMA2-7B2024.100.673
LoRAALLBackbone=LLaMA2-7B2024.100.669
LoRA¬K + MoDEK (+0.04%)Backbone=LLaMA2-7B2024.100.668
LoRA¬KBackbone=LLaMA2-7B2024.100.661
REFTBackbone=LLaMA2-7B2024.100.638
Self-ConsistencyBackbone=GPT-J 6B, Evaluation Protocol=Zero-shot2022.100.602
Generator onlyBackbone=GPT-J 6B, Evaluation Protocol=Zero-shot2022.100.492