List your company
ResearchFormerly WizWand

Question Answering on AGIEval

32.11AccuracyLLMBRACES
20.690823.655426.6229.5846Mar 20, 2025
Updated 8mo ago

Evaluation results

MethodLinks
LLMBRACESBase Model=Qwen2.5-1.5B, Rank (r)=32, Param.=1.2M, Param. (%)=0.08, Zero-shot=true2025.0332.11
LLMBRACESBase Model=Qwen2.5-1.5B, Rank (r)=16, Param.=0.6M, Param. (%)=0.04, Zero-shot=true2025.0331.67
LoRABase Model=Qwen2.5-1.5B, Rank (r)=16, Param.=2.2M, Param. (%)=0.14, Zero-shot=true2025.0330.43
LLMBRACESBase Model=Llama3-8B, Rank (r)=16, Param.=2.1M, Param. (%)=0.03, Zero-shot=true2025.0329.47
LLMBRACESBase Model=Llama3-8B, Rank (r)=32, Param.=4.2M, Param. (%)=0.05, Zero-shot=true2025.0329.23
LoRABase Model=Qwen2.5-1.5B, Rank (r)=32, Param.=4.4M, Param. (%)=0.28, Zero-shot=true2025.0329.18
LoRABase Model=Llama3-8B, Rank (r)=32, Param.=13.6M, Param. (%)=0.17, Zero-shot=true2025.0325.8
LoRABase Model=Llama3-8B, Rank (r)=16, Param.=6.8M, Param. (%)=0.08, Zero-shot=true2025.0324.61
LLMBRACESBase Model=Llama2-7B, Rank (r)=32, Param.=4.2M, Param. (%)=0.06, Zero-shot=true2025.0323.94
LLMBRACESBase Model=Llama2-7B, Rank (r)=16, Param.=2.1M, Param. (%)=0.03, Zero-shot=true2025.0323.83
LoRABase Model=Llama2-7B, Rank (r)=32, Param.=16.8M, Param. (%)=0.25, Zero-shot=true2025.0322.66
LoRABase Model=Llama2-7B, Rank (r)=16, Param.=8.4M, Param. (%)=0.12, Zero-shot=true2025.0321.13