List your company
ResearchFormerly WizWand

Multiple Choice Classification on OpenBookQA, ARC-Easy, WinoGrande, HellaSwag, PIQA, and MathQA

35Accuracy (OpenBookQA)Original
19.423.4527.531.55Nov 14, 2024
Updated 5mo ago

Evaluation results

MethodLinks
OriginalBackbone Model=Llama-3.1-8B, Compression Rate=None2024.1135807360804061
OriginalBackbone Model=Llama-7B, Compression Rate=None2024.1134757057792757
SVD-LLM V2Backbone Model=Llama-3.1-8B, Compression Rate=20%2024.1133797058773659
FiPSBackbone Model=Llama-3.1-8B, Compression Rate=20%2024.1133797259783860
SVD-LLM V2Backbone Model=Llama-7B, Compression Rate=20%2024.1132727052752454
FiPSBackbone Model=Llama-7B, Compression Rate=20%2024.1132727056782656
SVD-LLMBackbone Model=Llama-7B, Compression Rate=20%2024.1131716849712252
ASVDBackbone Model=Llama-7B, Compression Rate=20%2024.1129536441681745
SVD-LLMBackbone Model=Llama-3.1-8B, Compression Rate=20%2024.1129776451723054
Basis SharingBackbone Model=Llama-7B, Compression Rate=20%2024.1128666646712550
ASVDBackbone Model=Llama-3.1-8B, Compression Rate=20%2024.1120596141693047