Zero-shot Question Answering on AveQA
57.7AccuracyAMP
Evaluation Results
| Method | Links | |
|---|---|---|
| AMPModel Scale=OPT-30B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 57.7 | |
| ARB-RCModel Scale=OPT-30B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 57.11 | |
| AMPModel Scale=OPT-13B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 55.06 | |
| ARB-RCModel Scale=OPT-13B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 55.01 | |
| AMPModel Scale=OPT-6.7B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 53.33 | |
| ARB-RCModel Scale=OPT-6.7B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 52.58 | |
| ARB-XModel Scale=OPT-30B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 51.77 | |
| BiLLMModel Scale=OPT-30B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 49.61 | |
| ARB-XModel Scale=OPT-13B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 49.19 | |
| AMPModel Scale=OPT-2.7B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 49.03 | |
| ARB-RCModel Scale=OPT-2.7B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 48.25 | |
| BiLLMModel Scale=OPT-13B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 47 | |
| ARB-XModel Scale=OPT-6.7B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 46.57 | |
| AMPModel Scale=OPT-1.3B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 45.76 | |
| ARB-RCModel Scale=OPT-1.3B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 45.22 | |
| PB-LLMModel Scale=OPT-30B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.72025.12 | 43.7 | |
| ARB-XModel Scale=OPT-2.7B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 42.21 | |
| ARB-XModel Scale=OPT-1.3B, Output Alignment (OA)=true, Block Size=128, Weight Bits=1.112025.12 | 40.52 | |
| BiLLMModel Scale=OPT-2.7B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 40.44 | |
| BiLLMModel Scale=OPT-1.3B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 38.89 | |
| BiLLMModel Scale=OPT-6.7B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.112025.12 | 38.27 | |
| PB-LLMModel Scale=OPT-13B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.72025.12 | 37.4 | |
| PB-LLMModel Scale=OPT-2.7B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.72025.12 | 37.06 | |
| PB-LLMModel Scale=OPT-1.3B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.72025.12 | 36.6 | |
| PB-LLMModel Scale=OPT-6.7B, Weight Alignment (WA)=true, Block Size=128, Weight Bits=1.72025.12 | 35.95 |