Knowledge & Reasoning on TriviaQA
71.58Exact MatchLLaMA3-8B
Evaluation results
| Method | Links | |
|---|---|---|
| LLaMA3-8BSpike=✗, Bit=-, T x D=-, R=-, FLOPs (T)=7.97, Power (J)=36.672026.04 | 71.58 | |
| LLaMA2-13BSpike=✗, Bit=-, T x D=-, R=-, FLOPs (T)=13.42, Power (J)=61.742026.04 | 70.45 | |
| SDLLMBackbone=LLaMA3-8B, Spike=✓, Bit=6-1.58, T x D=1 × 32, R=0.213, FLOPs (T)=5.09, Power (J)=4.582026.04 | 70.26 | |
| SDLLMBackbone=LLaMA2-13B, Spike=✓, Bit=6-1.58, T x D=1 × 32, R=0.217, FLOPs (T)=8.74, Power (J)=7.862026.04 | 69.84 | |
| QuaRotBackbone=LLaMA2-13B, Spike=✗, Bit=6-6, T x D=-, R=-, FLOPs (T)=3.77, Power (J)=17.362026.04 | 69.14 | |
| SDLLMBackbone=LLaMA2-13B, Spike=✓, Bit=4-1.58, T x D=1 × 16, R=0.215, FLOPs (T)=2.89, Power (J)=2.602026.04 | 66.45 | |
| LLaMA2-7BSpike=✗, Bit=-, T x D=-, R=-, FLOPs (T)=6.91, Power (J)=31.772026.04 | 64.14 | |
| SDLLMBackbone=LLaMA2-7B, Spike=✓, Bit=6-1.58, T x D=1 × 32, R=0.222, FLOPs (T)=4.60, Power (J)=4.142026.04 | 64.05 | |
| QuaRotBackbone=LLaMA2-7B, Spike=✗, Bit=6-6, T x D=-, R=-, FLOPs (T)=1.94, Power (J)=8.942026.04 | 62.72 | |
| SDLLMBackbone=LLaMA2-13B, Spike=✓, Bit=4-1.58, T x D=1 × 8, R=0.209, FLOPs (T)=1.40, Power (J)=1.262026.04 | 62.44 | |
| SDLLMBackbone=LLaMA3-8B, Spike=✓, Bit=4-1.58, T x D=1 × 16, R=0.211, FLOPs (T)=1.68, Power (J)=1.512026.04 | 60.4 | |
| SDLLMBackbone=LLaMA2-7B, Spike=✓, Bit=4-1.58, T x D=1 × 16, R=0.221, FLOPs (T)=1.53, Power (J)=1.372026.04 | 58.86 | |
| QuaRotBackbone=LLaMA2-13B, Spike=✗, Bit=4-4, T x D=-, R=-, FLOPs (T)=1.68, Power (J)=7.722026.04 | 52.25 | |
| Engram-40BShots=5-shot2026.01 | 51.8 | |
| SDLLMBackbone=LLaMA2-7B, Spike=✓, Bit=4-1.58, T x D=1 × 8, R=0.216, FLOPs (T)=0.75, Power (J)=0.672026.04 | 51.71 | |
| Engram-27BShots=5-shot2026.01 | 50.7 | |
| SDLLMBackbone=LLaMA3-8B, Spike=✓, Bit=4-1.58, T x D=1 × 8, R=0.210, FLOPs (T)=0.84, Power (J)=0.752026.04 | 49.31 | |
| MoE-27BShots=5-shot2026.01 | 48.8 | |
| QuaRotBackbone=LLaMA2-7B, Spike=✗, Bit=4-4, T x D=-, R=-, FLOPs (T)=0.86, Power (J)=3.972026.04 | 33.2 | |
| Dense-4BShots=5-shot2026.01 | 33 |