Gemma
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Gemma-2-2B (layer 12)
11.72SCR
5
Gemma 175 categories 3-4B
95.8Mean AUC
5
Gemma-2-27b SAE: gemmascope-res-131k Layer 2
26.63GEN Score
5
Gemma-2-27b SAE gemmascope-res-131k Layer 1
53.63GEN
5
Gemma-2-9b SAE gemmascope-res-16k Layer 2
54.5Generation Score
5
Gemma-2-9b SAE gemmascope-res-16k Layer 1
46.13Generation Score
5
Gemma-2-2B Digit Addition
61.51Accuracy
5
Gemma-2-2B target τGemma=1.09 (test)
205.56Joint Z-Score
5
Gemma held-out (test)
2.17Throughput Ratio Improvement
5
Gemma 4B-it 3
25ASR
5
Gemma 4 E2B
127.7Throughput (tok/s)
4
Gemma-2-2B (layer 12)
0.963KL Score
4
Gemma-2-2B layer 12 activations
82Explained Variance (EV)
4
Gemma-2-2B Layer 12 match with Layer 13
2.32LLM Eval
4
Gemma 2 2b
1.58MSE
4
Gemma-2-2B
1,621,365.943SCE (Efficiency)
4
Gemma 3
11,322Final Depth Formula
4
Gemma 26B-A4B 4
69.3Throughput (tok/s)
3
Gemma-2-9B target L20, width-131k, τ9B=1.09 (test)
283.3Joint-Z Score
3
GEMMA-2-2B
1,621,365SCE
3
Gemma-2-2B circuits
0.635Clarity
3
Gemma vision encoder later layer SAE 3 (test)
20.4IoU (Masks)
3
Gemma SAE Features Key Input 3-1B
4.86Interpretability Score
2
Gemma 1B 3
0MSE
2
Gemma-2 9B-IT
93TPR
1