Multimodal Image Understanding on LLaVA Evaluation Suite 1.5 (test val)
63.5GQABaseline
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| BaselineTokens Retained=576, Token Retention Rate=100%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 63.5 | 68.21 | 62.63 | 1,787.33 | 88.52 | 72.38 | 48.91 | 56.14 | 35.7 | 100 | |
| TORINO-PTokens Retained=~217, Token Retention Rate=38%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 60.13 | 66.15 | 60.4 | 1,753.61 | 87.86 | 71.74 | 46.61 | 55.51 | 34.4 | 97.23 | |
| TORINO-MTokens Retained=~217, Token Retention Rate=38%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 59.92 | 66.32 | 60.31 | 1,728.88 | 87.94 | 71.64 | 46.53 | 55.56 | 34.4 | 97.04 | |
| FOLDERTokens Retained=~217, Token Retention Rate=38%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 59.8 | 65.81 | 59.97 | 1,678.18 | 88.38 | 71.59 | 42.79 | 55.46 | 34.6 | 95.8 | |
| PruMergeTokens Retained=~217, Token Retention Rate=38%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 59.22 | 65.98 | 61.34 | 1,729.59 | 87.04 | 72.24 | 43.07 | 55.63 | 35.8 | 96.69 | |
| PruneSIDTokens Retained=~217, Token Retention Rate=38%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 59.15 | 65.98 | 60.22 | 1,740.8 | 88.06 | 71.49 | 43.3 | 55.36 | 34.8 | 96.26 | |
| RandomTokens Retained=~217, Token Retention Rate=38%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 58.8 | 62.54 | 58.08 | 1,781.1 | 86.49 | 70.95 | 36.04 | 55.26 | 33.4 | 93.12 | |
| TORINO-PTokens Retained=~129, Token Retention Rate=22%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 58.62 | 65.46 | 59.97 | 1,745.8 | 86.96 | 70.8 | 43.93 | 55.97 | 34.5 | 95.99 | |
| PruneSIDTokens Retained=~129, Token Retention Rate=22%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 58.43 | 63.75 | 58.93 | 1,731.43 | 86.86 | 72.24 | 41.23 | 55.78 | 32.3 | 94.28 | |
| TORINO-MTokens Retained=~129, Token Retention Rate=22%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 58.2 | 65.21 | 59.97 | 1,725.12 | 86.5 | 71.1 | 43.98 | 56.19 | 34.6 | 95.82 | |
| FOLDERTokens Retained=~129, Token Retention Rate=22%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 58.01 | 61.68 | 56.44 | 1,693.52 | 86.89 | 71.24 | 35.88 | 56.1 | 30.3 | 91.27 | |
| RandomTokens Retained=~129, Token Retention Rate=22%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 57.29 | 58.93 | 54.47 | 1,774.34 | 84.5 | 70.6 | 30.06 | 55.67 | 27.8 | 88.26 | |
| PruMergeTokens Retained=~129, Token Retention Rate=22%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 57.25 | 66.07 | 59.36 | 1,676.44 | 84.14 | 72.43 | 42.49 | 55.86 | 33.9 | 94.67 | |
| PruneSIDTokens Retained=~47, Token Retention Rate=8%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 57.03 | 60.4 | 55.67 | 1,727.21 | 85.98 | 68.57 | 39.88 | 58.29 | 30.2 | 91.74 | |
| TORINO-PTokens Retained=~47, Token Retention Rate=8%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 55.76 | 62.37 | 56.79 | 1,676.53 | 80.74 | 69.51 | 35.66 | 57.43 | 31.7 | 90.55 | |
| TORINO-MTokens Retained=~47, Token Retention Rate=8%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 55.48 | 61.51 | 55.84 | 1,666.57 | 80.44 | 69.41 | 35.76 | 57.72 | 31.5 | 90.1 | |
| FOLDERTokens Retained=~47, Token Retention Rate=8%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 54.68 | 54.21 | 46.82 | 1,642.95 | 82.48 | 67.72 | 26.94 | 58.19 | 26 | 83.4 | |
| RandomTokens Retained=~47, Token Retention Rate=8%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 54.12 | 51.72 | 46.39 | 1,640.23 | 77.98 | 65.84 | 21.17 | 56.88 | 28.7 | 81.21 | |
| PruMergeTokens Retained=~47, Token Retention Rate=8%, SAE Configuration=Matryoshka BatchTopK 20 ε = 64, CLS-only2026.07 | 53.74 | 61.77 | 56.27 | 1,600.72 | 73.89 | 71.54 | 39.71 | 57.24 | 28.9 | 89 |