Copy on Copy OOD lengths: 2x, 4x, 8x, 16x, 32x, 64x
100Exact Match AccuracySoftmax
Evaluation results
| Method | Links | |
|---|---|---|
| SoftmaxLayers (L)=2, OOD Sequence Length Scale=2x2025.06 | 100 | |
| SSMaxLayers (L)=2, OOD Sequence Length Scale=2x2025.06 | 100 | |
| SSMaxLayers (L)=2, OOD Sequence Length Scale=4x2025.06 | 100 | |
| ASEntmaxLayers (L)=2, OOD Sequence Length Scale=2x2025.06 | 100 | |
| SoftmaxLayers (L)=2, OOD Sequence Length Scale=4x2025.06 | 99.9 | |
| SoftmaxLayers (L)=2, OOD Sequence Length Scale=8x2025.06 | 99.9 | |
| SSMaxLayers (L)=2, OOD Sequence Length Scale=8x2025.06 | 99.9 | |
| ASEntmaxLayers (L)=2, OOD Sequence Length Scale=4x2025.06 | 99.9 | |
| Top-K (K=32)Layers (L)=2, OOD Sequence Length Scale=2x2025.06 | 99.7 | |
| ASEntmaxLayers (L)=2, OOD Sequence Length Scale=8x2025.06 | 99.7 | |
| SSMaxLayers (L)=2, OOD Sequence Length Scale=16x2025.06 | 99.6 | |
| SoftmaxLayers (L)=2, OOD Sequence Length Scale=16x2025.06 | 99.4 | |
| ASEntmaxLayers (L)=2, OOD Sequence Length Scale=16x2025.06 | 99.4 | |
| SSMaxLayers (L)=2, OOD Sequence Length Scale=32x2025.06 | 99.3 | |
| EntmaxLayers (L)=2, OOD Sequence Length Scale=2x2025.06 | 99 | |
| Top-K (K=32)Layers (L)=2, OOD Sequence Length Scale=4x2025.06 | 96.8 | |
| ASEntmaxLayers (L)=2, OOD Sequence Length Scale=32x2025.06 | 96.3 | |
| SoftmaxLayers (L)=2, OOD Sequence Length Scale=32x2025.06 | 96.1 | |
| SSMaxLayers (L)=2, OOD Sequence Length Scale=64x2025.06 | 95.8 | |
| ASEntmaxLayers (L)=2, OOD Sequence Length Scale=64x2025.06 | 86.6 | |
| EntmaxLayers (L)=2, OOD Sequence Length Scale=4x2025.06 | 86 | |
| SoftmaxLayers (L)=2, OOD Sequence Length Scale=64x2025.06 | 85.5 | |
| EntmaxLayers (L)=2, OOD Sequence Length Scale=8x2025.06 | 28.5 | |
| Top-K (K=32)Layers (L)=2, OOD Sequence Length Scale=8x2025.06 | 26.7 | |
| EntmaxLayers (L)=2, OOD Sequence Length Scale=16x2025.06 | 0.2 | |
| Top-K (K=32)Layers (L)=2, OOD Sequence Length Scale=16x2025.06 | 0 | |
| Top-K (K=32)Layers (L)=2, OOD Sequence Length Scale=32x2025.06 | 0 | |
| Top-K (K=32)Layers (L)=2, OOD Sequence Length Scale=64x2025.06 | 0 | |
| EntmaxLayers (L)=2, OOD Sequence Length Scale=32x2025.06 | 0 | |
| EntmaxLayers (L)=2, OOD Sequence Length Scale=64x2025.06 | 0 |