Few-shot classification on Synthetic Few-shot (P_bursty=0.9, alpha=0)
91.2ICL AccuracyCoQE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoQEE=512, L=4, #Param=2.0M2026.03 | 91.2 | 89.4 | |
| CoQEE=256, L=4, #Param=1.0M2026.03 | 89.2 | 81 | |
| TransformerE=64, L=12, #Param=0.9M2026.03 | 73.1 | 4.1 | |
| Transformer + NoiseE=64, L=12, #Param=0.9M2026.03 | 70.3 | 1.2 | |
| TransformerE=64, L=8, #Param=0.6M2026.03 | 67.2 | 8.1 | |
| Transformer + RegularizationE=64, L=12, #Param=0.9M2026.03 | 64.2 | 3.1 | |
| TransformerE=256, L=12, #Param=1.4M2026.03 | 51.4 | 20.1 | |
| Transformer + Temporary ForgettingE=64, L=12, #Param=0.9M2026.03 | 51 | 0.8 | |
| Linear TransformerE=64, L=12, #Param=0.8M2026.03 | 51 | 8 | |
| TransformerE=256, L=8, #Param=1.2M2026.03 | 50.7 | 28.7 | |
| TransformerE=64, L=4, #Param=0.4M2026.03 | 50.5 | 15.1 | |
| TransformerE=512, L=12, #Param=2.4M2026.03 | 50.5 | 36 | |
| TransformerE=256, L=4, #Param=1.0M2026.03 | 50.3 | 38.8 | |
| TransformerE=512, L=4, #Param=2.0M2026.03 | 50 | 62.5 | |
| TransformerE=512, L=8, #Param=2.2M2026.03 | 49.8 | 49.3 |