Few-shot classification on Synthetic Few-shot (P_bursty=0.9, alpha=2)
88ICL AccuracyCoQE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoQEE=512, L=4, #Param=2.0M2026.03 | 88 | 67.9 | |
| CoQEE=256, L=4, #Param=1.0M2026.03 | 85.8 | 63.1 | |
| TransformerE=512, L=4, #Param=2.0M2026.03 | 51 | 99.1 | |
| TransformerE=256, L=4, #Param=1.0M2026.03 | 50.6 | 98.6 | |
| TransformerE=512, L=12, #Param=2.4M2026.03 | 50.3 | 98.9 | |
| TransformerE=512, L=8, #Param=2.2M2026.03 | 50.3 | 99 | |
| TransformerE=64, L=8, #Param=0.6M2026.03 | 50.2 | 97.9 | |
| TransformerE=256, L=8, #Param=1.2M2026.03 | 50.2 | 98.4 | |
| Transformer + RegularizationE=64, L=12, #Param=0.9M2026.03 | 50.2 | 63.2 | |
| TransformerE=64, L=4, #Param=0.4M2026.03 | 50.1 | 98.1 | |
| TransformerE=256, L=12, #Param=1.4M2026.03 | 50 | 98.4 | |
| Linear TransformerE=64, L=12, #Param=0.8M2026.03 | 50 | 98.4 | |
| Transformer + NoiseE=64, L=12, #Param=0.9M2026.03 | 49.9 | 91.3 | |
| Transformer + Temporary ForgettingE=64, L=12, #Param=0.9M2026.03 | 49.8 | 97.1 | |
| TransformerE=64, L=12, #Param=0.9M2026.03 | 49.7 | 98 |