Text Classification Average on AGNews, DBPedia, HateSpeech18, MR, SST-2, SST-5, Subj, TREC
87.84AGNews AccuracyICL
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ICLBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=502026.05 | 87.84 | 97.24 | 78.92 | 94.6 | 94.52 | 50.28 | 90.68 | 85.68 | 84.97 | |
| LTVBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=502026.05 | 83.92 | 92.44 | 74.52 | 87.92 | 89.8 | 41.8 | 69.88 | 76.72 | 77.12 | |
| State VectorBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=502026.05 | 83.56 | 81.12 | 65.6 | 84.28 | 79.04 | 33.64 | 61.48 | 67.72 | 69.56 | |
| Task VectorBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=502026.05 | 77.6 | 82.36 | 65.6 | 81.72 | 88.24 | 32.24 | 59.44 | 67.6 | 69.35 | |
| I2CLBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=502026.05 | 77.16 | 72.84 | 62.32 | 85.52 | 90.64 | 32.44 | 61.96 | 49.4 | 66.53 | |
| Function VectorBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=502026.05 | 76.56 | 69.92 | 63.56 | 83 | 87.48 | 26.16 | 59.48 | 67.08 | 66.66 | |
| Zero-shotBackbone=LLaMA-3.1-8B, Number of demonstrations (k)=02026.05 | 75 | 69 | 60.6 | 82.2 | 86.8 | 25.6 | 59.2 | 49.4 | 63.48 |