Instruction Following on Pronoun Changing
99.88P. ScoreAdaSEKA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AdaSEKAModel=Qwen3-14B2026.03 | 99.88 | 99.86 | |
| AdaSEKAModel=Qwen3-8B2026.03 | 99.68 | 99.52 | |
| PASTAModel=Qwen3-8B2026.03 | 98.86 | 98.72 | |
| **-markedModel=Qwen3-14B2026.03 | 98.86 | 98.68 | |
| SEKAModel=Qwen3-14B2026.03 | 98.66 | 98.54 | |
| SEKAModel=Qwen3-8B2026.03 | 98.56 | 98.26 | |
| OriginalModel=Qwen3-14B2026.03 | 98.42 | 98.22 | |
| w/o learnModel=Qwen3-8B, Configuration=Ablation: random projections2026.03 | 98.12 | 97.9 | |
| **-markedModel=Qwen3-8B2026.03 | 98.1 | 97.84 | |
| OriginalModel=Qwen3-8B2026.03 | 98 | 97.84 | |
| SEKAModel=Gemma3-12B2026.03 | 97.7 | 97.24 | |
| AdaSEKAModel=Gemma3-12B2026.03 | 96.88 | 95.84 | |
| w/o learnModel=Qwen3-14B, Configuration=Ablation: random projections2026.03 | 96.32 | 96.36 | |
| PASTAModel=Qwen3-4B2026.03 | 95.82 | 94.64 | |
| **-markedModel=Gemma3-12B2026.03 | 95.78 | 96.42 | |
| **-markedModel=Qwen3-4B2026.03 | 95.76 | 93.88 | |
| SEKAModel=Qwen3-4B2026.03 | 95.18 | 93.26 | |
| AdaSEKAModel=Qwen3-4B2026.03 | 94.54 | 92.08 | |
| OriginalModel=Gemma3-12B2026.03 | 93.92 | 94.96 | |
| AdaSEKAModel=Gemma3-4B2026.03 | 93.76 | 93.58 | |
| OriginalModel=Qwen3-4B2026.03 | 93.14 | 90.52 | |
| SPAModel=Qwen3-14B2026.03 | 91.6 | 92.2 | |
| PASTAModel=Qwen3-14B2026.03 | 90.98 | 90.94 | |
| w/o learnModel=Qwen3-4B, Configuration=Ablation: random projections2026.03 | 90.42 | 88.66 | |
| w/o learn&filtModel=Qwen3-14B, Configuration=Ablation: remove head filtering2026.03 | 88.6 | 89.76 | |
| SPAModel=Gemma3-12B2026.03 | 86.41 | 84.99 | |
| SEKAModel=Gemma3-4B2026.03 | 81.53 | 81.11 | |
| w/o learn&filtModel=Qwen3-8B, Configuration=Ablation: remove head filtering2026.03 | 80.53 | 80.85 | |
| SPAModel=Qwen3-4B2026.03 | 80.27 | 78.19 | |
| SPAModel=Gemma3-4B2026.03 | 76.05 | 74.45 | |
| SPAModel=Qwen3-8B2026.03 | 72.61 | 74.95 | |
| PASTAModel=Gemma3-12B2026.03 | 68.47 | 68.01 | |
| PASTAModel=Gemma3-4B2026.03 | 67.39 | 66.43 | |
| w/o learn&filtModel=Gemma3-12B, Configuration=Ablation: remove head filtering2026.03 | 55.56 | 58.76 | |
| w/o learnModel=Gemma3-4B, Configuration=Ablation: random projections2026.03 | 53.58 | 48.82 | |
| w/o learn&filtModel=Gemma3-4B, Configuration=Ablation: remove head filtering2026.03 | 51.78 | 51.94 | |
| w/o learnModel=Gemma3-12B, Configuration=Ablation: random projections2026.03 | 47.26 | 51.24 | |
| w/o learn&filtModel=Qwen3-4B, Configuration=Ablation: remove head filtering2026.03 | 41.98 | 36.95 | |
| OriginalModel=Gemma3-4B2026.03 | 41.34 | 35.25 | |
| **-markedModel=Gemma3-4B2026.03 | 38.86 | 32.45 |