Instruction Following on Tulu3 Evaluation Suite pool (test)
92.54ARCTAGS
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TAGSBase Model=Qwen3-4B, DataSize=50K2026.01 | 92.54 | 70.78 | 89.69 | 76.83 | 75.33 | 66.73 | 78.65 | 14.78 | 7.34 | 40.98 | 43.05 | 60.85 | |
| CaRBase Model=Qwen3-4B, DataSize=50K2026.01 | 91.86 | 68.51 | 79.3 | 77.44 | 75.26 | 54.9 | 74.54 | 13.54 | 7.33 | 38.8 | 41.88 | 58.21 | |
| IFDBase Model=Qwen3-4B, DataSize=50K2026.01 | 91.53 | 72.4 | 78.17 | 76.22 | 75.23 | 57.86 | 75.23 | 15.65 | 7.26 | 40.15 | 42.8 | 59.02 | |
| MIGBase Model=Qwen3-4B, DataSize=50K2026.01 | 91.53 | 72.1 | 70.43 | 81.1 | 75.47 | 65.8 | 76.07 | 14.04 | 7.64 | 40.66 | 43.7 | 59.88 | |
| QDITBase Model=Qwen3-4B, DataSize=50K2026.01 | 91.19 | 72.9 | 85.06 | 76.83 | 75.16 | 58.78 | 76.65 | 12.8 | 7.24 | 38.86 | 41.35 | 59 | |
| IFDBase Model=Qwen3-8B, DataSize=50K2026.01 | 90.85 | 77.65 | 91.36 | 81.71 | 78.79 | 57.12 | 79.58 | 15.16 | 7.62 | 42.32 | 44.56 | 62.07 | |
| PoolBase Model=Qwen3-8B, DataSize=939K2026.01 | 88.47 | 69.71 | 88.86 | 77.44 | 75.27 | 69.87 | 78.27 | 9.69 | 7.34 | 44.52 | 42.54 | 60.4 | |
| TAGSBase Model=Qwen3-8B, DataSize=50K2026.01 | 86.78 | 74.22 | 91.51 | 82.32 | 78.75 | 66.91 | 80.08 | 20.12 | 7.85 | 44.76 | 47.79 | 63.93 | |
| #InsTagBase Model=Qwen3-8B, DataSize=50K2026.01 | 85.42 | 70.78 | 90.75 | 80.49 | 78.81 | 60.63 | 77.81 | 10.68 | 7.67 | 42.16 | 43.18 | 60.5 | |
| RandomBase Model=Qwen3-4B, DataSize=50K2026.01 | 85.42 | 67.75 | 89.23 | 78.05 | 75.29 | 61.18 | 76.15 | 9.07 | 7.25 | 38.16 | 39.91 | 58.03 | |
| #InsTagBase Model=Qwen3-4B, DataSize=50K2026.01 | 84.07 | 66.88 | 86.96 | 78.66 | 75.35 | 63.59 | 75.92 | 9.57 | 7.38 | 38.26 | 40.54 | 58.23 | |
| TAGSBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 82.03 | 54.31 | 80.14 | 67.68 | 63.31 | 48.43 | 65.98 | 10.93 | 6.76 | 29.83 | 36.12 | 51.05 | |
| RandomBase Model=Qwen3-8B, DataSize=50K2026.01 | 80.68 | 72.29 | 90.75 | 82.93 | 78.6 | 64.14 | 78.23 | 9.44 | 7.51 | 41.74 | 42.09 | 60.16 | |
| MIGBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 80.34 | 53.53 | 78.77 | 66.46 | 64 | 44.36 | 64.58 | 9.69 | 6.54 | 27.92 | 34.34 | 49.46 | |
| CaRBase Model=Qwen3-8B, DataSize=50K2026.01 | 79.66 | 73.4 | 91.36 | 82.32 | 78.91 | 56.19 | 76.97 | 13.29 | 7.49 | 42.52 | 43.57 | 60.27 | |
| RandomBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 78.64 | 51.68 | 77.26 | 65.85 | 64.33 | 42.14 | 63.32 | 4.84 | 6.64 | 25.02 | 32.09 | 47.7 | |
| IFDBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 78.31 | 53.92 | 54.97 | 67.07 | 64.2 | 35.49 | 58.99 | 6.21 | 6.6 | 25.46 | 32.56 | 45.77 | |
| #InsTagBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 78.31 | 50.88 | 77.26 | 66.46 | 64.27 | 43.44 | 63.44 | 5.09 | 6.51 | 26.17 | 32.12 | 47.78 | |
| QDITBase Model=Qwen3-8B, DataSize=50K2026.01 | 76.95 | 77.4 | 90.67 | 81.71 | 78.82 | 58.23 | 77.3 | 14.04 | 7.41 | 40.9 | 43.01 | 60.16 | |
| DEITABase Model=Qwen3-4B, DataSize=50K2026.01 | 76.95 | 67.83 | 81.35 | 78.66 | 75.56 | 63.22 | 73.93 | 8.2 | 7.26 | 38.14 | 39.64 | 56.78 | |
| QDITBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 76.95 | 54.62 | 58.07 | 67.07 | 64.13 | 37.71 | 59.76 | 7.08 | 6.26 | 24.49 | 31.39 | 45.58 | |
| MIGBase Model=Qwen3-8B, DataSize=50K2026.01 | 76.61 | 74.49 | 91.51 | 79.88 | 79.04 | 65.8 | 77.89 | 16.65 | 7.59 | 43.49 | 45.35 | 61.46 | |
| CaRBase Model=Qwen3-1.7B, DataSize=50K2026.01 | 71.86 | 52.7 | 73.77 | 61.59 | 64.36 | 38.45 | 60.45 | 6.46 | 6.43 | 26.42 | 32.4 | 46.42 | |
| DEITABase Model=Qwen3-1.7B, DataSize=50K2026.01 | 71.53 | 51.42 | 62.55 | 65.85 | 64.29 | 42.33 | 59.66 | 5.22 | 6.33 | 25.4 | 31.31 | 45.48 | |
| DEITABase Model=Qwen3-8B, DataSize=50K2026.01 | 60 | 71.59 | 92.65 | 85.37 | 78.61 | 60.44 | 74.78 | 14.04 | 7.62 | 44.16 | 44.8 | 59.79 |