Instruction Following on DomainBench
21.85Agriculture ScoreSYTTA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 21.85 | 30.93 | 22.26 | 29.57 | 26.15 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Dynamic-Ref, Extra Tokens=162025.10 | 21.14 | 28.81 | 26.83 | 30.25 | 26.76 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 20.17 | 29.47 | 26.48 | 29.58 | 26.43 | |
| SYTTABackbone=LLAMA-3.2-3B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 20.12 | 29.45 | 17.59 | 29.07 | 24.06 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 20.09 | 30.57 | 31.05 | 30.14 | 27.96 | |
| SYTTABackbone=LLAMA-3.2-3B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 19.72 | 26.74 | 17.64 | 29.1 | 23.3 | |
| TentBackbone=QWEN-2.5-7B2025.10 | 19.64 | 22.15 | 5.31 | 28.59 | 18.92 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Dynamic-Ref, Extra Tokens=162025.10 | 19.56 | 26.52 | 25.03 | 29.55 | 25.16 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 19.52 | 30.45 | 28.91 | 29.53 | 27.1 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 19.4 | 29.37 | 29.56 | 29.67 | 27 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Dynamic-Ref, Extra Tokens=162025.10 | 18.82 | 28.72 | 29.79 | 29.95 | 26.82 | |
| SYTTABackbone=LLAMA-3.2-3B, Reference Mode=Dynamic-Ref, Extra Tokens=162025.10 | 18.37 | 27.15 | 17.85 | 28.18 | 22.89 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 18.31 | 29.47 | 25.92 | 29.79 | 25.87 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 17.68 | 29.42 | 29.74 | 29.69 | 26.63 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 16.49 | 29.52 | 24.82 | 29.5 | 25.08 | |
| TLMBackbone=LLAMA-3.1-8B2025.10 | 16.33 | 28.85 | 25.71 | 28.95 | 24.96 | |
| EATABackbone=QWEN-2.5-7B2025.10 | 16.3 | 21.24 | 12.83 | 22.57 | 18.23 | |
| SYTTABackbone=LLAMA-3.2-3B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 15.38 | 28.31 | 19.66 | 28.28 | 22.91 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 15.17 | 29.19 | 21.23 | 29.86 | 23.86 | |
| TLMBackbone=LLAMA-3.2-3B2025.10 | 14.23 | 27.56 | 24.29 | 26.73 | 23.2 | |
| TLMBackbone=QWEN-2.5-7B2025.10 | 11.23 | 26.21 | 29.67 | 28.13 | 23.81 | |
| TLMBackbone=QWEN-2.5-14B2025.10 | 11.09 | 28.7 | 32.2 | 29.48 | 25.37 | |
| Base ModelBackbone=QWEN-2.5-14B2025.10 | 10.67 | 23.46 | 14.42 | 24.36 | 18.23 | |
| Base ModelBackbone=QWEN-2.5-7B2025.10 | 9.43 | 22.03 | 12.51 | 23.88 | 16.96 | |
| Base ModelBackbone=LLAMA-3.1-8B2025.10 | 8.59 | 22.28 | 13.53 | 21.65 | 16.51 | |
| SYTTA-8Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 8.57 | 15.85 | 14.18 | 13.21 | 12.95 | |
| SYTTA-8Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 8.38 | 16.36 | 9.43 | 12.58 | 11.69 | |
| Base ModelBackbone=LLAMA-3.2-3B2025.10 | 8.34 | 22.02 | 14.13 | 21.45 | 16.48 | |
| SYTTA-16Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 8.19 | 15.37 | 7.19 | 12.01 | 10.69 | |
| SYTTA-8Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 8.1 | 15.62 | 13.61 | 12.79 | 12.53 | |
| SYTTA-16Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 8.05 | 15.28 | 11.38 | 13.02 | 11.93 | |
| SYTTA-4Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 7.71 | 15.83 | 14.1 | 12.64 | 12.57 | |
| SYTTA-4Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 7.68 | 16.25 | 14.79 | 12.01 | 12.68 | |
| SYTTA-4Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 7.52 | 15.79 | 13.29 | 12.04 | 12.16 | |
| TentModel=QWEN-2.5-7B2025.10 | 7.32 | 15.15 | 4.24 | 11.96 | 9.67 | |
| SYTTA-4Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 7.29 | 15.54 | 14.01 | 12.21 | 12.26 | |
| SYTTA-8Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 7.26 | 15.31 | 13.22 | 12.48 | 12.07 | |
| SYTTA-16Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 7.26 | 11.82 | 9.36 | 10.75 | 9.8 | |
| SYTTA-4Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 7.11 | 14.83 | 3.76 | 11.92 | 9.4 | |
| SYTTA-2Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 7.07 | 15.05 | 9.21 | 11.91 | 10.81 | |
| SYTTA-4Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 7.05 | 15.82 | 11.14 | 12.17 | 11.54 | |
| SYTTA-16Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 6.98 | 15.31 | 14.98 | 12.39 | 12.42 | |
| SYTTA-16Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 6.86 | 15.84 | 10.86 | 12.54 | 11.53 | |
| SYTTA-8Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 6.6 | 15.18 | 6.66 | 11.57 | 10 | |
| SYTTA-4Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 6.57 | 13.76 | 4.12 | 11.95 | 9.1 | |
| TLMModel=LLAMA-3.1-8B2025.10 | 6.43 | 15.17 | 9.61 | 11.89 | 10.78 | |
| SYTTA-2Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 6.41 | 16.47 | 10.99 | 12.27 | 11.54 | |
| SYTTA-2Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 6.38 | 16.53 | 14.76 | 12.35 | 12.5 | |
| SYTTA-16Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 6.25 | 9.49 | 3.64 | 10.63 | 7.5 | |
| SYTTA-8Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 6.09 | 15.43 | 10.92 | 11.91 | 11.09 | |
| SYTTA-2Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 6.08 | 15.64 | 7.33 | 12.11 | 10.29 | |
| SYTTA-2Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 6.07 | 15.85 | 13.67 | 12.04 | 11.91 | |
| SYTTA-8Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 6.03 | 14.41 | 7.37 | 11.24 | 9.76 | |
| SYTTA-8Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 6.02 | 16.22 | 9.65 | 12.51 | 11.1 | |
| SYTTA-2Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 5.73 | 13.89 | 4.19 | 11.62 | 8.86 | |
| SYTTA-4Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 5.73 | 16.05 | 9.03 | 11.99 | 10.7 | |
| SYTTA-16Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 5.65 | 14.61 | 7.23 | 11.71 | 9.8 | |
| TentModel=QWEN-2.5-14B2025.10 | 5.64 | 14.21 | 4.42 | 11.2 | 8.87 | |
| SYTTA-2Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 5.46 | 15.78 | 14.06 | 11.45 | 11.69 | |
| TLMModel=LLAMA-3.2-3B2025.10 | 5.37 | 14.89 | 8.49 | 10.58 | 9.83 | |
| EATAModel=QWEN-2.5-7B2025.10 | 5.32 | 15.09 | 4.1 | 11.6 | 9.03 | |
| SYTTA-2Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 5.31 | 15.87 | 14.27 | 11.83 | 11.82 | |
| SYTTA-16Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 5.07 | 13.39 | 3.62 | 10.63 | 8.18 | |
| TentBackbone=QWEN-2.5-14B2025.10 | 4.92 | 27.89 | 14.87 | 28.19 | 18.97 | |
| EATAModel=QWEN-2.5-14B2025.10 | 4.54 | 14.71 | 4.64 | 10.31 | 8.55 | |
| TLMModel=QWEN-2.5-14B2025.10 | 4.31 | 14.4 | 16.75 | 12.07 | 11.88 | |
| TLMModel=QWEN-2.5-7B2025.10 | 4.2 | 13.23 | 14.85 | 11.79 | 11.02 | |
| Base ModelModel=QWEN-2.5-14B2025.10 | 4.14 | 10.1 | 3.84 | 8.69 | 6.69 | |
| Base ModelModel=QWEN-2.5-7B2025.10 | 3.63 | 9.5 | 3.5 | 8.67 | 6.33 | |
| Base ModelModel=LLAMA-3.1-8B2025.10 | 3.32 | 9.97 | 3.28 | 7.48 | 6.01 | |
| EATAModel=LLAMA-3.1-8B2025.10 | 3.32 | 12.33 | 2.22 | 2.94 | 5.2 | |
| EATAModel=LLAMA-3.2-3B2025.10 | 3.11 | 12.96 | 3.18 | 7.21 | 6.61 | |
| TentModel=LLAMA-3.2-3B2025.10 | 3.05 | 8.21 | 2.77 | 5.17 | 4.8 | |
| Base ModelModel=LLAMA-3.2-3B2025.10 | 3.04 | 9.81 | 2.7 | 7.42 | 5.74 | |
| TentModel=LLAMA-3.1-8B2025.10 | 2.62 | 11.82 | 2.62 | 8.51 | 6.39 | |
| EATABackbone=QWEN-2.5-14B2025.10 | 1.88 | 28.23 | 3.17 | 27.97 | 15.31 | |
| EATABackbone=LLAMA-3.1-8B2025.10 | 1.52 | 6.43 | 1.86 | 14.6 | 6.1 | |
| TentBackbone=LLAMA-3.1-8B2025.10 | 1.16 | 3.79 | 0.74 | 13.22 | 4.73 | |
| TentBackbone=LLAMA-3.2-3B2025.10 | 0.98 | 4.59 | 9.32 | 2.33 | 4.3 | |
| EATABackbone=LLAMA-3.2-3B2025.10 | 0.39 | 4.89 | 5.49 | 0.03 | 2.7 |