Instruction Following on InstructBench
75.27Dolly (BLEU)SYTTA-8
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SYTTA-8Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 75.27 | 75.4 | 72.14 | 74.27 | |
| SYTTA-4Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 74.98 | 75.57 | 72.14 | 74.23 | |
| SYTTA-8Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 74.61 | 75.9 | 71.98 | 74.17 | |
| SYTTA-16Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 74.56 | 75.85 | 71.96 | 74.12 | |
| SYTTA-16Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 74.55 | 75.6 | 72.43 | 74.19 | |
| SYTTA-8Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 74.44 | 75.71 | 72.53 | 74.23 | |
| SYTTA-4Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 74.33 | 75.75 | 71.77 | 73.95 | |
| SYTTA-2Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 74.33 | 76.26 | 71.53 | 74.04 | |
| SYTTA-4Model=QWEN-2.5-14B, Reference Strategy=Static-Ref2025.10 | 74.22 | 76.05 | 72.03 | 74.1 | |
| SYTTA-16Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 74.1 | 75.3 | 71.87 | 73.76 | |
| SYTTA-2Model=QWEN-2.5-7B, Reference Strategy=Static-Ref2025.10 | 74.04 | 75.83 | 71.64 | 73.84 | |
| SYTTA-4Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 73.99 | 74.48 | 72.94 | 73.81 | |
| EATAModel=QWEN-2.5-14B2025.10 | 73.98 | 74.42 | 70.95 | 73.11 | |
| SYTTA-16Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 73.98 | 75.46 | 72.31 | 73.92 | |
| SYTTA-8Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 73.95 | 74.37 | 71.45 | 73.26 | |
| EATAModel=LLAMA-3.1-8B2025.10 | 73.83 | 59.81 | 68.98 | 67.54 | |
| SYTTA-4Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 73.83 | 74.73 | 72.45 | 73.67 | |
| SYTTA-16Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 73.74 | 74.37 | 72.24 | 73.45 | |
| TentModel=QWEN-2.5-14B2025.10 | 73.72 | 74.25 | 69.8 | 72.59 | |
| SYTTA-4Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 73.72 | 76.09 | 72.06 | 73.96 | |
| SYTTA-2Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 73.68 | 76.25 | 71.85 | 73.93 | |
| SYTTA-2Model=QWEN-2.5-7B, Reference Strategy=Dynamic-Ref2025.10 | 73.54 | 75.94 | 71.58 | 73.68 | |
| SYTTA-2Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 73.51 | 74.37 | 71.41 | 73.1 | |
| SYTTA-8Model=QWEN-2.5-14B, Reference Strategy=Dynamic-Ref2025.10 | 73.5 | 75.93 | 74.49 | 74.64 | |
| SYTTA-8Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 73.35 | 74.15 | 72.22 | 73.24 | |
| TLMModel=QWEN-2.5-14B2025.10 | 73.34 | 76.13 | 71.58 | 73.68 | |
| SYTTA-2Model=LLAMA-3.1-8B, Reference Strategy=Static-Ref2025.10 | 73.32 | 74.52 | 71.3 | 73.05 | |
| SYTTA-16Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 73.29 | 72.85 | 71.42 | 72.52 | |
| SYTTA-2Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 73.24 | 74.91 | 71.27 | 73.14 | |
| TLMModel=QWEN-2.5-7B2025.10 | 73.15 | 75.95 | 71.65 | 73.58 | |
| SYTTA-4Model=LLAMA-3.2-3B, Reference Strategy=Static-Ref2025.10 | 73.11 | 74.7 | 72.87 | 73.56 | |
| SYTTA-8Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 73 | 74.76 | 72.03 | 73.26 | |
| Base ModelModel=LLAMA-3.1-8B2025.10 | 72.91 | 72.05 | 70.14 | 71.7 | |
| SYTTA-4Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 72.75 | 74.71 | 72.06 | 73.17 | |
| SYTTA-8Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 72.59 | 74.56 | 71.41 | 72.86 | |
| SYTTA-2Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 71.91 | 74.7 | 72.03 | 72.88 | |
| TLMModel=LLAMA-3.1-8B2025.10 | 71.89 | 74.44 | 70.8 | 72.37 | |
| Base ModelModel=LLAMA-3.2-3B2025.10 | 71.74 | 72.11 | 70.18 | 71.34 | |
| EATAModel=QWEN-2.5-7B2025.10 | 71.17 | 72.92 | 71.14 | 71.74 | |
| SYTTA-16Model=LLAMA-3.1-8B, Reference Strategy=Dynamic-Ref2025.10 | 71.01 | 74.08 | 74 | 73.03 | |
| Base ModelModel=QWEN-2.5-14B2025.10 | 70.63 | 73.87 | 70.91 | 71.8 | |
| Base ModelModel=QWEN-2.5-7B2025.10 | 70.6 | 73.56 | 70.61 | 71.59 | |
| TentModel=QWEN-2.5-7B2025.10 | 70.54 | 74.42 | 70.78 | 71.91 | |
| SYTTA-16Model=LLAMA-3.2-3B, Reference Strategy=Dynamic-Ref2025.10 | 69.96 | 72.88 | 71.42 | 71.42 | |
| TLMModel=LLAMA-3.2-3B2025.10 | 69.87 | 74.32 | 70.61 | 71.6 | |
| TentModel=LLAMA-3.2-3B2025.10 | 68.04 | 72.82 | 69.75 | 70.2 | |
| EATAModel=LLAMA-3.2-3B2025.10 | 67.8 | 71.92 | 66.5 | 68.74 | |
| TentModel=LLAMA-3.1-8B2025.10 | 67.76 | 67.87 | 69.12 | 68.25 | |
| SYTTA-8Model=QWEN-2.5-7B, Reference Type=Static-Ref2025.10 | 42.08 | 45.61 | 37.02 | 41.57 | |
| SYTTA-16Model=QWEN-2.5-14B, Reference Type=Static-Ref2025.10 | 41.81 | 45.75 | 36.87 | 41.47 | |
| SYTTA-2Model=QWEN-2.5-14B, Reference Type=Static-Ref2025.10 | 41.68 | 46.73 | 34.6 | 41 | |
| SYTTA-4Model=QWEN-2.5-7B, Reference Type=Static-Ref2025.10 | 41.66 | 45.33 | 37.59 | 41.52 | |
| SYTTA-4Model=QWEN-2.5-14B, Reference Type=Static-Ref2025.10 | 41.55 | 46.63 | 37.45 | 41.88 | |
| SYTTA-8Model=QWEN-2.5-7B, Reference Type=Dynamic-Ref2025.10 | 41.35 | 46.24 | 37.99 | 41.86 | |
| SYTTA-8Model=QWEN-2.5-14B, Reference Type=Static-Ref2025.10 | 41 | 46.2 | 37.88 | 41.69 | |
| SYTTA-4Model=QWEN-2.5-14B, Reference Type=Dynamic-Ref2025.10 | 40.87 | 46.69 | 37.98 | 41.85 | |
| SYTTA-16Model=QWEN-2.5-7B, Reference Type=Dynamic-Ref2025.10 | 40.83 | 45.84 | 36.66 | 41.11 | |
| SYTTA-2Model=QWEN-2.5-7B, Reference Type=Dynamic-Ref2025.10 | 40.76 | 46.65 | 34.73 | 40.71 | |
| SYTTA-4Model=QWEN-2.5-7B, Reference Type=Dynamic-Ref2025.10 | 40.75 | 45.05 | 36.22 | 40.67 | |
| SYTTA-16Model=QWEN-2.5-7B, Reference Type=Static-Ref2025.10 | 40.5 | 46.15 | 37.1 | 41.25 | |
| SYTTA-2Model=QWEN-2.5-14B, Reference Type=Dynamic-Ref2025.10 | 40.38 | 46.87 | 36.04 | 41.1 | |
| SYTTA-2Model=QWEN-2.5-7B, Reference Type=Static-Ref2025.10 | 40.37 | 46.42 | 34.35 | 40.38 | |
| SYTTA-16Model=LLAMA-3.1-8B, Reference Type=Static-Ref2025.10 | 40.33 | 42.8 | 35.08 | 39.4 | |
| EATAModel=QWEN-2.5-14B2025.10 | 40.26 | 43.81 | 32.09 | 38.72 | |
| SYTTA-16Model=QWEN-2.5-14B, Reference Type=Dynamic-Ref2025.10 | 40.26 | 45.17 | 37.94 | 41.13 | |
| SYTTA-8Model=LLAMA-3.1-8B, Reference Type=Static-Ref2025.10 | 40.25 | 43.45 | 36.6 | 40.1 | |
| SYTTA-8Model=QWEN-2.5-14B, Reference Type=Dynamic-Ref2025.10 | 40.15 | 46.28 | 40.65 | 42.36 | |
| TentModel=QWEN-2.5-14B2025.10 | 40.09 | 42.74 | 32.26 | 38.36 | |
| SYTTA-4Model=LLAMA-3.1-8B, Reference Type=Static-Ref2025.10 | 39.8 | 44.12 | 39.5 | 41.14 | |
| SYTTA-2Model=LLAMA-3.2-3B, Reference Type=Static-Ref2025.10 | 38.83 | 43.48 | 35.48 | 39.26 | |
| EATAModel=LLAMA-3.1-8B2025.10 | 38.52 | 7.77 | 29.88 | 25.39 | |
| SYTTA-4Model=LLAMA-3.1-8B, Reference Type=Dynamic-Ref2025.10 | 38.52 | 44.67 | 40.09 | 41.09 | |
| SYTTA-8Model=LLAMA-3.2-3B, Reference Type=Static-Ref2025.10 | 38.4 | 42.49 | 38.72 | 39.87 | |
| SYTTA-2Model=LLAMA-3.1-8B, Reference Type=Static-Ref2025.10 | 38.1 | 44.31 | 33.7 | 38.7 | |
| TLMModel=QWEN-2.5-14B2025.10 | 37.94 | 45.64 | 33.33 | 38.97 | |
| SYTTA-16Model=LLAMA-3.2-3B, Reference Type=Static-Ref2025.10 | 37.76 | 39.13 | 35.31 | 37.4 | |
| SYTTA-2Model=LLAMA-3.1-8B, Reference Type=Dynamic-Ref2025.10 | 37.69 | 44.44 | 33.57 | 38.57 | |
| SYTTA-4Model=LLAMA-3.2-3B, Reference Type=Static-Ref2025.10 | 37.6 | 44.08 | 39.83 | 40.51 | |
| TLMModel=QWEN-2.5-7B2025.10 | 37.43 | 44.6 | 34.65 | 38.89 | |
| SYTTA-8Model=LLAMA-3.2-3B, Reference Type=Dynamic-Ref2025.10 | 37.39 | 43.4 | 36.64 | 39.14 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 37.04 | 43.24 | 34.45 | 38.24 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 37.04 | 42.9 | 34.46 | 38.13 | |
| Base ModelModel=LLAMA-3.1-8B2025.10 | 36.85 | 37.56 | 28.02 | 34.14 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 36.51 | 43.4 | 34.07 | 37.99 | |
| TLMModel=LLAMA-3.1-8B2025.10 | 36.44 | 41.72 | 31.73 | 36.63 | |
| SYTTA-8Model=LLAMA-3.1-8B, Reference Type=Dynamic-Ref2025.10 | 36.35 | 44.14 | 37.81 | 39.43 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 36.32 | 43.13 | 34.12 | 37.86 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 36.27 | 43.04 | 33.72 | 37.68 | |
| SYTTA-4Model=LLAMA-3.2-3B, Reference Type=Dynamic-Ref2025.10 | 36.24 | 44.05 | 38.34 | 39.55 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Dynamic-Ref, Extra Tokens=162025.10 | 35.93 | 43.13 | 33.67 | 37.58 | |
| SYTTABackbone=QWEN-2.5-7B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 35.69 | 43.33 | 32.95 | 37.32 | |
| SYTTABackbone=QWEN-2.5-14B, Reference Mode=Dynamic-Ref, Extra Tokens=162025.10 | 35.57 | 42.86 | 35.49 | 37.97 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 35.45 | 40.85 | 35.71 | 37.34 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 35.42 | 39.85 | 32.08 | 35.78 | |
| Base ModelModel=LLAMA-3.2-3B2025.10 | 34.66 | 37.6 | 27.97 | 33.41 | |
| SYTTABackbone=LLAMA-3.1-8B, Reference Mode=Dynamic-Ref, Extra Tokens=42025.10 | 34.61 | 41.27 | 36.15 | 37.34 | |
| SYTTABackbone=LLAMA-3.2-3B, Reference Mode=Static-Ref, Extra Tokens=42025.10 | 34.12 | 40.53 | 36.15 | 36.93 | |
| TLMBackbone=QWEN-2.5-14B2025.10 | 34.04 | 42.2 | 30.59 | 35.61 | |
| SYTTABackbone=LLAMA-3.2-3B, Reference Mode=Static-Ref, Extra Tokens=162025.10 | 33.46 | 39.67 | 32.65 | 35.26 | |
| SYTTA-2Model=LLAMA-3.2-3B, Reference Type=Dynamic-Ref2025.10 | 33.3 | 44.03 | 35.66 | 37.67 |