# Constraint Following on FollowBench SSR Source: [Constraint Following on FollowBench SSR]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:05:38.483Z Evaluation of model performance on constraint following tasks using the FollowBench SSR dataset, measured across five distinct difficulty levels (Level 1 to Level 5). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Constraint Following | | Dataset (including any stated split) | FollowBench SSR | | Metric | Success Rate (Level 1) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Success Rate (Level 1)](): Higher is better; unit: PERCENT. - [Success Rate (Level 2)](): Higher is better; unit: PERCENT. - [Success Rate (Level 3)](): Higher is better; unit: PERCENT. - [Success Rate (Level 4)](): Higher is better; unit: PERCENT. - [Success Rate (Level 5)](): Higher is better; unit: PERCENT. - [Average Success Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Success Rate (Level 1) | Result date | Report paper | | --- | --- | --- | --- | | GPT-4+ | 84.7 | Not recorded | 68fa74251e209131cea5ffa5 | | GPT-3.5 Turbo | 80.3 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (Qwen2-72B-Instruct w/ Online DPO) | 76.2 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (LLaMA-3-70B w/ Online DPO) | 71 | Not recorded | 68fa74251e209131cea5ffa5 | | Qwen2-72B-Instruct | 70.2 | Not recorded | 68fa74251e209131cea5ffa5 | | Mixtral-8x22B | 63.9 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (Qwen2-7B Offline DPO) | 61.4 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (Qwen2-7B Online DPO) | 61.4 | Not recorded | 68fa74251e209131cea5ffa5 | | LLaMA3-70B-Instruct | 60.7 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (Qwen2-7B-SFT) | 60.2 | Not recorded | 68fa74251e209131cea5ffa5 | | Qwen2-7B(ShareGPT) | 56.1 | Not recorded | 68fa74251e209131cea5ffa5 | | Qwen2-7B | 55.6 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (LLaMA3-8B Online DPO) | 54.6 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (LLaMA3-8B Offline DPO) | 51.9 | Not recorded | 68fa74251e209131cea5ffa5 | | AUTOIF (LLaMA3-8B-SFT) | 46.6 | Not recorded | 68fa74251e209131cea5ffa5 | | LLaMA3-8B(ShareGPT) | 44 | Not recorded | 68fa74251e209131cea5ffa5 | | Mistral-7B | 40.1 | Not recorded | 68fa74251e209131cea5ffa5 | | LLaMA3-8B | 10 | Not recorded | 68fa74251e209131cea5ffa5 | ### GPT-4+ Success Rate (Level 1): 84.7. Result ID: 699332893a6c314849145e89. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GPT-3.5 Turbo Success Rate (Level 1): 80.3. Result ID: 699332893a6c314849145e8a. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### AUTOIF (Qwen2-72B-Instruct w/ Online DPO) Success Rate (Level 1): 76.2. Result ID: 699332893a6c314849145e8e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-72B-Instruct","Supervision Model":"Qwen2-72B","Alignment Strategy":"Self-Alignment","Training Method":"Online DPO"}. This record does not assert independent reproduction. ### AUTOIF (LLaMA-3-70B w/ Online DPO) Success Rate (Level 1): 71. Result ID: 699332893a6c314849145e92. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-70B-Instruct","Supervision Model":"LLaMA3-70B","Alignment Strategy":"Self-Alignment","Training Method":"Online DPO"}. This record does not assert independent reproduction. ### Qwen2-72B-Instruct Success Rate (Level 1): 70.2. Result ID: 699332893a6c314849145e86. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-72B-Instruct"}. This record does not assert independent reproduction. ### Mixtral-8x22B Success Rate (Level 1): 63.9. Result ID: 699332893a6c314849145e88. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Mixtral-8x22B"}. This record does not assert independent reproduction. ### AUTOIF (Qwen2-7B Offline DPO) Success Rate (Level 1): 61.4. Result ID: 699332893a6c314849145e8c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-7B","Supervision Model":"Qwen2-72B","Alignment Strategy":"Strong-to-Weak","Training Method":"Offline DPO"}. This record does not assert independent reproduction. ### AUTOIF (Qwen2-7B Online DPO) Success Rate (Level 1): 61.4. Result ID: 699332893a6c314849145e8d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-7B","Supervision Model":"Qwen2-72B","Alignment Strategy":"Strong-to-Weak","Training Method":"Online DPO"}. This record does not assert independent reproduction. ### LLaMA3-70B-Instruct Success Rate (Level 1): 60.7. Result ID: 699332893a6c314849145e87. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-70B-Instruct"}. This record does not assert independent reproduction. ### AUTOIF (Qwen2-7B-SFT) Success Rate (Level 1): 60.2. Result ID: 699332893a6c314849145e8b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-7B","Supervision Model":"Qwen2-72B","Alignment Strategy":"Strong-to-Weak","Training Method":"SFT"}. This record does not assert independent reproduction. ### Qwen2-7B(ShareGPT) Success Rate (Level 1): 56.1. Result ID: 699332893a6c314849145e82. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-7B","Training Data":"ShareGPT"}. This record does not assert independent reproduction. ### Qwen2-7B Success Rate (Level 1): 55.6. Result ID: 699332893a6c314849145e81. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone Model":"Qwen2-7B"}. This record does not assert independent reproduction. ### AUTOIF (LLaMA3-8B Online DPO) Success Rate (Level 1): 54.6. Result ID: 699332893a6c314849145e91. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-8B","Supervision Model":"LLaMA3-70B","Alignment Strategy":"Strong-to-Weak","Training Method":"Online DPO"}. This record does not assert independent reproduction. ### AUTOIF (LLaMA3-8B Offline DPO) Success Rate (Level 1): 51.9. Result ID: 699332893a6c314849145e90. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-8B","Supervision Model":"LLaMA3-70B","Alignment Strategy":"Strong-to-Weak","Training Method":"Offline DPO"}. This record does not assert independent reproduction. ### AUTOIF (LLaMA3-8B-SFT) Success Rate (Level 1): 46.6. Result ID: 699332893a6c314849145e8f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-8B","Supervision Model":"LLaMA3-70B","Alignment Strategy":"Strong-to-Weak","Training Method":"SFT"}. This record does not assert independent reproduction. ### LLaMA3-8B(ShareGPT) Success Rate (Level 1): 44. Result ID: 699332893a6c314849145e84. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-8B","Training Data":"ShareGPT"}. This record does not assert independent reproduction. ### Mistral-7B Success Rate (Level 1): 40.1. Result ID: 699332893a6c314849145e85. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone Model":"Mistral-7B"}. This record does not assert independent reproduction. ### LLaMA3-8B Success Rate (Level 1): 10. Result ID: 699332893a6c314849145e83. [Reporting paper]() Recorded evaluation configuration: {"Backbone Model":"LLaMA3-8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.