# Alignment on IFEval strict prompt Source: [Alignment on IFEval strict prompt]() Collection index: [llms.txt]() Data updated: 2026-05-19T09:01:02.675Z Evaluation of model Alignment capabilities on the IFEval dataset using the strict prompt configuration, measured by pass@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Alignment | | Dataset (including any stated split) | IFEval strict prompt | | Metric | pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 26 of 26 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [pass@1](): Higher is better; unit: PERCENT. - [TPF](): Higher is better; unit: PERCENT. ## Recorded results | Method | pass@1 | Result date | Report paper | | --- | --- | --- | --- | | Nemotron Cascade-8B | 90.2 | Not recorded | 6940e7e2456146936284fca5 | | Gemini-2.5 Flash-Thinking | 89.8 | Not recorded | 6940e7e2456146936284fca5 | | SnapMLA | 87.8 | Not recorded | 698d5ec5c97a0b3e3b5e3f0d | | SnapMLA | 87.25 | Not recorded | 698d5ec5c97a0b3e3b5e3f0d | | FlashMLA | 86.9 | Not recorded | 698d5ec5c97a0b3e3b5e3f0d | | FlashMLA | 86.32 | Not recorded | 698d5ec5c97a0b3e3b5e3f0d | | Qwen3-14B | 86.14 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Nemotron-Nano 9B-v2 | 86.1 | Not recorded | 6940e7e2456146936284fca5 | | Qwen3 14B | 85.4 | Not recorded | 6940e7e2456146936284fca5 | | Qwen3 8B | 85 | Not recorded | 6940e7e2456146936284fca5 | | Qwen3-8B | 84.29 | Not recorded | 698abc32c97a0b3e3b5de49e | | DeepSeek-R1 0528 671B | 84.1 | Not recorded | 6940e7e2456146936284fca5 | | Qwen3-8B | 84.1 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Qwen3-14B + NGM | 83.92 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Qwen3-8B + NGM | 83.55 | Not recorded | 6a0beeb6dbc836b87d82c54f | | LLaDA2.1-mini | 83.18 | Not recorded | 698abc32c97a0b3e3b5de49e | | Nemotron-Cascade 14B-Thinking | 81.9 | Not recorded | 6940e7e2456146936284fca5 | | LLaDA2.1-mini | 81.33 | Not recorded | 698abc32c97a0b3e3b5de49e | | LLaDA2.0-mini | 80.78 | Not recorded | 698abc32c97a0b3e3b5de49e | | Qwen3-4B + NGM | 80.41 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Qwen3-4B | 80.22 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Ling-mini-2.0 | 76.16 | Not recorded | 698abc32c97a0b3e3b5de49e | | Qwen3-1.7B | 68.95 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Qwen3-1.7B + NGM | 67.47 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Qwen3-0.6B | 58.78 | Not recorded | 6a0beeb6dbc836b87d82c54f | | Qwen3-0.6B + NGM | 57.86 | Not recorded | 6a0beeb6dbc836b87d82c54f | ### Nemotron Cascade-8B pass@1: 90.2. Result ID: 6996bfff5698665ec40b1adc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B","Thinking Mode":"best of thinking/non-thinking"}. This record does not assert independent reproduction. ### Gemini-2.5 Flash-Thinking pass@1: 89.8. Result ID: 6996bfff5698665ec40b1adb. [Reporting paper]() Recorded evaluation configuration: {"Thinking Mode":"true"}. This record does not assert independent reproduction. ### SnapMLA pass@1: 87.8. Result ID: 6998c5195698665ec4286318. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LongCat-Flash-thinking","Precision":"FP8"}. This record does not assert independent reproduction. ### SnapMLA pass@1: 87.25. Result ID: 6998c5195698665ec4286316. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.1","Precision":"FP8"}. This record does not assert independent reproduction. ### FlashMLA pass@1: 86.9. Result ID: 6998c5195698665ec4286317. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LongCat-Flash-thinking","Precision":"BF16"}. This record does not assert independent reproduction. ### FlashMLA pass@1: 86.32. Result ID: 6998c5195698665ec4286315. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V3.1","Precision":"BF16"}. This record does not assert independent reproduction. ### Qwen3-14B pass@1: 86.14. Result ID: 6a0c26c2fbc373592c02620c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"14B","NGM Configuration":"False","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Nemotron-Nano 9B-v2 pass@1: 86.1. Result ID: 6996bfff5698665ec40b1ad8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"9B-v2"}. This record does not assert independent reproduction. ### Qwen3 14B pass@1: 85.4. Result ID: 6996bfff5698665ec40b1ad9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"14B"}. This record does not assert independent reproduction. ### Qwen3 8B pass@1: 85. Result ID: 6996bfff5698665ec40b1ad7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### Qwen3-8B pass@1: 84.29. Result ID: 6998d5a85698665ec4396d43. [Reporting paper]() Recorded evaluation configuration: {"no think":true}. This record does not assert independent reproduction. ### DeepSeek-R1 0528 671B pass@1: 84.1. Result ID: 6996bfff5698665ec40b1ada. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"671B","Thinking Mode":"true"}. This record does not assert independent reproduction. ### Qwen3-8B pass@1: 84.1. Result ID: 6a0c26c2fbc373592c02620a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"8B","NGM Configuration":"False","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Qwen3-14B + NGM pass@1: 83.92. Result ID: 6a0c26c2fbc373592c02620d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"14B","NGM Configuration":"True","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Qwen3-8B + NGM pass@1: 83.55. Result ID: 6a0c26c2fbc373592c02620b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"8B","NGM Configuration":"True","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### LLaDA2.1-mini pass@1: 83.18. Result ID: 6998d5a85698665ec4396d47. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"mode":"Q Mode"}. This record does not assert independent reproduction. ### Nemotron-Cascade 14B-Thinking pass@1: 81.9. Result ID: 6996bfff5698665ec40b1add. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"14B","Thinking Mode":"best of thinking/non-thinking"}. This record does not assert independent reproduction. ### LLaDA2.1-mini pass@1: 81.33. Result ID: 6998d5a85698665ec4396d46. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"mode":"S Mode"}. This record does not assert independent reproduction. ### LLaDA2.0-mini pass@1: 80.78. Result ID: 6998d5a85698665ec4396d45. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-4B + NGM pass@1: 80.41. Result ID: 6a0c26c2fbc373592c026209. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"4B","NGM Configuration":"True","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Qwen3-4B pass@1: 80.22. Result ID: 6a0c26c2fbc373592c026208. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"4B","NGM Configuration":"False","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Ling-mini-2.0 pass@1: 76.16. Result ID: 6998d5a85698665ec4396d44. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen3-1.7B pass@1: 68.95. Result ID: 6a0c26c2fbc373592c026206. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"1.7B","NGM Configuration":"False","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Qwen3-1.7B + NGM pass@1: 67.47. Result ID: 6a0c26c2fbc373592c026207. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"1.7B","NGM Configuration":"True","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Qwen3-0.6B pass@1: 58.78. Result ID: 6a0c26c2fbc373592c026204. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"0.6B","NGM Configuration":"False","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. ### Qwen3-0.6B + NGM pass@1: 57.86. Result ID: 6a0c26c2fbc373592c026205. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"0.6B","NGM Configuration":"True","Decoding Settings":"identical decoding settings"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.