# Scientific Reasoning on OlympiadBench Source: [Scientific Reasoning on OlympiadBench]() Collection index: [llms.txt]() Data updated: 2026-07-03T06:16:42.796Z Evaluation of model performance on scientific reasoning tasks, including mathematics and physics, using the OlympiadBench dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Scientific Reasoning | | Dataset (including any stated split) | OlympiadBench | | Metric | Average Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average Score](): Higher is better; unit: SCALAR. - [Math Score](): Higher is better; unit: SCALAR. - [Physics Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Average Score | Result date | Report paper | | --- | --- | --- | --- | | Ouro-2.6B-Thinking-R4 | 76.4 | Not recorded | 6902f1003ef7ccd74e5e4419 | | Qwen3-8B | 75.3 | Not recorded | 6902f1003ef7ccd74e5e4419 | | Qwen3-4B | 73.2 | Not recorded | 6902f1003ef7ccd74e5e4419 | | Deepseek-Distill-Qwen-7B | 72 | Not recorded | 6902f1003ef7ccd74e5e4419 | | Ouro-1.4B-Thinking-R4 | 71.6 | Not recorded | 6902f1003ef7ccd74e5e4419 | | Deepseek-Distill-Qwen-1.5B | 56.44 | Not recorded | 6902f1003ef7ccd74e5e4419 | | Qwen3-1.7B | 56.4 | Not recorded | 6902f1003ef7ccd74e5e4419 | | TCA-Attention | 31.27 | Not recorded | 693a50246cfd488761a54ba7 | | Qwen2.5-7B-Instruct | 31.2 | Not recorded | 693a50246cfd488761a54ba7 | | MInference | 30.99 | Not recorded | 693a50246cfd488761a54ba7 | | XAttention | 30.9 | Not recorded | 693a50246cfd488761a54ba7 | | FlexPrefill | 30.86 | Not recorded | 693a50246cfd488761a54ba7 | | TCA-Attention | 12.15 | Not recorded | 693a50246cfd488761a54ba7 | | MInference | 12.08 | Not recorded | 693a50246cfd488761a54ba7 | | XAttention | 11.95 | Not recorded | 693a50246cfd488761a54ba7 | | LLaMA3.1-8B-Instruct | 11.84 | Not recorded | 693a50246cfd488761a54ba7 | | FlexPrefill | 11.57 | Not recorded | 693a50246cfd488761a54ba7 | ### Ouro-2.6B-Thinking-R4 Average Score: 76.4. Result ID: 6a4753ac60b88bb339919aff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"2.6B","Configuration":"Thinking-R4"}. This record does not assert independent reproduction. ### Qwen3-8B Average Score: 75.3. Result ID: 6a4753ac60b88bb339919b02. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B"}. This record does not assert independent reproduction. ### Qwen3-4B Average Score: 73.2. Result ID: 6a4753ac60b88bb339919b01. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"4B"}. This record does not assert independent reproduction. ### Deepseek-Distill-Qwen-7B Average Score: 72. Result ID: 6a4753ac60b88bb339919b04. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"7B"}. This record does not assert independent reproduction. ### Ouro-1.4B-Thinking-R4 Average Score: 71.6. Result ID: 6a4753ac60b88bb339919afe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"1.4B","Configuration":"Thinking-R4"}. This record does not assert independent reproduction. ### Deepseek-Distill-Qwen-1.5B Average Score: 56.44. Result ID: 6a4753ac60b88bb339919b03. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"1.5B"}. This record does not assert independent reproduction. ### Qwen3-1.7B Average Score: 56.4. Result ID: 6a4753ac60b88bb339919b00. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"1.7B"}. This record does not assert independent reproduction. ### TCA-Attention Average Score: 31.27. Result ID: 6996c5ae5698665ec4112a17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Attention Method":"TCA-Attention"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Average Score: 31.2. Result ID: 6996c5ae5698665ec4112a13. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Attention Method":"Full Attention"}. This record does not assert independent reproduction. ### MInference Average Score: 30.99. Result ID: 6996c5ae5698665ec4112a14. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Attention Method":"MInference"}. This record does not assert independent reproduction. ### XAttention Average Score: 30.9. Result ID: 6996c5ae5698665ec4112a16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Attention Method":"XAttention"}. This record does not assert independent reproduction. ### FlexPrefill Average Score: 30.86. Result ID: 6996c5ae5698665ec4112a15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Attention Method":"FlexPrefill"}. This record does not assert independent reproduction. ### TCA-Attention Average Score: 12.15. Result ID: 6996c5ae5698665ec4112a12. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"LLaMA3.1-8B-Instruct","Attention Method":"TCA-Attention"}. This record does not assert independent reproduction. ### MInference Average Score: 12.08. Result ID: 6996c5ae5698665ec4112a0f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"LLaMA3.1-8B-Instruct","Attention Method":"MInference"}. This record does not assert independent reproduction. ### XAttention Average Score: 11.95. Result ID: 6996c5ae5698665ec4112a11. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"LLaMA3.1-8B-Instruct","Attention Method":"XAttention"}. This record does not assert independent reproduction. ### LLaMA3.1-8B-Instruct Average Score: 11.84. Result ID: 6996c5ae5698665ec4112a0e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"LLaMA3.1-8B-Instruct","Attention Method":"Full Attention"}. This record does not assert independent reproduction. ### FlexPrefill Average Score: 11.57. Result ID: 6996c5ae5698665ec4112a10. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"LLaMA3.1-8B-Instruct","Attention Method":"FlexPrefill"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.