# Scientific Reasoning on HLE (test) Source: [Scientific Reasoning on HLE (test)]() Collection index: [llms.txt]() Data updated: 2026-05-22T08:20:55.485Z Evaluation of model performance on Scientific Reasoning tasks using the test split of the HLE dataset, reporting Pass@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Scientific Reasoning | | Dataset (including any stated split) | HLE (test) | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | GPT-5.4-xhigh | 49 | Not recorded | 6a0fe32ef984739ed6739f26 | | GPT-5.4-xhigh | 40.8 | Not recorded | 6a0fe32ef984739ed6739f26 | | DeepSeek-V3.2 | 39.6 | Not recorded | 6a0fe32ef984739ed6739f26 | | Kimi-K2.5 | 34.4 | Not recorded | 6a0fe32ef984739ed6739f26 | | Tongyi-DeepResearch | 31.6 | Not recorded | 6a0fe32ef984739ed6739f26 | | SR2AM-v1.0-30B | 30.4 | Not recorded | 6a0fe32ef984739ed6739f26 | | MiroThinker-v1.5 | 30 | Not recorded | 6a0fe32ef984739ed6739f26 | | DeepSeek-V3.2 | 24 | Not recorded | 6a0fe32ef984739ed6739f26 | | GPT-OSS-120B-high | 20.4 | Not recorded | 6a0fe32ef984739ed6739f26 | | GLM-4.6 | 19 | Not recorded | 6a0fe32ef984739ed6739f26 | | ASearcher-Web-QWQ-v2 | 16.4 | Not recorded | 6a0fe32ef984739ed6739f26 | | WebExplorer-8B | 14.8 | Not recorded | 6a0fe32ef984739ed6739f26 | | SR2AM-v0.1-8B | 14.2 | Not recorded | 6a0fe32ef984739ed6739f26 | | WebSailor-32B | 12.2 | Not recorded | 6a0fe32ef984739ed6739f26 | | Qwen3-235B | 12 | Not recorded | 6a0fe32ef984739ed6739f26 | | K2-Think-V2-high | 11.6 | Not recorded | 6a0fe32ef984739ed6739f26 | | A2FM | 10.3 | Not recorded | 6a0fe32ef984739ed6739f26 | | Qwen3-30B-A3B | 10.2 | Not recorded | 6a0fe32ef984739ed6739f26 | | Qwen3-30B-A3B | 9.8 | Not recorded | 6a0fe32ef984739ed6739f26 | | WebSailor-7B | 9.4 | Not recorded | 6a0fe32ef984739ed6739f26 | | AFM-Web-7B-RL | 7.8 | Not recorded | 6a0fe32ef984739ed6739f26 | | ASearcher-Web-7B | 6.8 | Not recorded | 6a0fe32ef984739ed6739f26 | | SimpleTIR-7B | 4.6 | Not recorded | 6a0fe32ef984739ed6739f26 | | SimpleTIR-32B | 3.6 | Not recorded | 6a0fe32ef984739ed6739f26 | | AFM-Code-7B-RL | 2.8 | Not recorded | 6a0fe32ef984739ed6739f26 | ### GPT-5.4-xhigh Pass@1: 49. Result ID: 6a1011e2fbc373592c08fa48. [Reporting paper]() Recorded evaluation configuration: {"Size":"—","# Reas. Tokens":"4821.8","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### GPT-5.4-xhigh Pass@1: 40.8. Result ID: 6a1011e2fbc373592c08fa41. [Reporting paper]() Recorded evaluation configuration: {"Size":"—","# Reas. Tokens":"8810.3","Reasoning Paradigm":"Reasoning LLMs"}. This record does not assert independent reproduction. ### DeepSeek-V3.2 Pass@1: 39.6. Result ID: 6a1011e2fbc373592c08fa46. [Reporting paper]() Recorded evaluation configuration: {"Size":"685B","# Reas. Tokens":"3011.5","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### Kimi-K2.5 Pass@1: 34.4. Result ID: 6a1011e2fbc373592c08fa47. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"1024B","# Reas. Tokens":"6413.1","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### Tongyi-DeepResearch Pass@1: 31.6. Result ID: 6a1011e2fbc373592c08fa50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B","# Reas. Tokens":"7431.8","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### SR2AM-v1.0-30B Pass@1: 30.4. Result ID: 6a1011e2fbc373592c08fa56. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"30B","# Reas. Tokens":"5517.7","Reasoning Paradigm":"Self-Regulated Simulative Reasoning"}. This record does not assert independent reproduction. ### MiroThinker-v1.5 Pass@1: 30. Result ID: 6a1011e2fbc373592c08fa51. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"30B","# Reas. Tokens":"11295.2","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### DeepSeek-V3.2 Pass@1: 24. Result ID: 6a1011e2fbc373592c08fa40. [Reporting paper]() Recorded evaluation configuration: {"Size":"685B","# Reas. Tokens":"9306.2","Reasoning Paradigm":"Reasoning LLMs"}. This record does not assert independent reproduction. ### GPT-OSS-120B-high Pass@1: 20.4. Result ID: 6a1011e2fbc373592c08fa43. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"120B","# Reas. Tokens":"2700.4","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### GLM-4.6 Pass@1: 19. Result ID: 6a1011e2fbc373592c08fa45. [Reporting paper]() Recorded evaluation configuration: {"Size":"357B","# Reas. Tokens":"3580.4","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### ASearcher-Web-QWQ-v2 Pass@1: 16.4. Result ID: 6a1011e2fbc373592c08fa4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"32B","# Reas. Tokens":"116752.9","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### WebExplorer-8B Pass@1: 14.8. Result ID: 6a1011e2fbc373592c08fa4c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8B","# Reas. Tokens":"3616.9","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### SR2AM-v0.1-8B Pass@1: 14.2. Result ID: 6a1011e2fbc373592c08fa55. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"8B","# Reas. Tokens":"3697.6","Reasoning Paradigm":"Self-Regulated Simulative Reasoning"}. This record does not assert independent reproduction. ### WebSailor-32B Pass@1: 12.2. Result ID: 6a1011e2fbc373592c08fa4e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"32B","# Reas. Tokens":"1055.2","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### Qwen3-235B Pass@1: 12. Result ID: 6a1011e2fbc373592c08fa44. [Reporting paper]() Recorded evaluation configuration: {"Size":"235B","# Reas. Tokens":"6467.4","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### K2-Think-V2-high Pass@1: 11.6. Result ID: 6a1011e2fbc373592c08fa3f. [Reporting paper]() Recorded evaluation configuration: {"Size":"73B","# Reas. Tokens":"31905.2","Reasoning Paradigm":"Reasoning LLMs"}. This record does not assert independent reproduction. ### A2FM Pass@1: 10.3. Result ID: 6a1011e2fbc373592c08fa54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"32B","# Reas. Tokens":"23424.8","Reasoning Paradigm":"Mode Routing"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B Pass@1: 10.2. Result ID: 6a1011e2fbc373592c08fa3e. [Reporting paper]() Recorded evaluation configuration: {"Size":"30B","# Reas. Tokens":"8269.1","Reasoning Paradigm":"Reasoning LLMs"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B Pass@1: 9.8. Result ID: 6a1011e2fbc373592c08fa42. [Reporting paper]() Recorded evaluation configuration: {"Size":"30B","# Reas. Tokens":"5410.1","Reasoning Paradigm":"Pretrained LLMs + Tools"}. This record does not assert independent reproduction. ### WebSailor-7B Pass@1: 9.4. Result ID: 6a1011e2fbc373592c08fa4b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","# Reas. Tokens":"2211.5","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### AFM-Web-7B-RL Pass@1: 7.8. Result ID: 6a1011e2fbc373592c08fa52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","# Reas. Tokens":"2608.6","Reasoning Paradigm":"Workflow Distillation"}. This record does not assert independent reproduction. ### ASearcher-Web-7B Pass@1: 6.8. Result ID: 6a1011e2fbc373592c08fa49. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","# Reas. Tokens":"601.4","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### SimpleTIR-7B Pass@1: 4.6. Result ID: 6a1011e2fbc373592c08fa4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","# Reas. Tokens":"3551.7","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### SimpleTIR-32B Pass@1: 3.6. Result ID: 6a1011e2fbc373592c08fa4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"32B","# Reas. Tokens":"5174.8","Reasoning Paradigm":"Unregulated Deliberation"}. This record does not assert independent reproduction. ### AFM-Code-7B-RL Pass@1: 2.8. Result ID: 6a1011e2fbc373592c08fa53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","# Reas. Tokens":"11205.5","Reasoning Paradigm":"Workflow Distillation"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.