# Tool-based Question Answering on ToolQA hard Source: [Tool-based Question Answering on ToolQA hard]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:34:30.125Z Evaluation of model performance on the 'hard' subset of the ToolQA dataset, focusing on Tool-based Question Answering capabilities across various domains (Yelp, SciREX, Flight, Airbnb, Coffee). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Tool-based Question Answering | | Dataset (including any stated split) | ToolQA hard | | Metric | Accuracy (Yelp) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy (Yelp)](): Higher is better; unit: PERCENT. - [Accuracy (SciREX)](): Higher is better; unit: PERCENT. - [Accuracy (Flight)](): Higher is better; unit: PERCENT. - [Accuracy (Airbnb)](): Higher is better; unit: PERCENT. - [Accuracy (Coffee)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy (Yelp) | Result date | Report paper | | --- | --- | --- | --- | | React | 63 | Not recorded | 69311572d6b0dd3342933a6d | | React | 51 | Not recorded | 69311572d6b0dd3342933a6d | | React | 47 | Not recorded | 69311572d6b0dd3342933a6d | | Reflexion | 43 | Not recorded | 69311572d6b0dd3342933a6d | | RP-ReAct | 37 | Not recorded | 69311572d6b0dd3342933a6d | | Reflexion | 29 | Not recorded | 69311572d6b0dd3342933a6d | | RP-ReAct | 27 | Not recorded | 69311572d6b0dd3342933a6d | | Reflexion | 24 | Not recorded | 69311572d6b0dd3342933a6d | | RP-ReAct | 21 | Not recorded | 69311572d6b0dd3342933a6d | | RP-ReAct | 16 | Not recorded | 69311572d6b0dd3342933a6d | | ReAct | 10 | Not recorded | 69311572d6b0dd3342933a6d | | ReAct | 10 | Not recorded | 69311572d6b0dd3342933a6d | | React-100 | 6 | Not recorded | 69311572d6b0dd3342933a6d | | Reflexion | 6 | Not recorded | 69311572d6b0dd3342933a6d | | RP-ReAct | 3 | Not recorded | 69311572d6b0dd3342933a6d | | React | 2 | Not recorded | 69311572d6b0dd3342933a6d | | RP-ReAct | 1 | Not recorded | 69311572d6b0dd3342933a6d | ### React Accuracy (Yelp): 63. Result ID: 699631085698665ec472f3ea. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-oss-120b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### React Accuracy (Yelp): 51. Result ID: 699631085698665ec472f3ed. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen3-32B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### React Accuracy (Yelp): 47. Result ID: 699631085698665ec472f3f4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen3-14B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### Reflexion Accuracy (Yelp): 43. Result ID: 699631085698665ec472f3eb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-oss-120b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### RP-ReAct Accuracy (Yelp): 37. Result ID: 699631085698665ec472f3ec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-oss-120b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### Reflexion Accuracy (Yelp): 29. Result ID: 699631085698665ec472f3ee. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen3-32B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### RP-ReAct Accuracy (Yelp): 27. Result ID: 699631085698665ec472f3ef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-32B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### Reflexion Accuracy (Yelp): 24. Result ID: 699631085698665ec472f3f5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen3-14B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### RP-ReAct Accuracy (Yelp): 21. Result ID: 699631085698665ec472f3f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-14B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### RP-ReAct Accuracy (Yelp): 16. Result ID: 699631085698665ec472f3f3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-oss-20b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### ReAct Accuracy (Yelp): 10. Result ID: 699631085698665ec472f3f7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"DeepSeek-8B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### ReAct Accuracy (Yelp): 10. Result ID: 699631085698665ec472f3f9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"DeepSeek-7B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### React-100 Accuracy (Yelp): 6. Result ID: 699631085698665ec472f3f1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-oss-20b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### Reflexion Accuracy (Yelp): 6. Result ID: 699631085698665ec472f3f2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-oss-20b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### RP-ReAct Accuracy (Yelp): 3. Result ID: 699631085698665ec472f3f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-8B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### React Accuracy (Yelp): 2. Result ID: 699631085698665ec472f3f0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-oss-20b","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. ### RP-ReAct Accuracy (Yelp): 1. Result ID: 699631085698665ec472f3fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-7B","Domain examples in prompt (P)":"Yelp and Flight only"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.