# Intent Grasping on IntentGrasp Gem Set Source: [Intent Grasping on IntentGrasp Gem Set]() Collection index: [llms.txt]() Data updated: 2026-05-11T07:31:28.647Z Evaluation of Intent Grasping performance on the IntentGrasp Gem Set dataset, reporting metrics such as F1 score, Average, and Standard Deviation. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Intent Grasping | | Dataset (including any stated split) | IntentGrasp Gem Set | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 45 of 45 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. - [Standard Deviation](): Lower is better; unit: SCALAR. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | Gemini 3-Flash | 0.2499 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3-Flash | 0.2498 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3-Flash | 0.2495 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3-Flash | 0.2375 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Pro | 0.2214 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Pro | 0.2184 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Pro | 0.2154 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Pro | 0.2051 | Not recorded | 6a01628adbc836b87d823071 | | Claude Opus-4.7 | 0.1801 | Not recorded | 6a01628adbc836b87d823071 | | Claude Opus-4.7 | 0.1623 | Not recorded | 6a01628adbc836b87d823071 | | Claude Opus-4.7 | 0.1614 | Not recorded | 6a01628adbc836b87d823071 | | Claude Opus-4.7 | 0.1581 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4 | 0.1275 | Not recorded | 6a01628adbc836b87d823071 | | Claude Sonnet-4.6 | 0.1238 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Flash-Lite | 0.1212 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4 | 0.1189 | Not recorded | 6a01628adbc836b87d823071 | | Claude Sonnet-4.6 | 0.1184 | Not recorded | 6a01628adbc836b87d823071 | | Claude Sonnet-4.6 | 0.1174 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Flash-Lite | 0.1167 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Flash-Lite | 0.1166 | Not recorded | 6a01628adbc836b87d823071 | | Gemini 3.1-Flash-Lite | 0.1162 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4 | 0.114 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4 | 0.1079 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-nano | 0.0898 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-nano | 0.0863 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-nano | 0.0862 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-mini | 0.0855 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-nano | 0.0847 | Not recorded | 6a01628adbc836b87d823071 | | Claude Sonnet-4.6 | 0.0766 | Not recorded | 6a01628adbc836b87d823071 | | Claude Haiku-4.5 | 0.0723 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-mini | 0.0688 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-mini | 0.067 | Not recorded | 6a01628adbc836b87d823071 | | GPT 5.4-mini | 0.0617 | Not recorded | 6a01628adbc836b87d823071 | | Claude Haiku-4.5 | 0.0585 | Not recorded | 6a01628adbc836b87d823071 | | Claude Haiku-4.5 | 0.054 | Not recorded | 6a01628adbc836b87d823071 | | Claude Haiku-4.5 | 0.0454 | Not recorded | 6a01628adbc836b87d823071 | ### Gemini 3-Flash F1 Score: 0.2499. Result ID: 6a0185bb47f34ce924e2ffb3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### Gemini 3-Flash F1 Score: 0.2498. Result ID: 6a0185bb47f34ce924e2ffb4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### Gemini 3-Flash F1 Score: 0.2495. Result ID: 6a0185bb47f34ce924e2ffb5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### Gemini 3-Flash F1 Score: 0.2375. Result ID: 6a0185bb47f34ce924e2ffb6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Gemini 3.1-Pro F1 Score: 0.2214. Result ID: 6a0185bb47f34ce924e2ffaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### Gemini 3.1-Pro F1 Score: 0.2184. Result ID: 6a0185bb47f34ce924e2ffb0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### Gemini 3.1-Pro F1 Score: 0.2154. Result ID: 6a0185bb47f34ce924e2ffae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### Gemini 3.1-Pro F1 Score: 0.2051. Result ID: 6a0185bb47f34ce924e2ffb1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Claude Opus-4.7 F1 Score: 0.1801. Result ID: 6a0185bb47f34ce924e2ffbf. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### Claude Opus-4.7 F1 Score: 0.1623. Result ID: 6a0185bb47f34ce924e2ffbd. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### Claude Opus-4.7 F1 Score: 0.1614. Result ID: 6a0185bb47f34ce924e2ffbe. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### Claude Opus-4.7 F1 Score: 0.1581. Result ID: 6a0185bb47f34ce924e2ffc0. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### GPT 5.4 F1 Score: 0.1275. Result ID: 6a0185bb47f34ce924e2ffa1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### Claude Sonnet-4.6 F1 Score: 0.1238. Result ID: 6a0185bb47f34ce924e2ffc4. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### Gemini 3.1-Flash-Lite F1 Score: 0.1212. Result ID: 6a0185bb47f34ce924e2ffba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### GPT 5.4 F1 Score: 0.1189. Result ID: 6a0185bb47f34ce924e2ffa2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Claude Sonnet-4.6 F1 Score: 0.1184. Result ID: 6a0185bb47f34ce924e2ffc3. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### Claude Sonnet-4.6 F1 Score: 0.1174. Result ID: 6a0185bb47f34ce924e2ffc5. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Gemini 3.1-Flash-Lite F1 Score: 0.1167. Result ID: 6a0185bb47f34ce924e2ffbb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Gemini 3.1-Flash-Lite F1 Score: 0.1166. Result ID: 6a0185bb47f34ce924e2ffb8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### Gemini 3.1-Flash-Lite F1 Score: 0.1162. Result ID: 6a0185bb47f34ce924e2ffb9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### GPT 5.4 F1 Score: 0.114. Result ID: 6a0185bb47f34ce924e2ffa0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### GPT 5.4 F1 Score: 0.1079. Result ID: 6a0185bb47f34ce924e2ff9f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### GPT 5.4-nano F1 Score: 0.0898. Result ID: 6a0185bb47f34ce924e2ffab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### GPT 5.4-nano F1 Score: 0.0863. Result ID: 6a0185bb47f34ce924e2ffa9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### GPT 5.4-nano F1 Score: 0.0862. Result ID: 6a0185bb47f34ce924e2ffaa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### GPT 5.4-mini F1 Score: 0.0855. Result ID: 6a0185bb47f34ce924e2ffa7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### GPT 5.4-nano F1 Score: 0.0847. Result ID: 6a0185bb47f34ce924e2ffac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Claude Sonnet-4.6 F1 Score: 0.0766. Result ID: 6a0185bb47f34ce924e2ffc2. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### Claude Haiku-4.5 F1 Score: 0.0723. Result ID: 6a0185bb47f34ce924e2ffc9. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### GPT 5.4-mini F1 Score: 0.0688. Result ID: 6a0185bb47f34ce924e2ffa6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-42"}. This record does not assert independent reproduction. ### GPT 5.4-mini F1 Score: 0.067. Result ID: 6a0185bb47f34ce924e2ffa5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### GPT 5.4-mini F1 Score: 0.0617. Result ID: 6a0185bb47f34ce924e2ffa4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. ### Claude Haiku-4.5 F1 Score: 0.0585. Result ID: 6a0185bb47f34ce924e2ffca. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-365"}. This record does not assert independent reproduction. ### Claude Haiku-4.5 F1 Score: 0.054. Result ID: 6a0185bb47f34ce924e2ffc8. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"shuffle-7"}. This record does not assert independent reproduction. ### Claude Haiku-4.5 F1 Score: 0.0454. Result ID: 6a0185bb47f34ce924e2ffc7. [Reporting paper]() Recorded evaluation configuration: {"shuffle":"no shuffle"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.