Intent Grasping on IntentGrasp All Set (downsample)
61.68F1 ScoreGemini 3.1-Pro
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 3.1-Prosample=sample-432026.05 | 61.68 | — | — | |
| Gemini 3.1-Prosample=sample-442026.05 | 60.41 | — | — | |
| Claude Opus-4.7sample=sample-442026.05 | 60.07 | — | — | |
| Claude Opus-4.7sample=sample-432026.05 | 58.41 | — | — | |
| Gemini 3-Flashsample=sample-442026.05 | 57.08 | — | — | |
| Gemini 3.1-Prosample=sample-422026.05 | 56.96 | — | — | |
| Gemini 3-Flashsample=sample-432026.05 | 55.55 | — | — | |
| Gemini 3-Flashsample=sample-422026.05 | 53.57 | — | — | |
| Claude Opus-4.7sample=sample-422026.05 | 53.44 | — | — | |
| Claude Sonnet-4.6sample=sample-432026.05 | 53.2 | — | — | |
| GPT 5.4sample=sample-442026.05 | 53.17 | — | — | |
| Gemini 3.1-Flash-Litesample=sample-432026.05 | 52.45 | — | — | |
| Gemini 3.1-Flash-Litesample=sample-442026.05 | 52.14 | — | — | |
| GPT 5.4sample=sample-432026.05 | 51.18 | — | — | |
| Claude Sonnet-4.6sample=sample-442026.05 | 50.35 | — | — | |
| Claude Haiku-4.5sample=sample-442026.05 | 49.25 | — | — | |
| Gemini 3.1-Flash-Litesample=sample-422026.05 | 48.38 | — | — | |
| Claude Haiku-4.5sample=sample-432026.05 | 48.35 | — | — | |
| GPT 5.4sample=sample-422026.05 | 47.71 | — | — | |
| Claude Sonnet-4.6sample=sample-422026.05 | 47.59 | — | — | |
| GPT 5.4-minisample=sample-442026.05 | 46.13 | — | — | |
| GPT 5.4-minisample=sample-432026.05 | 45.32 | — | — | |
| GPT 5.4-nanosample=sample-442026.05 | 43.7 | — | — | |
| Claude Haiku-4.5sample=sample-422026.05 | 42.12 | — | — | |
| GPT 5.4-minisample=sample-422026.05 | 40.68 | — | — | |
| GPT 5.4-nanosample=sample-422026.05 | 38.77 | — | — | |
| GPT 5.4-nanosample=sample-432026.05 | 38.58 | — | — | |
| Claude Haiku-4.5aggregation=Average across samples2026.05 | — | 46.57 | 3.17 | |
| Claude Opus-4.7aggregation=Average across samples2026.05 | — | 57.31 | 2.82 | |
| Claude Sonnet-4.6aggregation=Average across samples2026.05 | — | 50.38 | 2.29 | |
| Gemini 3-Flashaggregation=Average across samples2026.05 | — | 55.4 | 1.44 | |
| Gemini 3.1-Flash-Liteaggregation=Average across samples2026.05 | — | 50.99 | 1.85 | |
| Gemini 3.1-Proaggregation=Average across samples2026.05 | — | 59.68 | 1.99 | |
| GPT 5.4aggregation=Average across samples2026.05 | — | 50.69 | 2.26 | |
| GPT 5.4-miniaggregation=Average across samples2026.05 | — | 44.04 | 2.4 | |
| GPT 5.4-nanoaggregation=Average across samples2026.05 | — | 40.35 | 2.37 |