Reaction Position Prediction on USPTO-LLM (test)
16.3Average Number of PredictionsGemini-2.5-Flash
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini-2.5-FlashTotal predictions=539, Failed predictions=22025.10 | 16.3 | 91.84 | 61.6 | 35.81 | 75 | |
| GPT-5Total predictions=538, Failed predictions=32025.10 | 15.1 | 91.08 | 54.28 | 47.03 | 70 | |
| Gemini-2.5-ProTotal predictions=541, Failed predictions=02025.10 | 11.1 | 91.87 | 66.54 | 40.3 | 77 | |
| Claude Sonnet 4Total predictions=538, Failed predictions=32025.10 | 10 | 92.57 | 58.55 | 39.03 | 74 | |
| DeepSeek-R1-670BTotal predictions=541, Failed predictions=02025.10 | 7.3 | 87.25 | 53.23 | 29.21 | 67 | |
| Qwen3-235BTotal predictions=539, Failed predictions=22025.10 | 5.9 | 88.5 | 58.07 | 25.97 | 70 | |
| Qwen3-4BTotal predictions=541, Failed predictions=02025.10 | 4 | 73.01 | 31.61 | 3.51 | 48 | |
| Qwen3-30BTotal predictions=541, Failed predictions=02025.10 | 3.8 | 74.86 | 34.75 | 14.23 | 49 |