Property Extraction on Materials Science Scientific Tables Property expert-annotated (368 tables) (test)
92.4PrecisionMatSKRAFT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MatSKRAFTTime (s/table)=0.222025.09 | 92.4 | 86.45 | 89.33 | |
| Gemini-3.5-FlashTime (s/table)=3.972025.09 | 91.45 | 48.61 | 63.48 | |
| DeepSeek-V4-Pro-ThTime (s/table)=23.312025.09 | 89.02 | 79.91 | 84.22 | |
| Claude-Opus-4.7Time (s/table)=3.352025.09 | 88.18 | 82.67 | 85.65 | |
| GPT-5.4Time (s/table)=2.052025.09 | 79.64 | 78.69 | 79.17 | |
| Claude-3.5-SonnetTime (s/table)=4.902025.09 | 78.8 | 67.64 | 72.79 | |
| DeepSeek-V3Time (s/table)=14.952025.09 | 75.17 | 71.48 | 73.28 | |
| DeepSeek-V4-ProTime (s/table)=4.292025.09 | 71.91 | 65.6 | 68.61 | |
| DeepSeek-R1Time (s/table)=114.942025.09 | 70.43 | 68.07 | 69.23 | |
| Gemini-1.5-ProTime (s/table)=8.252025.09 | 66.09 | 58.13 | 61.85 | |
| GPT-4oTime (s/table)=5.842025.09 | 61.61 | 59.03 | 60.29 |