Extracting Composition on MaTSKRAFT MCC and NC tables (test)
82.31PrecisionMatSKRAFT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MatSKRAFTTime (s/table)=0.392025.09 | 82.31 | 62.97 | 71.35 | |
| Matskraft2025.09 | 79.87 | 60.37 | 68.77 | |
| TAPEX-MSBBackbone=MatSciBERT2025.09 | 70.07 | 55.07 | 61.67 | |
| TaBERT-MSBBackbone=MatSciBERT2025.09 | 67.66 | 43.88 | 53.23 | |
| Claude-Opus-4.7Time (s/table)=1.382025.09 | 64.69 | 64.32 | 64.51 | |
| TAPAS-MSBBackbone=MatSciBERT2025.09 | 63.58 | 51.98 | 57.2 | |
| Gemini-3.5-FlashTime (s/table)=2.232025.09 | 61.54 | 61.77 | 61.65 | |
| TaBERT2025.09 | 60.93 | 37.09 | 46.11 | |
| TaBERT-BBackbone=bert-base-uncased2025.09 | 60.57 | 37.62 | 46.41 | |
| DeepSeek-V4-ProTime (s/table)=5.862025.09 | 55.92 | 52.59 | 54.2 | |
| TAPAS2025.09 | 55.4 | 42.47 | 48.08 | |
| DeepSeek-R1Time (s/table)=187.592025.09 | 52.85 | 55.79 | 54.28 | |
| GPT-5.4Time (s/table)=1.762025.09 | 52.52 | 63.08 | 57.32 | |
| DeepSeek-V4-Pro-ThTime (s/table)=37.442025.09 | 50.51 | 61.35 | 55.41 | |
| DeepSeek-V3Time (s/table)=18.072025.09 | 49.71 | 52.26 | 50.95 | |
| GPT-4oTime (s/table)=9.302025.09 | 47.46 | 53.42 | 50.27 | |
| Claude-3.5-SonnetTime (s/table)=6.562025.09 | 47.12 | 55.38 | 50.92 | |
| Gemini-1.5-ProTime (s/table)=11.892025.09 | 39.6 | 44.02 | 41.69 | |
| TAPEX2025.09 | 32.66 | 21.5 | 25.93 |