Entity Classification on RELBENCH rel-f1 driver-top3 (test)
91.4AUROCGemma
Evaluation results
| Method | Links | |
|---|---|---|
| GemmaTarget DB in pretraining=Maybe, Parameter count=27B2025.10 | 91.4 | |
| GemmaTarget DB in pretraining=Maybe, Parameter count=12B2025.10 | 90.5 | |
| RTTarget DB in pretraining=No, Parameter count=22M2025.10 | 89.3 | |
| RTTarget DB in pretraining=Yes, Parameter count=22M2025.10 | 89.3 | |
| ICLEvaluation Protocol=In-context learning2025.06 | 88.47 | |
| ICL + MLPEvaluation Protocol=MLP on LLM representations2025.06 | 87.36 | |
| Entity MeanTarget DB in pretraining=No, Parameter count=02025.10 | 85 | |
| GemmaTarget DB in pretraining=Maybe, Parameter count=4B2025.10 | 84.6 | |
| FROG2026.05 | 82.73 | |
| GriffinTarget DB in pretraining=No, Parameter count=22M2025.10 | 82.5 | |
| Rel-LLMEvaluation Protocol=Fine-tuned2025.06 | 82.22 | |
| RelGNN2026.05 | 82.21 | |
| GriffinTarget DB in pretraining=Yes, Parameter count=22M2025.10 | 81.8 | |
| RDL2026.05 | 78.31 | |
| RDLModel Type=GNN + Tabular2025.06 | 75.54 | |
| LightGBMModel Type=Gradient Boosted Trees2025.06 | 73.92 | |
| Rel-ZeroEvaluation Protocol=Zero-shot2025.06 | 70.64 | |
| RelLLMTarget DB in pretraining=Yes, Parameter count=3B2025.10 | 70.6 | |
| LightGBM2026.05 | 69.52 |