Numerical Reasoning on Spotify
63.6Hit Rate @1NumCoKE
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| NumCoKEModel Category=Ours2024.11 | 63.6 | 69.4 | 75.8 | 30 | 68 | |
| RAKGEModel Category=Self-supervised2024.11 | 50.2 | 60.8 | 67.4 | 37 | 57.3 | |
| NRNModel Category=Deterministic2024.11 | 34.2 | 44.2 | 54.6 | 102 | 41.4 | |
| R-GCNModel Category=GNNs2024.11 | 28.8 | 38.2 | 50.4 | 89 | 36.4 | |
| KGDMModel Category=Euclidean2024.11 | 27.4 | 37.4 | 48.9 | 99 | 34.5 | |
| DaBRModel Category=Euclidean2024.11 | 26.6 | 36.7 | 47.7 | 104 | 33.4 | |
| LiteralEModel Category=Attributed2024.11 | 26.4 | 37.1 | 49.8 | 76 | 34.5 | |
| SliCEModel Category=Self-supervised2024.11 | 26.1 | 35.4 | 46.9 | 117 | 33.2 | |
| BiGIModel Category=Self-supervised2024.11 | 26 | 35.4 | 46.8 | 118 | 33.1 | |
| TransEModel Category=Euclidean2024.11 | 25.9 | 35.5 | 46.2 | 115 | 33.2 | |
| NEKGModel Category=Deterministic2024.11 | 24.6 | 32.3 | 42.3 | 121 | 33.6 | |
| LorentzKGModel Category=Hyperbolic2024.11 | 23.9 | 31.2 | 41.2 | 126 | 32 | |
| ConvEModel Category=Euclidean2024.11 | 23.1 | 30.7 | 41.4 | 110 | 29.5 | |
| TuckERModel Category=Euclidean2024.11 | 21.1 | 29.6 | 41.1 | 98 | 27.8 | |
| RuIEModel Category=Euclidean2024.11 | 12.4 | 18.8 | 28.4 | 173 | 18.2 | |
| GIEModel Category=Hyperbolic2024.11 | 12.1 | 18.3 | 28.1 | 185 | 17.4 | |
| MT-KGNNModel Category=Attributed2024.11 | 10.8 | 18.2 | 30.2 | 142 | 17.6 | |
| WGCNModel Category=GNNs2024.11 | 9.5 | 16.2 | 26.1 | 331 | 15.3 | |
| AttHModel Category=Hyperbolic2024.11 | 5.2 | 8.2 | 14.6 | 348 | 8.7 | |
| MuRPModel Category=Hyperbolic2024.11 | 2.4 | 17.9 | 32.4 | 135 | 11.9 | |
| KBLRNModel Category=Attributed2024.11 | 1.7 | 3.9 | 8.6 | 347 | 4.4 | |
| HAKEModel Category=Euclidean2024.11 | 0.8 | 8.5 | 12 | 196 | 7.5 | |
| ConEModel Category=Hyperbolic2024.11 | 0.1 | 0.6 | 6.4 | 227 | 2.8 | |
| SimGCLModel Category=Self-supervised2024.11 | 0 | 25.5 | 46.7 | 59 | 16.7 |