Code runtime regression on CodeNet (test)
240.8Overall MAEL-MDS
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| L-MDS2026.06 | 240.8 | 109.8 | 159 | 52 | 101.9 | 94.5 | 159.6 | 248.2 | 375.4 | |
| L-MDSNumber of random seeds=52026.06 | 243.4 | 243.1 | 299 | 163.7 | 257.7 | 245 | 321.8 | 440.2 | 623.4 | |
| SqrtReWeight2026.06 | 246.6 | 125.2 | 159.7 | 57.4 | 112.9 | 113.5 | 208.8 | 284.4 | 371.3 | |
| L-MDS + F-MDS2026.06 | 248.1 | 133.5 | 178.4 | 85.5 | 163.8 | 106.8 | 202 | 251.4 | 377.3 | |
| SqrtReWeightNumber of random seeds=52026.06 | 248.2 | 248.3 | 299.4 | 179.2 | 247.2 | 242.1 | 328.2 | 444.6 | 609 | |
| L-MDS + F-MDSNumber of random seeds=52026.06 | 249.4 | 249.2 | 299.5 | 205.4 | 309 | 231.2 | 306 | 413.5 | 622.6 | |
| F-MDS2026.06 | 249.6 | 131.4 | 165.9 | 69.7 | 116.7 | 111 | 206.1 | 282.7 | 373.8 | |
| F-MDSNumber of random seeds=52026.06 | 250.5 | 250.4 | 287.2 | 196 | 279.6 | 235.6 | 306 | 429 | 592.2 | |
| CBLoss2026.06 | 250.7 | 128.5 | 160.7 | 64 | 103.3 | 103 | 193.1 | 294.7 | 401.9 | |
| CBLossNumber of random seeds=52026.06 | 251.9 | 251.8 | 301.3 | 161.3 | 229.2 | 253.9 | 333.9 | 472.9 | 624 | |
| ReWeight2026.06 | 251.9 | 132.5 | 172 | 71.5 | 120.8 | 111.9 | 209.3 | 279.4 | 378.9 | |
| ReWeightNumber of random seeds=52026.06 | 253.7 | 253.5 | 306.3 | 179.4 | 253.6 | 249.4 | 374.3 | 463.4 | 616.5 | |
| LDS2026.06 | 260.9 | 126.4 | 175.6 | 52.9 | 103.1 | 115.2 | 190.7 | 297.9 | 427.7 | |
| LDSNumber of random seeds=52026.06 | 263.1 | 263.3 | 322.4 | 178.6 | 275.6 | 261.4 | 365.5 | 484.6 | 686.3 | |
| ERM2026.06 | 266.4 | 142.8 | 203.7 | 51.9 | 92.5 | 127.1 | 249.3 | 377.3 | 455.6 | |
| ERMNumber of random seeds=52026.06 | 268.7 | 269 | 350.3 | 165.4 | 228.7 | 268.8 | 398.4 | 529.8 | 711.3 | |
| DANN2026.06 | 273.7 | 153.5 | 189.1 | 54.5 | 104.6 | 138.1 | 222.2 | 418.3 | 531.5 | |
| DANNNumber of random seeds=52026.06 | 276 | 276.4 | 348.8 | 148.3 | 228.9 | 292.9 | 427 | 551.4 | 716.9 |