Off-Policy Evaluation on KuaiRec slate experiments
0.617Downstream OPE RMSEFF-DR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FF-DRK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | 0.617 | — | |
| FF-DRK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | 0.949 | — | |
| FF-DRK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | 1.08 | — | |
| FF-OISK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | 1.27 | — | |
| DRK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | 1.66 | — | |
| FF-OISK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | 1.75 | — | |
| DRK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | 2.02 | — | |
| FF-OISK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | 2.19 | — | |
| OISK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | 3.36 | — | |
| DRK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | 7.81 | — | |
| OISK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | 11.46 | — | |
| OISK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | 12.3 | — | |
| Forward-DPK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | — | 0.36 | |
| Forward-DPK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | — | 2.93 | |
| Forward-DPK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | — | 8.94 | |
| Gumbel-MCK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | — | 26.9 | |
| Gumbel-MCK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | — | 29.6 | |
| Gumbel-MCK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | — | 105 | |
| K! enumK (slate size)=4, Pool size (M)=15, (M choose K)=1,3652026.05 | — | 6.2 | |
| K! enumK (slate size)=6, Pool size (M)=15, (M choose K)=5,0052026.05 | — | 1,010 | |
| K! enumK (slate size)=8, Pool size (M)=15, (M choose K)=6,4352026.05 | — | 97,108 |