Policy Bias Reduction on 3-state × 2-action tabular MDP
4.5Fast-only Biasnaive
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| naiveK=2, E[Delta]=25, rho=12026.06 | 4.5 | 4.5 | 1 | 1 | 1 | |
| waitK=2, E[Delta]=25, rho=12026.06 | 4.5 | 0.17 | 27.1 | 20.3 | 26 | |
| Retrace-AK=2, E[Delta]=25, rho=12026.06 | 4.5 | 3.4 | 1.5 | 1.3 | 1.2 | |
| Retroactive Advantage Correction (RAC)K=2, E[Delta]=25, rho=12026.06 | 4.5 | 0.09 | 47.9 | 20.9 | 1 | |
| Retroactive Advantage Correction (RAC)K=1, sigma_f=0.5, Delta_k={1, ..., 5}2026.06 | 8.2 | 0.6 | 13.7 | 12.1 | — | |
| Retroactive Advantage Correction (RAC)K=2, sigma_f=0.5, Delta_k={1, ..., 5}2026.06 | 14.3 | 0.3 | 47.9 | 44.2 | — | |
| Retroactive Advantage Correction (RAC)K=3, sigma_f=0.5, Delta_k={1, ..., 5}2026.06 | 18.1 | 0.8 | 22.7 | 20.8 | — | |
| Retroactive Advantage Correction (RAC)K=4, sigma_f=0.5, Delta_k={1, ..., 5}2026.06 | 21.6 | 0.7 | 33 | 30.5 | — | |
| Retroactive Advantage Correction (RAC)K=5, sigma_f=0.5, Delta_k={1, ..., 5}2026.06 | 24.4 | 0.6 | 39.8 | 36.9 | — | |
| Retroactive Advantage Correction (RAC)K=5 (sat), kernel=saturated row-stochastic (eta=0), sigma_f=0.5, Delta_k={1, ..., 5}2026.06 | 25.1 | 0.52 | 48.3 | 45 | — |