Temporal Difference Learning on Markovian sampling
3Convergence Rate ComplexityTD(0) with exponential step-size
Evaluation results
| Method | Links | |
|---|---|---|
| TD(0) with exponential step-sizeStep-size=O(1/ln T (1/T)^{t/T}), Parameters needed=ω, Projection=No, Last or Average iterate convergence=Last2026.03 | 3 | |
| Regularized TD(0) with exponential step-sizeStep-size=O(1/(√T ln T) (1/T)^{t/T}) for regularized TD, Parameters needed=None, Projection=No, Last or Average iterate convergence=Last2026.03 | 3 | |
| Samsonov et al., 2024Step-size=O(1) for TD with data drop, Parameters needed=τ_mix, Projection=No, Last or Average iterate convergence=Average2026.03 | 2 | |
| Mitra, 2025Step-size=O(ω/τ_mix), Parameters needed=τ_mix, ω, Projection=No, Last or Average iterate convergence=Average2026.03 | 2 | |
| Bhandari et al., 2018Step-size=1/√T, Parameters needed=No, Projection=Yes, Last or Average iterate convergence=Average2026.03 | 1 | |
| Bhandari et al., 2018Step-size=O(1/(ω(t+1))), Parameters needed=ω, Projection=Yes, Last or Average iterate convergence=Average2026.03 | 1 | |
| Bhandari et al., 2018Step-size=O(1/ω), Parameters needed=ω, Projection=Yes, Last or Average iterate convergence=Last2026.03 | -2 |