Offline Reinforcement Learning on D4RL (Full Suite Evaluation)
65.9HalfCheetah-MediumEDAC
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EDACRuntime (s/epoch)=19.6, GPU Memory (GB)=1.92023.06 | 65.9 | 28.4 | 25.3 | 16.6 | 101.6 | 92.5 | 61.3 | 101 | 87.1 | 106.3 | 110.7 | 114.7 | 106.8 | 110.1 | 115.1 | 1,243.4 | 16.7 | 0 | 0 | 0 | 0 | 0 | 16.7 | 52.1 | 68.2 | 120.3 | 1,380.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CPI-RERuntime (s/epoch)=19.1, GPU Memory (GB)=1.42023.06 | 65.9 | 30.7 | 30.4 | 5.5 | 97.9 | 86.3 | 55.9 | 103.2 | 93.8 | 95.6 | 110.1 | 111.2 | 97.4 | 112.3 | 111.2 | 1,207.4 | 99.2 | 92.6 | 84.8 | 80.6 | 33.6 | 48 | 438.8 | 87 | 70.7 | 157.7 | 1,803.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ReBRACHyperparameter configuration=dataset-specific, Robust Critic=false2025.08 | 65.6 | 29.5 | 8.1 | 18.4 | 102 | 82.5 | 51 | 98.1 | 77.3 | 101.1 | 107 | 111.6 | 105.9 | 100.1 | 112.3 | — | 106.8 | 88.3 | 84 | 76.3 | 60.4 | 54.4 | — | 103.5 | 91.8 | — | — | 82.1 | 107.1 | 102.2 | 81.2 | 105.1 | 78.3 | 96.7 | 76.8 | 154.1 | 0 | 1.1 | 104.6 | 0.2 | 6.7 | 133.8 | 0 | 0.9 | 106.6 | 58.6 | 74.8 | — | |
| CPIRuntime (s/epoch)=8.5, GPU Memory (GB)=1.42023.06 | 64.4 | 29.7 | 29.5 | 5.9 | 98.5 | 85.8 | 54.6 | 101.7 | 91.8 | 94.7 | 106.4 | 110.9 | 96.5 | 112.2 | 110.6 | 1,193.2 | 98.8 | 88.6 | 82.4 | 80.4 | 20.6 | 45.2 | 416 | 80.1 | 71.8 | 151.9 | 1,761.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ASPCHyperparameter configuration=fixed, Robust Critic=false2025.08 | 58.7 | 20.8 | 9.4 | 15.6 | 92.7 | 92.4 | 50.6 | 101.3 | 97.6 | 99.9 | 111 | 111.1 | 105.1 | 112.3 | 110.8 | — | 128.1 | 92.5 | 85 | 70 | 55 | 52.5 | — | 81.1 | 87.2 | — | — | 79.3 | 107.2 | 102.1 | 82.1 | 117.8 | 195.8 | 147.2 | 74.5 | 141.2 | 0 | 0 | 105.6 | 2.2 | 12 | 128.6 | 0.1 | 0 | 111.2 | 55.7 | 77.9 | — | |
| A2PRHyperparameter configuration=fixed, Robust Critic=false2025.08 | 56.1 | 21.1 | 20.1 | 1.2 | 78.3 | 84.2 | 49 | 98.9 | 84.5 | 95.9 | 110.8 | 88.2 | 99.9 | 83.9 | 84.8 | — | 102.5 | 32.5 | 40 | 40 | 5 | 22.5 | — | -2.1 | 6.5 | — | — | 79.5 | 97.1 | 102.5 | 74.2 | 90.4 | 177.7 | 123.53 | 38.75 | 51.5 | -0.2 | -0.3 | -0.3 | 1.1 | 0.3 | 0.3 | -0.3 | -0.3 | -0.3 | 4.65 | 51.2 | — | |
| wPCHyperparameter configuration=fixed, Robust Critic=true2025.08 | 54.8 | 18.8 | 8.5 | 12.5 | 81.8 | 89.6 | 48.1 | 100.8 | 93.4 | 98.9 | 109.1 | 110.1 | 103.8 | 79.1 | 111.5 | — | 73.1 | 75 | 85 | 85 | 65 | 65 | — | 39.9 | 34.6 | — | — | 76.7 | 105.6 | 99.5 | 77.8 | 87.4 | 123.3 | 94.6 | 78.7 | 141.8 | -0.2 | 0 | 51.4 | 0 | 0.1 | 57.6 | 0.1 | 0.1 | 6.7 | 28.8 | 64.2 | — | |
| Diffusion-QL (D-QL)Runtime (s/epoch)=39.8, GPU Memory (GB)=1.52023.06 | 51.5 | 22 | 18.3 | 5.5 | 96.6 | 87.3 | 48.3 | 102 | 98 | 97.2 | 112.3 | 111.2 | 96.3 | 102.6 | 109.5 | 1,158.6 | 96 | 84 | 79.8 | 82 | 49 | 61.7 | 452.5 | 75.7 | 60.8 | 136.5 | 1,747.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IQLHyperparameter configuration=dataset-specific, Robust Critic=false2025.08 | 50 | 19.5 | 10.1 | 11.3 | 65.2 | 80.7 | 42.1 | 89.6 | 75.4 | 92.7 | 85.5 | 112.1 | 95.5 | 108.8 | 96.9 | — | -8.9 | 70.6 | 64.6 | 61.7 | 42.5 | 27.6 | — | 81.5 | 77.2 | — | — | 75 | 104.4 | 97.5 | 72.9 | 34.8 | 61.7 | 46.2 | 58.3 | 133.6 | 3.1 | 0.8 | 105.3 | 2.5 | 1.1 | 129.6 | 0.1 | 0.2 | 106.5 | 53.4 | 62.6 | — | |
| TD3+BCHyperparameter configuration=fixed, Robust Critic=true2025.08 | 49.6 | 10.6 | 8.6 | 5.9 | 62 | 62 | 45.8 | 47.4 | 76.6 | 97.9 | 103.3 | 103.3 | 100.4 | 108.2 | 108.2 | — | 34.5 | 87.5 | 7.5 | 12.5 | 2.5 | 2.5 | — | 53.8 | 71.7 | — | — | 74.5 | 90.3 | 88.3 | 70.7 | 63.3 | 108.9 | 68.9 | 35.4 | 126.6 | 0 | 0 | 81.6 | 0 | 0.1 | 132.8 | 0 | 0 | 90.6 | 46.4 | 57.7 | — | |
| POR2023.06 | 48.8 | 29 | 12 | 6.3 | 98.2 | 81.1 | 43.5 | 98.9 | 76.6 | 94.7 | 90 | 109.1 | 93.2 | 110.4 | 102.9 | 1,094.7 | 76.8 | 64.8 | 87.2 | 75.2 | 24.4 | 59.2 | 387.6 | 76.9 | 67.6 | 144.5 | 1,626.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TD3+BCRuntime (s/epoch)=7.4, GPU Memory (GB)=1.42023.06 | 48.3 | 11 | 8.5 | 1.6 | 59.3 | 83.7 | 44.6 | 60.9 | 81.8 | 90.7 | 98 | 110.1 | 96.7 | 107.8 | 110.2 | 1,013.2 | 78.6 | 71.4 | 3 | 10.6 | 0 | 0.2 | 163.8 | -1.9 | 9.6 | 7.7 | 1,184.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| InAC2023.06 | 48.3 | 19.6 | 32.4 | 6.3 | 60.3 | 82.7 | 44.3 | 92.1 | 69.8 | 83.5 | 93.8 | 109 | 93.6 | 103.4 | 110.6 | 1,049.7 | 84.8 | 82.4 | — | — | — | — | — | 52.3 | -8 | 44.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Cal-QL (OffRL Whole)Training Data=Whole offline dataset (D_off)2024.10 | 47.6 | — | — | — | 56.4 | 84 | 45.8 | 98 | 87.4 | 50.8 | 107.3 | 109 | — | — | — | 70.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IQL2023.06 | 47.4 | 13.7 | 8.4 | 5.9 | 66.3 | 78.3 | 44.2 | 94.7 | 73.9 | 86.7 | 91.5 | 109.6 | 94.9 | 108.8 | 109.7 | 1,034 | 87.5 | 62.2 | 71.2 | 70 | 39.6 | 47.5 | 378 | 71.5 | 37.3 | 108.8 | 1,520.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CQL2023.06 | 46.9 | 31.3 | 5.3 | 5.4 | 61.9 | 79.5 | 45.3 | 86.3 | 76.8 | 95 | 96.9 | 109.1 | 97.3 | 106.5 | 109.3 | 1,052.8 | 74 | 84 | 61.2 | 53.7 | 15.8 | 14.9 | 303.6 | 35.2 | 27.2 | 62.4 | 1,418.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DT2023.06 | 42.6 | 2.2 | 5.4 | 2.2 | 67.6 | 74 | 36.6 | 82.7 | 66.6 | 86.8 | 107.6 | 108.1 | 87.7 | 94.2 | 108.3 | 972.6 | 59.2 | 53 | 0 | 0 | 0 | 0 | 112.2 | 73.9 | 67.3 | 141.2 | 1,226 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BC (Whole)Training Data=Whole offline dataset (D_off)2024.10 | 42.3 | — | — | — | 50.2 | 65.9 | 14 | 22.9 | 14.6 | 59.8 | 51.7 | 89.6 | — | — | — | 45.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Av-PBCTraining Data=Synthetic (N_syn=256)2024.10 | 36.9 | — | — | — | 32.5 | 39.5 | 35.9 | 40.9 | 55 | 22 | 38.7 | 42.1 | — | — | — | 38.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PBCTraining Data=Synthetic (N_syn=256)2024.10 | 30.9 | — | — | — | 25.1 | 33.2 | 18.4 | 35.6 | 41.5 | 20.5 | 33.4 | 34 | — | — | — | 30.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DBCTraining Data=Synthetic (N_syn=256)2024.10 | 28.2 | — | — | — | 37.8 | 29.3 | 2.5 | 12.1 | 6.1 | 29 | 31.1 | 11.7 | — | — | — | 20.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RandomTraining Data=Synthetic (N_syn=256), Selection=Randomly selecting real state-action pairs2024.10 | 1.8 | — | — | — | 19.2 | 4.9 | 0.9 | 19.1 | 1.9 | 2 | 11.6 | 6.7 | — | — | — | 7.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BCPolicy Category=Gaussian2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38 | |
| DriftQLPolicy Category=Drift2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 72 | |
| FQLAlgorithm=FQL2025.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84 | — | — | 52 | 62.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FQLPolicy Category=Flow2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 66 | |
| GFP actorAlgorithm=GFP, Policy Component=actor πθ2025.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.1 | — | — | 52.8 | 63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GFP VaBCAlgorithm=GFP, Policy Component=VaBC πω2025.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 70.2 | — | — | 49.6 | 56.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IDQLPolicy Category=Diffusion2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 71 | |
| IFQLPolicy Category=Flow2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 56 | |
| IQLAlgorithm=IQL2025.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 17 | — | — | 48 | 54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IQLPolicy Category=Gaussian2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 64 | |
| ReBRACAlgorithm=ReBRAC2025.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.8 | — | — | 59 | 64.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ReBRACPolicy Category=Gaussian2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 76 | |
| SRPOPolicy Category=Diffusion2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 71 |