Offline Reinforcement Learning on D4RL AntMaze
94.4AntMaze Medium Play ReturnMPCwDWM
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MPCwDWMPre-trained policy=ReBRAC, Number of seeds=102026.03 | 94.4 | — | 99 | 90 | 94 | 67 | 66 | — | 85.07 | — | |
| ReBRACPolicy Category=Gaussian, Seeds=52026.03 | 90 | — | 98 | 84 | 84 | 52 | 64 | — | 78.7 | — | |
| ReBRAC2026.03 | 89.5 | — | 97.75 | 83.5 | 83.5 | 52.25 | 64 | — | 78.42 | 470.5 | |
| OFQLPolicy Type=One-Step Flow/Diffusion Policy2026.06 | 88.1 | 84.6 | — | — | 90.2 | 84 | 76.1 | — | — | — | |
| BFQPolicy Type=One-Step Flow/Diffusion Policy2026.06 | 87 | 83.9 | — | — | 84.1 | 88.5 | 76.1 | — | — | — | |
| ReBRACNumber of seeds=102026.03 | 86.7 | — | 99 | 89 | 83 | 59 | 51 | — | 77.62 | — | |
| DQLPolicy Type=Diffusion Policy2026.06 | 86 | 81.3 | — | — | 94.7 | 83.3 | 61.3 | — | — | — | |
| EPQ2024.05 | 85 | — | 99.4 | 78.3 | 86.7 | 40 | 36.7 | 426.1 | — | — | |
| IDQL2026.03 | 84.5 | — | 94 | 80.2 | 84.8 | 63.5 | 67.9 | — | 79.15 | 474.9 | |
| IDQLPolicy Category=Diffusion, Seeds=52026.03 | 84 | — | 94 | 80 | 85 | 64 | 68 | — | 79.2 | — | |
| SRPOPolicy Category=Diffusion, Seeds=52026.03 | 81 | — | 97 | 82 | 75 | 54 | 54 | — | 73.8 | — | |
| SRPOPolicy Type=One-Step Flow/Diffusion Policy2026.06 | 80.7 | 73.6 | — | — | 75 | 53.6 | 53.6 | — | — | — | |
| DMG2024.11 | 80.2 | — | 92.4 | 75.4 | 77.2 | 55.4 | 58.8 | 439.4 | — | — | |
| SORLPolicy Type=One-Step Flow/Diffusion Policy2026.06 | 80.1 | 65 | — | — | 70 | 57.3 | 52.5 | — | — | — | |
| SPQR-CQL-Min2024.01 | 80 | — | 93.3 | 80 | — | — | — | — | 84.4 | — | |
| GEM2026.03 | 78.33 | — | 83 | 79.33 | 82.67 | 37.33 | 37 | — | 66.28 | 397.66 | |
| FQLPolicy Category=Flow, Seeds=52026.03 | 78 | — | 96 | 89 | 71 | 84 | 83 | — | 83.5 | — | |
| FQL2026.03 | 78 | — | 96 | 89 | 71 | 84 | 83 | — | 83.5 | 501 | |
| FQLPolicy Type=One-Step Flow/Diffusion Policy2026.06 | 78 | 79 | — | — | 71 | 84 | 83 | — | — | — | |
| GSFlow (Ours)Policy Category=Flow, Seeds=52026.03 | 77.2 | — | 99.6 | 93.2 | 75.5 | 86.5 | 84.8 | — | 86.1 | — | |
| FBRACPolicy Category=Flow, Seeds=52026.03 | 77 | — | 94 | 82 | 77 | 32 | 20 | — | 63.7 | — | |
| Diffusion-QL2026.03 | 76.6 | — | 93.4 | 66.2 | 78.6 | 46.4 | 56.6 | — | 69.63 | 417.8 | |
| IQL2022.02 | 76.4 | — | 89.6 | 65.6 | 72.8 | 42 | 46 | 392.4 | — | — | |
| IQL2024.11 | 76.4 | — | 89.6 | 65.6 | 72.8 | 42 | 46 | 392.4 | — | — | |
| IQLPolicy Type=Gaussian Policy2026.06 | 75.5 | 60 | — | — | 78 | 38.6 | 48 | — | — | — | |
| SPOTseeds=102022.02 | 74.7 | — | 93.5 | 65.6 | 79.1 | 35.3 | 36.3 | 359.6 | — | — | |
| EDPPolicy Type=Diffusion Policy2026.06 | 73.3 | 50.2 | — | — | 52.7 | 33.3 | 41.3 | — | — | — | |
| IQL2024.05 | 71.2 | — | 87.5 | 62.2 | 70 | 39.6 | 47.5 | 378 | — | — | |
| CQLsource=reproduced2024.05 | 67.5 | — | 80.4 | 56.3 | 62.5 | 35 | 13.3 | 315 | — | — | |
| IDQLPolicy Type=Diffusion Policy2026.06 | 67.3 | 59.8 | — | — | 83.3 | 48.7 | 40 | — | — | — | |
| IQLPolicy Category=Gaussian, Seeds=52026.03 | 66 | — | 77 | 54 | 74 | 42 | 30 | — | 57.2 | — | |
| CQL2026.03 | 65.75 | — | 92.75 | 37.25 | 67.25 | 20.75 | 20.5 | — | 50.71 | 304.25 | |
| IQL2026.03 | 65.75 | — | 77 | 54.25 | 73.75 | 42 | 30.25 | — | 57.17 | 343 | |
| KFC++2021.11 | 63.1 | — | 99.8 | 91.1 | 90.5 | 25.6 | 34 | — | — | — | |
| MISA2024.05 | 63 | — | 92.3 | 89.1 | 62.8 | 17.5 | 23.4 | 348.1 | — | — | |
| S4RL-(N)Augmentation=Normal2021.11 | 61.9 | — | 91.3 | 87.8 | 78.1 | 24.4 | 27 | — | — | — | |
| S4RL-(Adv)Augmentation=Adversarial2021.11 | 61.6 | — | 94.1 | 88 | 82.3 | 25.1 | 26.2 | — | — | — | |
| CQL2021.11 | 61.2 | — | 74 | 84 | 53.7 | 15.8 | 14.9 | — | — | — | |
| CQL-Min2024.01 | 61.2 | — | 74 | 84 | — | — | — | — | 73.1 | — | |
| CQLsource=original paper2024.05 | 61.2 | — | 74 | 84 | 53.7 | 15.8 | 14.9 | 303.6 | — | — | |
| KFC2021.11 | 60 | — | 96.9 | 91.2 | 87.1 | 24.8 | 33.1 | — | — | — | |
| SSAR2026.03 | 59.8 | — | 94.7 | 65.1 | 59.6 | 35.5 | 26.7 | — | 56.9 | 341.4 | |
| CQL2022.02 | 59 | — | 82.6 | 10.2 | 46.6 | 16.4 | 3.2 | 218 | — | — | |
| CQL2024.11 | 59 | — | 82.6 | 10.2 | 46.6 | 16.4 | 3.2 | 218 | — | — | |
| IFQLPolicy Category=Flow, Seeds=52026.03 | 56 | — | 92 | 62 | 60 | 55 | 64 | — | 64.8 | — | |
| MCQ2024.05 | 52.5 | — | 98.3 | 80 | 37.5 | 2.5 | 7.5 | 278.3 | — | — | |
| FAWACPolicy Category=Flow, Seeds=52026.03 | 52 | — | 90 | 55 | 44 | 10 | 16 | — | 44.5 | — | |
| CACPolicy Category=Diffusion, Seeds=52026.03 | 49 | — | 66 | 66 | 0 | 0 | 0 | — | 30.2 | — | |
| CAC-2Policy Type=Diffusion Policy, Inference variant=two-step inference2026.06 | 49 | 12.3 | — | — | 0 | 0 | 0 | — | — | — | |
| PLAS2022.02 | 31.4 | — | 62 | 45.4 | 20.6 | 2.2 | 3 | 164.6 | — | — | |
| TD3+BC2024.05 | 10.6 | — | 78.6 | 71.4 | 3 | 0.2 | 0 | 163.8 | — | — | |
| TD3-BCPolicy Type=Gaussian Policy2026.06 | 10.6 | 4.1 | — | — | 5.7 | 0.2 | 0 | — | — | — | |
| 10% BCUsage=top 10% of demonstrations2024.05 | 5.4 | — | 62.8 | 50.2 | 9.8 | 0 | 6 | 134.2 | — | — | |
| Onestep2024.05 | 5 | — | 72.5 | 75 | 5 | 2.5 | 2.5 | 162.5 | — | — | |
| BC2022.02 | 0.4 | — | 49.2 | 41.8 | 0.2 | 0 | 0 | 91.6 | — | — | |
| TD3+BC2026.03 | 0.25 | — | 70.75 | 44.75 | 0.25 | 0 | 0 | — | 19.33 | 116 | |
| BCQ2022.02 | 0 | — | 78.9 | 55 | 0 | 6.7 | 2.2 | 142.8 | — | — | |
| BEAR2022.02 | 0 | — | 73 | 61 | 8 | 0 | 0 | 142 | — | — | |
| Decision Transformer2022.02 | 0 | — | 54.2 | 41.2 | 0 | 0 | 0 | 95.4 | — | — | |
| TD3+BC2022.02 | 0 | — | 73 | 47 | 0.2 | 0 | 0 | 120.2 | — | — | |
| BC2024.01 | 0 | — | 65 | 55 | — | — | — | — | 40 | — | |
| SAC-Min2024.01 | 0 | — | 0 | 0 | — | — | — | — | 0 | — | |
| BEAR2024.01 | 0 | — | 73 | 61 | — | — | — | — | 44.7 | — | |
| BC2024.05 | 0 | — | 54.6 | 45.6 | 0 | 0 | 0 | 100.2 | — | — | |
| BC2024.11 | 0 | — | 66.8 | 56.8 | 0 | 0 | 0 | 123.6 | — | — | |
| BCQ2024.11 | 0 | — | 78.9 | 55 | 0 | 6.7 | 2.2 | 142.8 | — | — | |
| BEAR2024.11 | 0 | — | 73 | 61 | 8 | 0 | 0 | 142 | — | — | |
| DT2024.11 | 0 | — | 54.2 | 41.2 | 0 | 0 | 0 | 95.4 | — | — | |
| AWAC2024.11 | 0 | — | 80 | 52 | 0.2 | 0 | 0 | 132.2 | — | — | |
| OneStep2024.11 | 0 | — | 54 | 57.8 | 0.6 | 0 | 0.2 | 112.6 | — | — | |
| TD3BC2024.11 | 0 | — | 73 | 47 | 0.2 | 0 | 0 | 120.2 | — | — | |
| BCPolicy Category=Gaussian, Seeds=52026.03 | 0 | — | 55 | 47 | 1 | 0 | 0 | — | 17.2 | — | |
| AWAC2026.03 | 0 | — | 56.75 | 54.75 | 0 | 0 | 0 | — | 18.58 | 111.5 | |
| SAC-N2026.03 | 0 | — | 0 | 0 | 0 | 0 | 0 | — | 0 | 0 | |
| EDAC2026.03 | 0 | — | 0 | 0 | 0 | 0 | 0 | — | 0 | 0 | |
| DT2026.03 | 0 | — | 57 | 51.75 | 0 | 0 | 0 | — | 18.12 | 108.75 | |
| BCPolicy Type=Gaussian Policy2026.06 | 0 | 0.2 | — | — | 0.6 | 0 | 0 | — | — | — | |
| CACPolicy Type=Diffusion Policy2026.06 | 0 | 0 | — | — | 0 | 0 | 0 | — | — | — | |
| ASTROAlgorithm=IQL2025.11 | — | 70.4 | — | — | — | — | — | — | — | — | |
| ASTROAlgorithm=FQL2025.11 | — | 88.6 | — | — | — | — | — | — | — | — | |
| DiffStitchAlgorithm=IQL2025.11 | — | 65.3 | — | — | — | — | — | — | — | — | |
| DiffStitchAlgorithm=FQL2025.11 | — | 85 | — | — | — | — | — | — | — | — | |
| OriginalAlgorithm=IQL2025.11 | — | 57.3 | — | — | — | — | — | — | — | — | |
| OriginalAlgorithm=FQL2025.11 | — | 78.2 | — | — | — | — | — | — | — | — | |
| SynthERAlgorithm=IQL2025.11 | — | 63.5 | — | — | — | — | — | — | — | — | |
| SynthERAlgorithm=FQL2025.11 | — | 79 | — | — | — | — | — | — | — | — |