CORL: Research-oriented Deep Offline Reinforcement Learning Library

About

CORL is an open-source library that provides thoroughly benchmarked single-file implementations of both deep offline and offline-to-online reinforcement learning algorithms. It emphasizes a simple developing experience with a straightforward codebase and a modern analysis tracking tool. In CORL, we isolate methods implementation into separate single files, making performance-relevant details easier to recognize. Additionally, an experiment tracking feature is available to help log metrics, hyperparameters, dependencies, and more to the cloud. Finally, we have ensured the reliability of the implementations by benchmarking commonly employed D4RL datasets providing a transparent source of results that can be reused for robust evaluation tools such as performance profiles, probability of improvement, or expected online performance.

Denis Tarasov, Alexander Nikulin, Dmitry Akimov, Vladislav Kurenkov, Sergey Kolesnikov · 2022

Related benchmarks

Task	Dataset	Result
Offline Reinforcement Learning	Maze2D medium v1	Normalized Return0.79	30
Offline Reinforcement Learning	Maze2D large v1	Normalized Return2.26	30
Offline Reinforcement Learning	D4RL MuJoCo v2 (test)	HalfCheetah-Medium Score68.2	19
Planning and Control	maze2d-umaze v1 (100 episodes, 300 steps/ep)	Score0.36	16
Offline Reinforcement Learning	AntMaze Medium-Diverse v2	Average Score0.008	14
Offline Reinforcement Learning	AntMaze medium-play v2	Average Score0	14
Offline Reinforcement Learning	AntMaze large-play v2	D4RL Score0	11
Offline Reinforcement Learning	AntMaze large-diverse v2	D4RL Score0	11
Offline Reinforcement Learning	D4RL MuJoCo v2 (various)	HalfCheetah Medium v2 Score68.2	9