Reinforcement Learning on ALE Atari 2600 (2013 Full Set)
19,564.9Asterix ScorePersistent A.L.
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Persistent A.L.Operator=Persistent Advantage Learning2015.12 | 19,564.9 | 13,145.34 | 19.76 | 13,230.74 | 3,277.59 | 5,699.81 | 1,451.65 | 3,304.33 | 1,673.52 | 1,465,250 | 874.99 | 34,583.07 | 1,328.25 | 71.59 | 94.3 | 431.89 | 4,679.93 | 4,539.55 | 5,734.93 | 130,002.71 | 32,038.93 | 70,908.17 | -2.51 | 29,100 | 1,343.1 | 28.13 | 32.3 | 3,248.96 | 10,611.81 | 446.92 | 24,175.79 | -0.25 | 772.09 | 11,478.46 | 8,689.81 | 34,650.91 | 1.72 | 3,917.55 | 10,431.33 | 14,495.56 | 0 | 5,858.84 | 339.15 | 14,254.78 | 12,813.27 | 37,856.16 | 70.53 | -12,173.35 | 3,274.7 | 61,521.87 | 0.72 | 0 | 8,749.26 | 197.33 | 13,542.07 | 243.75 | 542,052 | 10,254.01 | 17,141.56 | 8,155.6 | 31 | |
| Advantage LearningOperator=Advantage Learning Operator2015.12 | 12,852.08 | 10,054.58 | 19.66 | 8,670.5 | 3,460.79 | 4,990.91 | 1,557.43 | 3,661.51 | 1,924.42 | 553,591.67 | 633.63 | 28,789.29 | 747.26 | 57.41 | 93.94 | 425.32 | 5,111.4 | 4,225.18 | 5,431.36 | 123,410.71 | 30,643.59 | 27,153.48 | -0.15 | 27,088.89 | 1,252.7 | 21.32 | 31.72 | 2,305.82 | 11,912.68 | 417.65 | 24,788.86 | -1.24 | 848.46 | 10,809.16 | 9,548.92 | 32,182.99 | 0.42 | 4,065.8 | 11,025.26 | 22,038.27 | 0 | 4,801.27 | 5,276.16 | 14,368.03 | 10,585.12 | 52,351.23 | 69.31 | -13,264.51 | 4,785.16 | 61,353.59 | -4.15 | 0 | 8,969.12 | 245.22 | 13,909.74 | 198.69 | 543,504 | 9,541.14 | 24,240.03 | 9,129.61 | 21 | |
| A3CTraining frames=200 million2016.06 | 9,546.96 | 7,004.58 | 20.84 | 1,697.19 | 1,741.27 | 1,658.25 | 1,034.15 | 2,881.69 | 3,946.22 | 1,634,837.98 | 1,301.51 | 3,393.84 | 1,233.47 | 35 | 3.07 | 432.42 | — | 5,184.76 | 3,324.24 | 111,493.76 | 39,388.08 | 39,293.17 | 0.19 | — | 0 | 32 | 0 | 283.99 | 6,872.6 | 201.29 | 34,880.51 | -5.13 | 422.42 | 46.63 | 7,603.84 | 29,369.9 | 0.17 | 2,327.8 | 6,087.31 | 13,893.06 | -6.98 | 4,198.61 | 97.36 | 19,175.72 | 11,902.24 | 41,059.12 | 2.22 | -20,958.97 | 2,102.13 | 59,218.08 | -7.1 | -16.18 | 9,000.91 | 273.66 | 44,883.4 | 0 | 68,287.63 | 4,347.76 | 20,006.02 | 152.11 | 26 | |
| A3C+Training frames=200 million, Exploration bonus=N^-1/22016.06 | 7,922.7 | 6,723.89 | 20.75 | 2,015.55 | 1,531.64 | 1,945.66 | 861.14 | 2,584.4 | 2,406.57 | 1,801,392.35 | 1,182.89 | 7,969.06 | 1,863.6 | 75.97 | 15.75 | 473.93 | — | 5,442.94 | 5,088.17 | 112,885.03 | 38,976.66 | 30,930.33 | -7.84 | — | 694.83 | 31.11 | 30.48 | 325.42 | 6,611.28 | 238.68 | 15,210.62 | -6.45 | 1,001.19 | 4,883.53 | 8,605.27 | 28,615.43 | 273.7 | 2,401.04 | 7,021.3 | 23,818.47 | -259.09 | 4,305.57 | 99.32 | 19,257.55 | 10,712.54 | 50,645.74 | 7.68 | -22,177.5 | 2,270.15 | 55,233.43 | -7.21 | -23.06 | 4,103 | 112.14 | 23,106.24 | 0 | 97,372.8 | 3,355.09 | 13,398.73 | 7,451.25 | 25 | |
| A3C+Training frames=200 million, Exploration bonus=N^-1/22016.06 | 7,262.77 | 5,992.08 | 17.33 | 2,274.06 | 1,466.01 | 1,848.33 | 964.77 | 2,607.28 | 2,257.92 | 1,733,528.71 | 991.96 | 7,428.99 | 1,720.56 | 68.72 | 13.82 | 323.21 | — | 5,338.24 | 5,388.22 | 104,083.51 | 36,377.6 | 19,589.95 | -8.88 | — | 749.11 | 29.46 | 27.33 | 506.61 | 5,948.4 | 246.02 | 15,077.42 | -7.05 | 1,024.16 | 5,475.73 | 7,587.58 | 26,593.67 | 142.5 | 2,380.58 | 6,427.51 | 20,300.72 | -155.97 | 3,943.37 | 100 | 15,804.72 | 10,331.56 | 49,029.74 | 6.68 | -20,066.65 | 2,175.7 | 52,466.84 | -6.99 | -20.49 | 3,816.38 | 132.67 | 8,705.64 | 0 | 35,515.91 | 3,657.65 | 12,317.49 | 7,956.05 | 24 | |
| A3CTraining frames=200 million2016.06 | 7,212.45 | 6,129.51 | 18.21 | 1,680.84 | 1,653.59 | 1,968.4 | 1,065.24 | 2,660.55 | 2,680.72 | 1,752,259.74 | 1,071.89 | 3,142.95 | 1,203.09 | 32.91 | 4.48 | 322.04 | — | 4,488.43 | 4,377.91 | 108,896.28 | 42,147.48 | 26,803.86 | 0.53 | — | 0 | 30.42 | 0 | 290.02 | 5,724.01 | 204.65 | 32,612.96 | -5.22 | 424.11 | 47.19 | 7,263.37 | 26,878.72 | 0.06 | 2,163.43 | 6,202.67 | 12,169.75 | -8.83 | 3,706.93 | 94.87 | 15,007.55 | 10,559.82 | 36,933.62 | 2.13 | -23,669.98 | 2,156.96 | 55,221.64 | -7.79 | -12.44 | 7,417.08 | 250.03 | 34,362.8 | 0 | 53,488.73 | 4,402.1 | 19,039.24 | 121.35 | 26 | |
| BellmanOperator=Consistent Bellman Operator2015.12 | 6,074.98 | 9,316.1 | 19.8 | 5,458.17 | 2,067.19 | 3,154.67 | 969.88 | 4,573.67 | 1,827.97 | 636,657.62 | 511 | 28,082.91 | 667.61 | 74.62 | 88.66 | 378.69 | 5,238.14 | 5,719.11 | 8,195.88 | 114,105.56 | 16,746.68 | 23,212.19 | -6.23 | 26,675 | 776.14 | 11.65 | 31.14 | 1,485.42 | 8,479.98 | 448.74 | 18,490.97 | -2.13 | 867.84 | 9,157.98 | 8,500.48 | 25,977.53 | 0.64 | 3,081.29 | 8,585.03 | 14,278.95 | 0 | 4,736.79 | 957.83 | 10,840.83 | 7,315.2 | 38,042.07 | 61.97 | -13,049.42 | 4,638.85 | 55,558.27 | -5.79 | 0 | 5,788.96 | 200.17 | 12,831.57 | 373.79 | 611,840.72 | 2,410.47 | 21,440.45 | 6,416.06 | 12 | |
| DQNTraining frames=200 million2016.06 | 4,062.55 | 7,949.08 | 19.17 | 3,035.32 | 1,101.43 | 1,418.47 | 654.4 | 1,707.87 | 735.05 | 281,448.8 | 315.93 | 17,927.46 | 471.76 | 30.34 | 80.17 | 259.4 | — | 1,184.46 | 1,569.84 | 102,736.12 | 6,225.82 | 6,183.58 | -13.99 | — | 441.24 | -8.68 | 30.12 | 506.1 | 4,946.39 | 219.39 | 11,419.16 | -10.34 | 465.76 | 5,972.64 | 6,140.24 | 11,187.13 | 0 | 2,391.89 | 6,565.41 | 7,835.2 | -86.85 | 2,992.56 | -12.86 | 7,094.91 | 2,365.18 | 24,933.39 | 40.53 | -27,972.63 | 1,752.72 | 40,171.44 | -8.19 | -8 | 4,067.51 | 75.21 | 5,208.67 | 0 | 52,995.08 | 378.7 | 15,042.75 | 2,481.4 | 9 | |
| DQNTraining frames=200 million2016.06 | 3,256.07 | 4,653.24 | 15.1 | 1,180.7 | 1,428.94 | 1,802.08 | 781.76 | 1,246.83 | 525.09 | 77,670.03 | 419.5 | 16,757.88 | 416.03 | 29.07 | 66.13 | 85.82 | — | 4,698.76 | 1,927.5 | 86,126.17 | 4,593.79 | 4,831.12 | -11.57 | — | 348.3 | -27.83 | 30.59 | 707.41 | 3,946.13 | 43.04 | 12,140.76 | -9.78 | 511.76 | 4,170.09 | 5,775.23 | 15,125.08 | 0.02 | 2,480.39 | 3,631.9 | 3,015.64 | -84.4 | 2,817.36 | 69.53 | 5,259.18 | 8,934.68 | 31,613.83 | 50.8 | -26,402.39 | 805.66 | 47,557.16 | -8.77 | -12.98 | 2,808.92 | 70.84 | 4,139.2 | 54.86 | 55,326.08 | 1,231.23 | 14,236.94 | 2,333.52 | 8 |