Text-based Reinforcement Learning on Jericho benchmark (test)
35.8DeepHome ScoreDRIFT
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DRIFTSteps=300K, Seeds=12026.05 | 35.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 291.5 | 20 | 13.8 | 8 | 26.8 | 8 | 25 | 2.5 | 5 | 23.2 | |
| DRRNSteps=300K, Seeds=12026.05 | 6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 50 | 20 | 6 | 10 | 25 | 5 | 35 | 3 | 5 | 15.3 | |
| DQNSteps=300K, Seeds=12026.05 | 6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 70 | 20 | 6 | 10 | 25 | 8 | 25 | 0 | 5 | 12.1 | |
| IQLSteps=300K, Seeds=12026.05 | 6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 50 | 20 | 6 | 5 | 25 | 8 | 25 | 0 | 5 | 10.6 | |
| AWACSteps=300K, Seeds=12026.05 | 3.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 66.5 | 17 | 11.4 | 5.5 | 11.75 | 7.4 | 1.25 | 0 | 2 | 8.3 | |
| CALMSteps=300K, Seeds=12026.05 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 289.7 | 19.1 | 10.1 | 6.9 | 0 | 0 | 30.4 | 0.5 | 3.7 | 12.6 | |
| KG-A2CSteps=300K, Seeds=12026.05 | 1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 207.9 | 12.1 | 17.8 | 3 | 50.7 | 7.6 | 34 | 0 | 9.2 | 19.2 | |
| HEX-RLTraining Reward=Game Only2021.12 | — | 29.8 | 40 | 16 | 19 | 276.7 | 330 | 10 | 10 | 34.6 | 55 | 5 | 5 | 14 | 18 | 1 | 1 | 8 | 8 | 27.2 | 33.9 | 0.03 | 0.01 | — | — | — | — | — | — | — | — | — | — | |
| HEX-RLTraining Reward=Game and IM (Intrinsic Motivation)2021.12 | — | 30.2 | 40 | 13.8 | 21 | 276.9 | 330 | 10 | 10 | 44.7 | 60 | 5.1 | 9 | 17.6 | 18 | 1 | 1 | 7.6 | 8 | 28.2 | 35.8 | 0.03 | 0.02 | — | — | — | — | — | — | — | — | — | — | |
| KG-A2CKnowledge Graph Construction=Rules based approach (OpenIE)2021.12 | — | 34 | 35 | 14.3 | 19 | 207.9 | 214 | 10 | 10 | 50.7 | 56 | 5 | 5 | 17.8 | 19 | 1 | 1 | 7.6 | 8 | 27.3 | 30.8 | 0.06 | 0.01 | — | — | — | — | — | — | — | — | — | — | |
| LSTM-A2CState Representation=Natural language observations, Policy Network=LSTM-based2021.12 | — | 27 | 31.2 | 8.2 | 10 | 141 | 188 | 10 | 10 | 50.4 | 55 | 5 | 5 | 14.4 | 18 | 1 | 1 | 8 | 8 | 22.6 | 25.9 | 0.02 | 0.01 | — | — | — | — | — | — | — | — | — | — | |
| Q*BERTKnowledge Graph Construction=Question Answering2021.12 | — | 35 | 35 | 18 | 18 | 274 | 310 | 10 | 10 | 50 | 56 | 5 | 5 | 18 | 19 | 1 | 1 | 8 | 8 | 30.8 | 34.9 | 0.03 | 0 | — | — | — | — | — | — | — | — | — | — | |
| SHA-KGArchitecture=Hierarchical Graph Attention network, Edge Relationship Information=None2021.12 | — | 33.6 | 34.5 | 10 | 15.8 | 246.1 | 308 | 9.8 | 10 | 48.2 | 51.3 | 5 | 25 | 17.6 | 17.8 | 1 | 1 | 7.9 | 6.9 | 27.2 | 33.1 | — | — | — | — | — | — | — | — | — | — | — | — |