Text-based Science Simulation on ScienceWorld (test)
63.35ScoreGPT-J (No variations)
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-J (No variations)Train=3 589, Training data variations=No variations2023.10 | 63.35 | 6.94 | 3.5 | 1 | 372 | 90.39 | 0.07 | 4.28 | 3.08 | 2 | 19 | |
| GPT-J (All train)Train=7 359, Markov assumption=false2023.10 | 62.57 | 4.32 | 3.5 | 1 | 383 | 90.51 | 0.06 | 4.68 | 2.51 | 2.07 | 17 | |
| GPT-J (Up to 18 games)Train=4802023.10 | 39.78 | 2.35 | 2.2 | 479 | 682 | 83.59 | 0.39 | 11.46 | 2.81 | 1.46 | 30 | |
| GPT-J (All train Markov)Train=7 359, Markov assumption=true2023.10 | 24.74 | 1.05 | 1.4 | 117 | 74 | 61.13 | 3.51 | 31.14 | 2.43 | 1.77 | 2 | |
| DRRNTrain=N/A2023.10 | 17.95 | — | 1 | — | — | — | — | — | — | — | — |