LOReL Sawyer
Benchmarks
TaskDatasetSOTA resultTrendResults
LOReL Sawyer Dataset (test)
9.532Close Drawer Success Rate
6
LOReL Sawyer Average across rephrasals
39.71Success Rate
3
LOReL Sawyer unseen noun + verb rephrasals
42.02Success Rate
3
LOReL Sawyer unseen verb rephrasals
0.367Success Rate
3
LOReL Sawyer unseen noun rephrasals
36.01Success Rate
3
LOReL Sawyer seen rephrasals
43.65Success Rate
3
LOReL Sawyer 15 unseen instructions (test)
20.89Success Rate
3