Graph prediction on ProceL (test)
82.5Make PBJ SandwichGround-truth labels → Graph
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Ground-truth labels → GraphInput source=Ground truth key step sequences, Type=Oracle/Upper Bound2023.02 | 82.5 | 83.9 | 78.1 | 78.1 | 96.4 | 83.8 | |
| Unsupervised Task Graph Generation PipelineInput source=Summary steps generated by GPT, Labeling=Algorithm 1, Ranking/Filtering=Top-k filtering2023.02 | 76.2 | 80.4 | 90.6 | 51 | 62.5 | 72.1 | |
| ASR → GPT → Labels → GraphInput source=Summary steps generated by GPT, Labeling=Algorithm 12023.02 | 68.8 | 69.6 | 87.5 | 53.1 | 55.4 | 66.9 | |
| ProscriptModel Type=Fine-tuned language model, Training Data=Manually curated script data, Inputs=Task description and set of key steps2023.02 | 65 | 51.8 | 46.9 | 52.1 | 53.6 | 53.9 | |
| ASR → Labels → GraphInput source=ASR sentences, Labeling=Algorithm 12023.02 | 52.5 | 57.1 | 78.1 | 59.4 | 53.6 | 60.1 | |
| ASR → VPs → Labels → GraphInput source=Verb phrases extracted from ASR, Labeling=Algorithm 12023.02 | 52.5 | 53.6 | 56.2 | 59.4 | 53.6 | 55.1 |