List your company
ResearchFormerly WizWand

Command-to-action mapping on SCAN (length)

99.7AccuracyLeast-to-Most
-3.98822.93149.8576.769May 21, 2022Jul 23, 2022Sep 24, 2022Nov 27, 2022Jan 29, 2023Apr 2, 2023Jun 5, 2023
Updated 8mo ago

Evaluation results

MethodLinks
Least-to-Mostmodel=code-davinci-0022022.0599.7
vNQ¹constraint=hard constraint d(r) < inf2023.0695.7
PModelconstraint=hard constraint d(r) < inf2023.0691.72
Least-to-Mostmodel=text-davinci-002, sample_size=random subset of 100 commands2022.0576
Least-to-Mostmodel=code-davinci-0012022.0560.7
Standard promptingmodel=code-davinci-0022022.0516.7
Chain-of-Thoughtmodel=code-davinci-0022022.0516.2
Standard promptingmodel=text-davinci-002, sample_size=random subset of 100 commands2022.056
Standard promptingmodel=code-davinci-0012022.050.4
Chain-of-Thoughtmodel=text-davinci-002, sample_size=random subset of 100 commands2022.050
Chain-of-Thoughtmodel=code-davinci-0012022.050