EDA on Things-30 EDA (subsampled)
19.7TurnsV-FT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| V-FTBackbone=Vicuna 7B, Training Data=Celebs2023.10 | 19.7 | 3 | 1.6 | 3 | |
| Vicuna 7BSize=7B, Version=v1.32023.10 | 19.5 | 9 | 5.7 | 7 | |
| V-RLGPBackbone=Vicuna 7B2023.10 | 19.3 | 15 | 3.6 | 12 | |
| V-FT 7B (All)Size=7B, Training Data=All turns2023.10 | 19.2 | 13 | 6.1 | 10 | |
| VicunaModel size=7B2023.10 | 19.1 | 11 | 5.7 | 10 | |
| Mistral 7BSize=7B2023.10 | 18.9 | 13 | 3.8 | 11 | |
| Claude-12023.10 | 18.8 | 16 | 4.2 | 13 | |
| Claude-12023.10 | 18.7 | 15 | 4.3 | 13 | |
| VicunaModel size=13B2023.10 | 18.7 | 20 | 5.2 | 17 | |
| V-FT 13B (All)Size=13B, Training Data=All turns2023.10 | 18.6 | 21 | 6.1 | 17 | |
| V-FTBackbone=Vicuna 7B, Training Data=Things2023.10 | 18.5 | 22 | 6.6 | 18 | |
| V-FTBackbone=Vicuna 7B, Training Data=Success trajectories2023.10 | 18.5 | 28 | 6.8 | 23 | |
| Human2023.10 | 18.5 | 24 | 5.2 | 20 | |
| GPT-3.52023.10 | 18.4 | 25 | 7.1 | 21 | |
| V-FTBackbone=Vicuna 7B, Training Data=All2023.10 | 18.4 | 20 | 6.8 | 17 | |
| Claude-22023.10 | 18.4 | 21 | 5 | 18 | |
| Vicuna 13BSize=13B, Version=v1.32023.10 | 18.4 | 18 | 5 | 15 | |
| GPT-3.52023.10 | 18.2 | 28 | 6.3 | 23 | |
| V-FTBackbone=Vicuna 13B, Training Data=Success trajectories2023.10 | 18 | 29 | 6.9 | 24 | |
| V-FT 7B (Suc.)Size=7B, Training Data=Successful turns only2023.10 | 18 | 23 | 5.1 | 20 | |
| V-FT 13B (Suc.)Size=13B, Training Data=Successful turns only2023.10 | 18 | 25 | 4.5 | 21 | |
| V-RLGP 13BSize=13B2023.10 | 17.9 | 27 | 4.5 | 23 | |
| V-RLGPBackbone=Vicuna 13B2023.10 | 17.8 | 31 | 4 | 26 | |
| GPT-42023.10 | 17.8 | 31 | 5.9 | 26 | |
| V-RLGP 7BSize=7B2023.10 | 17.8 | 26 | 4.7 | 22 | |
| Claude-22023.10 | 17.6 | 29 | 4.5 | 25 | |
| GPT-42023.10 | 16.9 | 49 | 6 | 40 |