Instruction Tracking on InsTrack 1.0 (test)
0.549J&FTrackGPT-13B
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TrackGPT-13BYear=2023, Execution=online, Reasoning Ability=true, Instruction Tuning=true2023.12 | 0.549 | 0.528 | 0.569 | 0.594 | 0.629 | |
| TrackGPT-13BYear=2023, Execution=online, Reasoning Ability=true, Instruction Tuning=false2023.12 | 0.505 | 0.488 | 0.521 | 0.541 | 0.58 | |
| TrackGPT-7BYear=2023, Execution=online, Reasoning Ability=true, Instruction Tuning=true2023.12 | 0.492 | 0.489 | 0.494 | 0.559 | 0.549 | |
| LISA-13B*Year=2023, Execution=online, Reasoning Ability=true2023.12 | 0.452 | 0.448 | 0.457 | 0.474 | 0.447 | |
| TrackGPT-7BYear=2023, Execution=online, Reasoning Ability=true, Instruction Tuning=false2023.12 | 0.433 | 0.421 | 0.445 | 0.474 | 0.462 | |
| LISA-7B*Year=2023, Execution=online, Reasoning Ability=true2023.12 | 0.397 | 0.385 | 0.409 | 0.37 | 0.375 | |
| OnlineReferYear=2023, Execution=online, Reasoning Ability=false2023.12 | 0.368 | 0.338 | 0.399 | 0.386 | 0.4 | |
| SgMgYear=2023, Execution=offline, Reasoning Ability=false2023.12 | 0.334 | 0.319 | 0.35 | 0.346 | 0.351 | |
| R2-VOSYear=2023, Execution=online, Reasoning Ability=false2023.12 | 0.247 | 0.211 | 0.282 | 0.216 | 0.274 | |
| ReferFormerYear=2022, Execution=offline, Reasoning Ability=false2023.12 | 0.24 | 0.229 | 0.252 | 0.247 | 0.231 |