List your company
ResearchFormerly WizWand

Science Question Answering on ARC (Accuracy)

25.8AccuracyTTT+CT-KV
0.0086.70413.420.096Jul 6, 2025
Updated 4mo ago

Evaluation results

MethodLinks
TTT+CT-KVTraining time per task (s)=632025.0725.8
TTTTraining time per task (s)=562025.0723.8
CT-KVTraining time per task (s)=262025.0723.8
CT-PromptTraining time per task (s)=522025.0722.5
Prefix TuningTraining time per task (s)=24, m=# demo2025.0720.5
Prompt TuningTraining time per task (s)=49, m=# demo2025.0714.5
LoRATraining time per task (s)=142025.0713.5
ICLTraining time per task (s)=0, Evaluation protocol=In-Context Learning2025.0713.3
DoRATraining time per task (s)=152025.0713
Rank-Stabilized LoRATraining time per task (s)=152025.0712.8
Prompt TuningTraining time per task (s)=13, m=322025.0712
Prefix TuningTraining time per task (s)=14, m=322025.079.3
Zero-ShotTraining time per task (s)=0, Evaluation protocol=Zero-Shot2025.071