Operator Induction on VL-ICL Bench OP_IND (test)
47.7Mean AccuracyMAPD
Evaluation results
| Method | Links | |
|---|---|---|
| MAPDTTA (Test-Time Adaptation) method=FT (K ≤ 30), MT (Meta-Tasks used during training)=Yes2025.06 | 47.7 | |
| Multi-TaskPDTTA (Test-Time Adaptation) method=FT (K ≤ 30), MT (Meta-Tasks used during training)=Yes2025.06 | 45.1 | |
| Model-AvgPDTTA (Test-Time Adaptation) method=FT (K ≤ 30), MT (Meta-Tasks used during training)=No2025.06 | 40 | |
| NoMeta-taskPDTTA (Test-Time Adaptation) method=FT (K ≤ 30), MT (Meta-Tasks used during training)=No2025.06 | 38.8 | |
| In-ContextPDTTA (Test-Time Adaptation) method=FT (K ≤ 30), MT (Meta-Tasks used during training)=Yes2025.06 | 30.9 | |
| LoRA ([0-15] LLM layers + ATT)TTA (Test-Time Adaptation) method=FT (K ≤ 30)2025.06 | 30.5 | |
| LoRA ([0-15] LLM layers)TTA (Test-Time Adaptation) method=FT (K ≤ 30)2025.06 | 25.5 | |
| In-ContextPDTTA (Test-Time Adaptation) method=ICL, MT (Meta-Tasks used during training)=Yes2025.06 | 20.6 | |
| LoRA (All LLM layers)TTA (Test-Time Adaptation) method=FT (K ≤ 30)2025.06 | 13.3 | |
| NoMeta-taskPDTTA (Test-Time Adaptation) method=ICL, MT (Meta-Tasks used during training)=No2025.06 | 12.1 | |
| Multi-TaskPDTTA (Test-Time Adaptation) method=ICL, MT (Meta-Tasks used during training)=Yes2025.06 | 10 | |
| MAPDTTA (Test-Time Adaptation) method=ICL, MT (Meta-Tasks used during training)=Yes2025.06 | 9.6 | |
| Model-AvgPDTTA (Test-Time Adaptation) method=ICL, MT (Meta-Tasks used during training)=No2025.06 | 9.2 |