Multitask Language Modeling on FV Benchmark
90.02AccuracySITE (M = 50)
Evaluation Results
| Method | Links | |
|---|---|---|
| SITE (M = 50)# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B, M=502025.07 | 90.02 | |
| SITE (M = 1)# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B, M=12025.07 | 89.67 | |
| LoRA# Trainable Parameters=3407.87K, Backbone=Llama-3.1-8B2025.07 | 86.76 | |
| (IA)3# Trainable Parameters=524.29K, Backbone=Llama-3.1-8B2025.07 | 81.97 | |
| I2CL# Trainable Parameters=0.13K, Backbone=Llama-3.1-8B2025.07 | 79.89 | |
| 10-shot ICLBackbone=Llama-3.1-8B2025.07 | 76.76 | |
| Prompt Tuning# Trainable Parameters=81.92K, Backbone=Llama-3.1-8B2025.07 | 75.06 | |
| MTV# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B2025.07 | 73.24 | |
| TVBackbone=Llama-3.1-8B2025.07 | 59.07 | |
| IVBackbone=Llama-3.1-8B2025.07 | 42.52 | |
| FVBackbone=Llama-3.1-8B2025.07 | 33.16 | |
| LIVE# Trainable Parameters=131.10K, Backbone=Llama-3.1-8B2025.07 | 23.34 | |
| 0-shot ICLBackbone=Llama-3.1-8B2025.07 | 9.85 |