List your company
ResearchFormerly WizWand

Headline Generation on News Headline

16.1ROUGE-1TiToK
11.52412.71213.915.088Oct 6, 2025
Updated 8mo ago

Evaluation results

MethodLinks
TiToKTransfer=Mistral 7B → Mistral 7B, Evaluation Mode=Zero-shot2025.1016.1—14.3——
TITOKTransfer setting=Mistral 7B → Mistral 7B, k=70%, Evaluation Protocol=Zero-shot2025.1016.1—14.3——
TransLoRATransfer=Mistral 7B → Mistral 7B, Evaluation Mode=Zero-shot2025.1015.6—13.7——
TransLoRATransfer setting=Mistral 7B → Mistral 7B, Evaluation Protocol=Zero-shot2025.1015.6—13.7——
TiToKTransfer=Mistral 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1013.9—12.3——
TITOKTransfer setting=Mistral 7B → Llama3 8B, k=70%, Evaluation Protocol=Zero-shot2025.1013.9—12.3——
TiToKTransfer=Llama2 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1013.8—12——
TITOKTransfer setting=Llama2 7B → Llama3 8B, k=70%, Evaluation Protocol=Zero-shot2025.1013.8—12——
KD (+MinED)Transfer=Mistral 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.7—11.2——
TiToKTransfer=Llama3 3B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.7—11.3——
KDTransfer setting=Mistral 7B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.7—11.2——
TITOKTransfer setting=Llama3 3B → Llama3 8B, k=30%, Evaluation Protocol=Zero-shot2025.1012.7—11.3——
TransLoRATransfer=Mistral 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.6—11——
TransLoRATransfer setting=Mistral 7B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.6—11——
VanillaTransfer=Mistral 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.5—11——
VanillaTransfer=Llama3 3B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.5—11——
KD (+MinED)Transfer=Llama3 3B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.5—11——
VanillaTransfer=Llama2 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.5—11——
KD (+MinED)Transfer=Llama2 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.5—11——
VanillaTransfer setting=Mistral 7B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.5—11——
VanillaTransfer setting=Llama3 3B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.5—11——
KDTransfer setting=Llama3 3B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.5—11——
VanillaTransfer setting=Llama2 7B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.5—11——
KDTransfer setting=Llama2 7B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.5—11——
TransLoRATransfer=Llama2 7B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.3—10.9——
TransLoRATransfer setting=Llama2 7B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.3—10.9——
TransLoRATransfer=Llama3 3B → Llama3 8B, Evaluation Mode=Zero-shot2025.1012.2—10.8——
TransLoRATransfer setting=Llama3 3B → Llama3 8B, Evaluation Protocol=Zero-shot2025.1012.2—10.8——
VanillaTransfer=Mistral 7B → Mistral 7B, Evaluation Mode=Zero-shot2025.1011.7—10.1——
KD (+MinED)Transfer=Mistral 7B → Mistral 7B, Evaluation Mode=Zero-shot2025.1011.7—10.4——
VanillaTransfer setting=Mistral 7B → Mistral 7B, Evaluation Protocol=Zero-shot2025.1011.7—10.1——
KDTransfer setting=Mistral 7B → Mistral 7B, Evaluation Protocol=Zero-shot2025.1011.7—10.4——