List your company
ResearchFormerly WizWand

Language Modeling on WT-103

8.28PerplexityCE + TMKL
7.99129.940611.8913.8394May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CE + TMKLBackbone (LoRA)=Llama-3.1-8B2026.058.282
no adaptationBackbone (LoRA)=Llama-3.1-8B2026.058.45—
CE + TMKLBackbone (LoRA)=Mistral-7B-v0.32026.058.811
no adaptationBackbone (LoRA)=Mistral-7B-v0.32026.058.9—
CEBackbone (LoRA)=Llama-3.1-8B2026.0510.0519
PiSSA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=PiSSA, Method=CE + TMKL2026.0510.192.8
LoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=LoRA, Method=CE + TMKL2026.0510.252.2
DoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=DoRA, Method=CE + TMKL2026.0510.291.8
CEBackbone (LoRA)=Mistral-7B-v0.32026.0510.4117
no adaptationBackbone=Phi-3.5-mini-instruct 3.8B, Adapter=None, Method=no adaptation2026.0510.48—
RandLoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=RandLoRA, Method=CE + TMKL2026.0510.50.2
SineLoRA (CE + TMKL)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=SineLoRA, Method=CE + TMKL2026.0510.692
SineLoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=SineLoRA, Method=CE2026.0511.287.7
LoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=LoRA, Method=CE2026.0512.1716
DoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=DoRA, Method=CE2026.0512.1716
CE + TMKLBackbone (LoRA)=Llama-3.2-1B2026.0512.311.5
PiSSA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=PiSSA, Method=CE2026.0512.4219
no adaptationBackbone (LoRA)=Llama-3.2-1B2026.0512.5—
RandLoRA (CE)Backbone=Phi-3.5-mini-instruct 3.8B, Adapter=RandLoRA, Method=CE2026.0513.2226
CEBackbone (LoRA)=Llama-3.2-1B2026.0515.524