List your company
ResearchFormerly WizWand

Reasoning on BigBench-Hard (Averaged Collection)

45.41Ours AccuracyTrans-LoRA
43.3343.8744.4144.95May 27, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Trans-LoRASource Model=Llama-2-7b, Target Model=Gemma-7b, Discriminator Model=Gemma-2b, Evaluation=zero-shot2024.0545.41——
Trans-LoRASource Model=Llama-2-7b, Target Model=Gemma-7b, Discriminator Model=Llama-2-7b, Evaluation=zero-shot2024.0544.12——
Trans-LoRASource Model=Gemma-2b, Target Model=Gemma-7b, Discriminator Model=Gemma-2b, Evaluation=zero-shot2024.0543.61——
Trans-LoRASource Model=Llama-2-7b, Target Model=Llama-2-13b, Discriminator Model=Llama-2-7b, Evaluation=zero-shot2024.0543.41——
Gemma-2b (Source Model)Training=LoRA, Evaluation=zero-shot2024.05—31.84—
Gemma-7b (Target Model)Training=none, Evaluation=zero-shot2024.05——37.75
Llama-2-13b (Target Model)Training=none, Evaluation=zero-shot2024.05——37.85
Llama-2-7b (Source Model)Training=LoRA, Evaluation=zero-shot2024.05—43.32—