List your company
ResearchFormerly WizWand

Compositional Generalization on Evaluation Dataset (Unseen Average)

42.86ScoreMistral-7B
19.782425.773731.76537.7563Jan 29, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Mistral-7BFine-tuning Strategy=LoRA, Model Backbone=Mistral-7B2026.0142.86-0.1948
Qwen2.5-7BFine-tuning Strategy=LoRA, Model Backbone=Qwen2.5-7B2026.0142.46-0.1578
Claude 3.5 Sonnet + FS*Evaluation Protocol=Few-shot, Model Backbone=Claude 3.5 Sonnet2026.0142.03-0.0092
LLaMA-3-8BFine-tuning Strategy=LoRA, Model Backbone=LLaMA-3-8B2026.0141.76-0.1573
LLaMA-3-8B + COGLMFine-tuning Strategy=LoRA, Model Backbone=LLaMA-3-8B, Method Components=COGLM2026.0141.13-0.2118
DeepSeek V3Evaluation Protocol=Zero-shot, Model Backbone=DeepSeek V32026.0141.08-0.0088
GPT-4o + FS*Evaluation Protocol=Few-shot, Model Backbone=GPT-4o2026.0140.410.0099
Gemma-7BFine-tuning Strategy=LoRA, Model Backbone=Gemma-7B2026.0140.25-0.1817
Mistral-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=Mistral-7B, Method Components=COGLM2026.0140.04-0.2203
Gemma-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=Gemma-7B, Method Components=COGLM2026.0139.57-0.2096
Qwen2.5-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=Qwen2.5-7B, Method Components=COGLM2026.0139.1-0.2031
Claude 3.5 SonnetEvaluation Protocol=Zero-shot, Model Backbone=Claude 3.5 Sonnet2026.0138.18-0.0162
DeepSeek V3 + FS*Evaluation Protocol=Few-shot, Model Backbone=DeepSeek V32026.0136.37-0.0685
GPT-4oEvaluation Protocol=Zero-shot, Model Backbone=GPT-4o2026.0135.950.0022
DeepSeek-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=DeepSeek-7B, Method Components=COGLM2026.0132.92-0.1433
Llama 3 (70B) + FS*Evaluation Protocol=Few-shot, Model Backbone=Llama 3 (70B)2026.0132.690.0213
DeepSeek-7BFine-tuning Strategy=LoRA, Model Backbone=DeepSeek-7B2026.0131.88-0.085
Llama 3 (70B)Evaluation Protocol=Zero-shot, Model Backbone=Llama 3 (70B)2026.0120.67-0.0064