List your company
ResearchFormerly WizWand

Compositional Generalization on Evaluation Dataset (Fold 3 Seen)

66.69ScoreLLaMA-3-8B + COGLM
20.534832.517444.556.4826Jan 29, 2026
Updated 8mo ago

Evaluation results

MethodLinks
LLaMA-3-8B + COGLMFine-tuning Strategy=LoRA, Model Backbone=LLaMA-3-8B, Method Components=COGLM2026.0166.69
Mistral-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=Mistral-7B, Method Components=COGLM2026.0165.1
Mistral-7BFine-tuning Strategy=LoRA, Model Backbone=Mistral-7B2026.0162.64
Qwen2.5-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=Qwen2.5-7B, Method Components=COGLM2026.0161.73
Gemma-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=Gemma-7B, Method Components=COGLM2026.0161.7
Qwen2.5-7BFine-tuning Strategy=LoRA, Model Backbone=Qwen2.5-7B2026.0161.09
Gemma-7BFine-tuning Strategy=LoRA, Model Backbone=Gemma-7B2026.0160.43
LLaMA-3-8BFine-tuning Strategy=LoRA, Model Backbone=LLaMA-3-8B2026.0159.72
DeepSeek V3 + FS*Evaluation Protocol=Few-shot, Model Backbone=DeepSeek V32026.0146.62
DeepSeek-7B + COGLMFine-tuning Strategy=LoRA, Model Backbone=DeepSeek-7B, Method Components=COGLM2026.0145.47
Claude 3.5 Sonnet + FS*Evaluation Protocol=Few-shot, Model Backbone=Claude 3.5 Sonnet2026.0145.04
DeepSeek V3Evaluation Protocol=Zero-shot, Model Backbone=DeepSeek V32026.0142.46
Claude 3.5 SonnetEvaluation Protocol=Zero-shot, Model Backbone=Claude 3.5 Sonnet2026.0141.59
GPT-4o + FS*Evaluation Protocol=Few-shot, Model Backbone=GPT-4o2026.0140.05
DeepSeek-7BFine-tuning Strategy=LoRA, Model Backbone=DeepSeek-7B2026.0139.02
GPT-4oEvaluation Protocol=Zero-shot, Model Backbone=GPT-4o2026.0135.58
Llama 3 (70B) + FS*Evaluation Protocol=Few-shot, Model Backbone=Llama 3 (70B)2026.0133.15
Llama 3 (70B)Evaluation Protocol=Zero-shot, Model Backbone=Llama 3 (70B)2026.0122.31