List your company
ResearchFormerly WizWand

Instruction Following on NaturalInstructions

47.24ROUGE ScoreVanilla
40.781642.458344.13545.8117May 22, 2025
Updated 8mo ago

Evaluation results

MethodLinks
VanillaModel=o4-mini, Planning Strategy=Direct2025.0547.24460.994.84
Global BudgetModel=o4-mini, Planning Strategy=Direct2025.0545.39422.24.88
Global BudgetModel=QwQ-32B, Planning Strategy=Planned2025.0545.131,265.781.61
Global BudgetModel=QwQ-32B, Planning Strategy=Direct2025.0544.771,360.491.47
PLAN-AND-BUDGET + PolynomialModel=QwQ-32B, Planning Strategy=Planned2025.0544.66995.952
PLAN-AND-BUDGET + PolynomialModel=o4-mini, Planning Strategy=Planned2025.0544.63363.165.48
PLAN-AND-BUDGET + CosineModel=QwQ-32B, Planning Strategy=Planned2025.0544.531,000.641.98
PLAN-AND-BUDGET + UniformModel=QwQ-32B, Planning Strategy=Planned2025.0544.47996.911.98
PLAN-AND-BUDGET + WeightedModel=QwQ-32B, Planning Strategy=Planned2025.0544.41,025.021.92
PLAN-AND-BUDGET + CosineModel=o4-mini, Planning Strategy=Planned2025.0544.36363.055.42
VanillaModel=QwQ-32B, Planning Strategy=Planned2025.0544.241,426.741.37
PLAN-AND-BUDGET + LinearModel=QwQ-32B, Planning Strategy=Planned2025.0544.221,003.241.95
PLAN-AND-BUDGET + LinearModel=o4-mini, Planning Strategy=Planned2025.0544.21363.655.37
PLAN-AND-BUDGET + UniformModel=o4-mini, Planning Strategy=Planned2025.0544.08348.745.57
PLAN-AND-BUDGET + ExponentialModel=QwQ-32B, Planning Strategy=Planned2025.0543.991,026.891.88
Global BudgetModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Direct2025.0543.8772.982.48
VanillaModel=o4-mini, Planning Strategy=Planned2025.0543.78344.995.56
Global BudgetModel=o4-mini, Planning Strategy=Planned2025.0543.78358.845.34
PLAN-AND-BUDGET + WeightedModel=o4-mini, Planning Strategy=Planned2025.0543.72371.855.14
PLAN-AND-BUDGET + ExponentialModel=o4-mini, Planning Strategy=Planned2025.0543.68364.865.23
VanillaModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Direct2025.0543.47968.171.95
VanillaModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0543.4821.272.29
PLAN-AND-BUDGET + PolynomialModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0543.19641.622.91
VanillaModel=QwQ-32B, Planning Strategy=Direct2025.0543.161,818.341.02
VanillaModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Direct2025.0543.13894.462.08
PLAN-AND-BUDGET + UniformModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0543.06665.942.78
PLAN-AND-BUDGET + WeightedModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0543.05626.372.96
PLAN-AND-BUDGET + CosineModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0542.83657.932.79
Global BudgetModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Direct2025.0542.81787.252.33
PLAN-AND-BUDGET + ExponentialModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0542.73622.722.93
Global BudgetModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0542.5717.982.52
VanillaModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0542.48860.852.1
Global BudgetModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0542.48691.792.61
PLAN-AND-BUDGET + LinearModel=DeepSeek-R1-Distill-Llama-70B, Planning Strategy=Planned2025.0542.05613.052.88
PLAN-AND-BUDGET + LinearModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0541.56633.792.73
PLAN-AND-BUDGET + ExponentialModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0541.44650.192.64
PLAN-AND-BUDGET + PolynomialModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0541.44600.042.86
PLAN-AND-BUDGET + CosineModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0541.43628.22.73
PLAN-AND-BUDGET + WeightedModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0541.29663.942.57
PLAN-AND-BUDGET + UniformModel=DeepSeek-R1-Distill-Qwen-32B, Planning Strategy=Planned2025.0541.03644.872.61