List your company
ResearchFormerly WizWand

Complex Reasoning on BBH (val)

65.81AccuracyG2IS
53.413256.631659.8563.0684Feb 16, 2025
Updated 8mo ago

Evaluation results

MethodLinks
G2ISBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=5%2025.0265.81
G2ISBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=1%2025.0265.01
G2ISBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=1%2025.0264.6
LESSBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=1%2025.0264.05
BERTBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=1%2025.0263.74
LESSBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=1%2025.0263.69
G2ISBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=5%2025.0263.69
Full Dataset TuningBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=100%2025.0263.52
LESSBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=5%2025.0263.46
BERTBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=5%2025.0263.4
BERTBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=1%2025.0263.23
BERTBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=5%2025.0262.4
G2ISBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=5%2025.0261.36
G2ISBackbone=Gemma-7B, Prompting=COT, Selection Ratio=1%2025.0261.36
G2ISBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=1%2025.0261.05
BERTBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=1%2025.0260.82
G2ISBackbone=Gemma-7B, Prompting=COT, Selection Ratio=5%2025.0260.74
BERTBackbone=Gemma-7B, Prompting=COT, Selection Ratio=1%2025.0260.57
Full Dataset TuningBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=100%2025.0260.18
BERTBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=5%2025.0260.07
LESSBackbone=Gemma-7B, Prompting=COT, Selection Ratio=1%2025.0259.35
LESSBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=5%2025.0259.07
G2ISBackbone=Mistral-7B, Prompting=COT, Selection Ratio=1%2025.0258.93
LESSBackbone=Gemma-7B, Prompting=COT, Selection Ratio=5%2025.0258.65
G2ISBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=1%2025.0258.45
Full Dataset TuningBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=100%2025.0258.29
G2ISBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=5%2025.0258.09
G2ISBackbone=Mistral-7B, Prompting=COT, Selection Ratio=5%2025.0258.04
LESSBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=1%2025.0256.87
LESSBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=1%2025.0256.7
Full Dataset TuningBackbone=Gemma-7B, Prompting=COT, Selection Ratio=100%2025.0256.58
BERTBackbone=Gemma-7B, Prompting=COT, Selection Ratio=5%2025.0256.54
BERTBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=1%2025.0256.37
LESSBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=5%2025.0256.14
BERTBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=5%2025.0256.09
LESSBackbone=Mistral-7B, Prompting=COT, Selection Ratio=1%2025.0256.01
BERTBackbone=Mistral-7B, Prompting=COT, Selection Ratio=1%2025.0255.64
BERTBackbone=Mistral-7B, Prompting=COT, Selection Ratio=5%2025.0255.55
LESSBackbone=Mistral-7B, Prompting=COT, Selection Ratio=5%2025.0255.24
Full Dataset TuningBackbone=Mistral-7B, Prompting=COT, Selection Ratio=100%2025.0254.58
Full Dataset TuningBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=100%2025.0254.29
LESSBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=5%2025.0253.89