Complex Reasoning on BBH (val)
65.81AccuracyG2IS
Evaluation Results
| Method | Links | |
|---|---|---|
| G2ISBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=5%2025.02 | 65.81 | |
| G2ISBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=1%2025.02 | 65.01 | |
| G2ISBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=1%2025.02 | 64.6 | |
| LESSBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=1%2025.02 | 64.05 | |
| BERTBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=1%2025.02 | 63.74 | |
| LESSBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=1%2025.02 | 63.69 | |
| G2ISBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=5%2025.02 | 63.69 | |
| Full Dataset TuningBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=100%2025.02 | 63.52 | |
| LESSBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=5%2025.02 | 63.46 | |
| BERTBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=5%2025.02 | 63.4 | |
| BERTBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=1%2025.02 | 63.23 | |
| BERTBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=5%2025.02 | 62.4 | |
| G2ISBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=5%2025.02 | 61.36 | |
| G2ISBackbone=Gemma-7B, Prompting=COT, Selection Ratio=1%2025.02 | 61.36 | |
| G2ISBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=1%2025.02 | 61.05 | |
| BERTBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=1%2025.02 | 60.82 | |
| G2ISBackbone=Gemma-7B, Prompting=COT, Selection Ratio=5%2025.02 | 60.74 | |
| BERTBackbone=Gemma-7B, Prompting=COT, Selection Ratio=1%2025.02 | 60.57 | |
| Full Dataset TuningBackbone=Llama3.1-8B, Prompting=COT, Selection Ratio=100%2025.02 | 60.18 | |
| BERTBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=5%2025.02 | 60.07 | |
| LESSBackbone=Gemma-7B, Prompting=COT, Selection Ratio=1%2025.02 | 59.35 | |
| LESSBackbone=Llama3.1-8B, Prompting=FLAN, Selection Ratio=5%2025.02 | 59.07 | |
| G2ISBackbone=Mistral-7B, Prompting=COT, Selection Ratio=1%2025.02 | 58.93 | |
| LESSBackbone=Gemma-7B, Prompting=COT, Selection Ratio=5%2025.02 | 58.65 | |
| G2ISBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=1%2025.02 | 58.45 | |
| Full Dataset TuningBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=100%2025.02 | 58.29 | |
| G2ISBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=5%2025.02 | 58.09 | |
| G2ISBackbone=Mistral-7B, Prompting=COT, Selection Ratio=5%2025.02 | 58.04 | |
| LESSBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=1%2025.02 | 56.87 | |
| LESSBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=1%2025.02 | 56.7 | |
| Full Dataset TuningBackbone=Gemma-7B, Prompting=COT, Selection Ratio=100%2025.02 | 56.58 | |
| BERTBackbone=Gemma-7B, Prompting=COT, Selection Ratio=5%2025.02 | 56.54 | |
| BERTBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=1%2025.02 | 56.37 | |
| LESSBackbone=Gemma-7B, Prompting=FLAN, Selection Ratio=5%2025.02 | 56.14 | |
| BERTBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=5%2025.02 | 56.09 | |
| LESSBackbone=Mistral-7B, Prompting=COT, Selection Ratio=1%2025.02 | 56.01 | |
| BERTBackbone=Mistral-7B, Prompting=COT, Selection Ratio=1%2025.02 | 55.64 | |
| BERTBackbone=Mistral-7B, Prompting=COT, Selection Ratio=5%2025.02 | 55.55 | |
| LESSBackbone=Mistral-7B, Prompting=COT, Selection Ratio=5%2025.02 | 55.24 | |
| Full Dataset TuningBackbone=Mistral-7B, Prompting=COT, Selection Ratio=100%2025.02 | 54.58 | |
| Full Dataset TuningBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=100%2025.02 | 54.29 | |
| LESSBackbone=Mistral-7B, Prompting=FLAN, Selection Ratio=5%2025.02 | 53.89 |