World Knowledge and Reading Comprehension on NQ, MMLU STEM, ARC, SciQ, LogiQA, BoolQ
29.81NQ AccuracyMistral-7B Base
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Mistral-7B BaseBackbone=Mistral-7B, Selection Method=None (Base Model), Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 29.81 | 52.39 | 84.68 | 57.25 | 97.4 | 30.26 | 83.58 | 54.3 | |
| DSIRBackbone=Mistral-7B, Selection Method=DSIR, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 29.22 | 52.62 | 84.72 | 57.25 | 97.3 | 30.26 | 73.76 | 53.54 | |
| Uniform (OpenWebMath)Backbone=Mistral-7B, Selection Method=Uniform Sampling, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 29.17 | 52.17 | 84.18 | 56.66 | 97.2 | 31.03 | 83.82 | 54.91 | |
| AutoDSBackbone=Mistral-7B, Selection Method=AutoDS, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 29.06 | 52.3 | 84.18 | 55.2 | 96.8 | 31.03 | 83.12 | 55.19 | |
| QuRatingBackbone=Mistral-7B, Selection Method=QuRating, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 28.89 | 52.01 | 85.48 | 57.76 | 97.3 | 31.18 | 82.81 | 54.03 | |
| Uniform (OpenWebMath)Backbone=LLaMA2-7B, Selection Method=Uniform Sampling, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 26.07 | 40.09 | 80.77 | 50.09 | 96.7 | 27.65 | 78.41 | 46.62 | |
| LLaMA2-7B BaseBackbone=LLaMA2-7B, Selection Method=None (Base Model), Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 26.01 | 37.08 | 80.72 | 49.74 | 96.8 | 26.57 | 77.68 | 44.99 | |
| QuRatingBackbone=LLaMA2-7B, Selection Method=QuRating, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 25.96 | 37.84 | 80.43 | 50.6 | 96.8 | 27.65 | 77.71 | 44.97 | |
| AutoDSBackbone=LLaMA2-7B, Selection Method=AutoDS, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 25.84 | 40.66 | 80.09 | 49.74 | 96.7 | 27.96 | 77.19 | 47.07 | |
| DSIRBackbone=LLaMA2-7B, Selection Method=DSIR, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 25.76 | 36.63 | 80.98 | 48.98 | 96.5 | 26.73 | 72.54 | 44.27 | |
| Gemma-2B BaseBackbone=Gemma-2B, Selection Method=None (Base Model), Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 14.88 | 36.6 | 77.61 | 46.5 | 96.3 | 25.35 | 69.54 | 42.92 | |
| QuRatingBackbone=Gemma-2B, Selection Method=QuRating, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 14.32 | 33.81 | 77.95 | 46.76 | 96.2 | 24.42 | 68.53 | 41.67 | |
| Uniform (OpenWebMath)Backbone=Gemma-2B, Selection Method=Uniform Sampling, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 13.8 | 36.54 | 77.4 | 45.39 | 96.4 | 26.27 | 68.35 | 42.95 | |
| DSIRBackbone=Gemma-2B, Selection Method=DSIR, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 13.27 | 34.44 | 77.27 | 45.82 | 95.9 | 23.5 | 54.92 | 39.71 | |
| AutoDSBackbone=Gemma-2B, Selection Method=AutoDS, Few-shot settings=NQ: 5, MMLU STEM: 5, ARC: 25, SciQ: 2, LogiQA: 2, BoolQ: 02024.02 | 13.27 | 36.09 | 76.81 | 46.08 | 96.1 | 27.19 | 71.28 | 43.16 |