Mathematical Reasoning on Math Reasoning Out-domain (SVAMP, Mathematics, SimulEq) (test)
79.6SVAMP AccuracyS2L
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| S2LTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=50K2024.03 | 79.6 | 56.4 | 40.1 | 57.3 | |
| Full-262KTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=262K2024.03 | 78.4 | 58.4 | 44.2 | 57.7 | |
| RandomTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=50K2024.03 | 77.1 | 51.2 | 37.5 | 54.1 | |
| High LearnabilityTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=50K2024.03 | 76.6 | 41.8 | 27.2 | 48.7 | |
| Least ConfidenceTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=50K2024.03 | 76.5 | 43.3 | 52.5 | 54.3 | |
| Middle PerplexityTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=50K2024.03 | 74.8 | 50.4 | 39.8 | 52.5 | |
| Facility LocationsTarget Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data Size=50K2024.03 | 74.4 | 58.4 | 34.6 | 54.5 | |
| Phi-2 (Pretrained)Target Model=Phi-2 (2.7B), Evaluation Protocol=Zero-shot, Fine-tuning Data=None2024.03 | 67.9 | 31.1 | 27.4 | 38.5 |