Commonsense Reasoning on HellaSwag (Accuracy, AUROC, ECE)
95.1AccuracyMashup Learning
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mashup LearningBackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 95.1 | — | — | |
| From scratchBackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 93.2 | — | — | |
| Text-to-LoRA used as initBackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 91.6 | — | — | |
| Qwen3 30Bconfidence_type=normalized, evaluation_protocol=output probabilities2026.02 | 87.36 | 0.857 | 0.102 | |
| GLM4 9Bconfidence_type=normalized, evaluation_protocol=output probabilities2026.02 | 83.26 | 0.827 | 0.016 | |
| Gemma3 12Bconfidence_type=normalized, evaluation_protocol=output probabilities2026.02 | 79.97 | 0.804 | 0.181 | |
| Qwen3 4Bconfidence_type=normalized, evaluation_protocol=output probabilities2026.02 | 79.64 | 0.809 | 0.182 | |
| Gemma3 4Bconfidence_type=normalized, evaluation_protocol=output probabilities2026.02 | 63.1 | 0.698 | 0.357 | |
| Mashup Learning at initBackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 61.2 | — | — | |
| Bloom-7b1Ratio=0%, Evaluation Protocol=Zero-shot2024.03 | 59.67 | — | — | |
| SmolLM-1.7B-Instruct0-shot=true2026.03 | 56.1 | — | — | |
| SmolLM-1.7B-Instructshot=0-shot2026.03 | 56.1 | — | — | |
| TinyLlama-1.1B0-shot=true2026.03 | 55 | — | — | |
| TinyLlama-1.1Bshot=0-shot2026.03 | 55 | — | — | |
| OPT-1.3B0-shot=true2026.03 | 53.7 | — | — | |
| OPT-1.3Bshot=0-shot2026.03 | 53.7 | — | — | |
| DATEDGPT-INSTRUCT-20240-shot=true, Training data cutoff year=20242026.03 | 53.2 | — | — | |
| DATEDGPT-INSTRUCT-2024shot=0-shot2026.03 | 53.2 | — | — | |
| DATEDGPT-INSTRUCT-2023shot=0-shot2026.03 | 52.7 | — | — | |
| DATEDGPT-INSTRUCT-2022shot=0-shot2026.03 | 51.4 | — | — | |
| DATEDGPT-INSTRUCT-2020shot=0-shot2026.03 | 51.1 | — | — | |
| DATEDGPT-INSTRUCT-20210-shot=true, Training data cutoff year=20212026.03 | 51 | — | — | |
| DATEDGPT-INSTRUCT-2021shot=0-shot2026.03 | 51 | — | — | |
| GPT2-XL0-shot=true2026.03 | 50.8 | — | — | |
| GPT2-XLshot=0-shot2026.03 | 50.8 | — | — | |
| DATEDGPT-INSTRUCT-2019shot=0-shot2026.03 | 50.6 | — | — | |
| DATEDGPT-INSTRUCT-2018shot=0-shot2026.03 | 50.5 | — | — | |
| DATEDGPT-INSTRUCT-20170-shot=true, Training data cutoff year=20172026.03 | 50 | — | — | |
| DATEDGPT-INSTRUCT-2017shot=0-shot2026.03 | 50 | — | — | |
| HyWIARatio=25%, Evaluation Protocol=Zero-shot2024.03 | 49.49 | — | — | |
| DATEDGPT-INSTRUCT-2016shot=0-shot2026.03 | 48.4 | — | — | |
| LLM-Pruner Element²⋆Ratio=25%, Evaluation Protocol=Zero-shot2024.03 | 48.28 | — | — | |
| DATEDGPT-INSTRUCT-2015shot=0-shot2026.03 | 48 | — | — | |
| DATEDGPT-INSTRUCT-2014shot=0-shot2026.03 | 47.8 | — | — | |
| LLM-Pruner Vector⋆Ratio=25%, Evaluation Protocol=Zero-shot2024.03 | 47.65 | — | — | |
| Text-to-LoRABackbone=Mistral-7B-Instruct-v0.2, Setup=Lots-of-LoRAs2026.03 | 47.6 | — | — | |
| DATEDGPT-INSTRUCT-20130-shot=true, Training data cutoff year=20132026.03 | 47.6 | — | — | |
| DATEDGPT-INSTRUCT-2013shot=0-shot2026.03 | 47.6 | — | — | |
| Pythia-1B0-shot=true2026.03 | 47.1 | — | — | |
| Pythia-1Bshot=0-shot2026.03 | 47.1 | — | — | |
| DoGraphModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26.3 | — | — | |
| Dynamic Loss-BasedModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26.2 | — | — | |
| DOGEModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26.2 | — | — | |
| DoReMiModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26.1 | — | — | |
| Data Mixing LawModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26.1 | — | — | |
| UniformModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26 | — | — | |
| RegMixModel=GPT-2 Small, Training Data=SlimPajama2026.04 | 26 | — | — |