List your company
ResearchFormerly WizWand

Commonsense reasoning on HellaSwag 1.0 (test)

85.6AccuracyALUM-RoBERTa-Large
46.80856.87966.9577.021Apr 20, 2020Dec 8, 2020Jul 28, 2021Mar 17, 2022Nov 4, 2022Jun 24, 2023Feb 12, 2024
Updated 8mo ago

Evaluation results

MethodLinks
ALUM-RoBERTa-LargeBackbone=RoBERTa-Large, Training=Adversarial pre-training2020.0485.6
RoBERTa-LargeBackbone=RoBERTa-Large2020.0485.2
Mistral-7B + DSIRModel=Mistral-7B, Selection Method=DSIR, Protocol=10-shot2024.0263.1
Mistral-7B BaseModel=Mistral-7B, Selection Method=Base, Protocol=10-shot2024.0262.82
Mistral-7B + AutoDSModel=Mistral-7B, Selection Method=AutoDS, Protocol=10-shot2024.0262.72
Mistral-7B + QuRatingModel=Mistral-7B, Selection Method=QuRating, Protocol=10-shot2024.0262.64
Mistral-7B + UniformModel=Mistral-7B, Selection Method=Uniform, Protocol=10-shot2024.0262.21
LLaMA2-7B BaseModel=LLaMA2-7B, Selection Method=Base, Protocol=10-shot2024.0258.88
LLaMA2-7B + QuRatingModel=LLaMA2-7B, Selection Method=QuRating, Protocol=10-shot2024.0258.79
LLaMA2-7B + UniformModel=LLaMA2-7B, Selection Method=Uniform, Protocol=10-shot2024.0258.43
LLaMA2-7B + DSIRModel=LLaMA2-7B, Selection Method=DSIR, Protocol=10-shot2024.0258.38
LLaMA2-7B + AutoDSModel=LLaMA2-7B, Selection Method=AutoDS, Protocol=10-shot2024.0258.28
Gemma-2B + QuRatingModel=Gemma-2B, Selection Method=QuRating, Protocol=10-shot2024.0253.1
Gemma-2B + DSIRModel=Gemma-2B, Selection Method=DSIR, Protocol=10-shot2024.0252.95
Gemma-2B + UniformModel=Gemma-2B, Selection Method=Uniform, Protocol=10-shot2024.0252.91
Gemma-2B + AutoDSModel=Gemma-2B, Selection Method=AutoDS, Protocol=10-shot2024.0252.82
Gemma-2B BaseModel=Gemma-2B, Selection Method=Base, Protocol=10-shot2024.0248.3