List your company
ResearchFormerly WizWand

Reading Comprehension on RACE-h (test)

89.95AccuracyQwen-1.5 14B (Teacher)
20.238838.336956.43574.5331Dec 13, 2021Sep 8, 2022Jun 5, 2023Feb 29, 2024Nov 25, 2024Aug 21, 2025May 18, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen-1.5 14B (Teacher)Model Scale=14B, Evaluation Protocol=Zero-shot2024.0789.95
GopherModel Size=280B, Evaluation Protocol=Few-shot, Shots=52021.1271.6
Qwen-1.5 1.8B + DDKModel Scale=1.8B, Distillation Strategy=DDK2024.0771.56
Qwen-1.5 1.8B + CPT & DoReMiModel Scale=1.8B, Distillation Strategy=CPT & DoReMi2024.0770.12
Qwen-1.5 1.8B + KDModel Scale=1.8B, Distillation Strategy=KD2024.0770.12
Qwen-1.5 1.8B + TEDModel Scale=1.8B, Distillation Strategy=TED2024.0769.95
Qwen-1.5 1.8B (Student)Model Scale=1.8B, Evaluation Protocol=Zero-shot2024.0769.57
Qwen-1.5 1.8B + CPTModel Scale=1.8B, Distillation Strategy=CPT2024.0769
Qwen-1.5 1.8B + MiniLLMModel Scale=1.8B, Distillation Strategy=MiniLLM2024.0768.92
LLaMA2-13BModel Size=13B, Shots=0-shot2024.0762.85
GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=22021.1249.3
GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.1248.7
Megatron-NLGModel Size=530B, Evaluation Protocol=Few-shot2021.1247.9
GLaMModel Size=64B/64E, Evaluation Protocol=Zero-shot2021.1246.9
GPT-3Model Size=175B, Evaluation Protocol=Few-shot, Shots=102021.1246.8
GPT-3Model Size=175B, Evaluation Protocol=One-shot2021.1245.9
GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2021.1245.5
AMOOptimizer=AMO, Model=Llama3.1-1.4B, Few-shot=02026.0533
AMOOptimizer=AMO, Model=Llama3.1-760M, Few-shot=02026.0531.43
TinyLLaMA-1.1B + DDKModel Size=1.1B, Shots=0-shot2024.0727.76
TinyLLaMA-1.1B + MiniLLMModel Size=1.1B, Shots=0-shot2024.0724.32
TinyLLaMA-1.1B + CPTModel Size=1.1B, Shots=0-shot2024.0724.24
TinyLLaMA-1.1B + TEDModel Size=1.1B, Shots=0-shot2024.0724.18
TinyLLaMA-1.1B + CPT & DoReMiModel Size=1.1B, Shots=0-shot2024.0724.04
TinyLLaMA-1.1B + KDModel Size=1.1B, Shots=0-shot2024.0723.84
TinyLLaMA-1.1BModel Size=1.1B, Shots=0-shot2024.0722.92