List your company
ResearchFormerly WizWand

Reading Comprehension on RACE-m

0.931AccuracyFine-tuned SOTA
0.2046640.3932320.58180.770368May 28, 2020May 10, 2021Apr 22, 2022Apr 5, 2023Mar 17, 2024Feb 27, 2025Feb 10, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Fine-tuned SOTASetting=Fine-tuned2020.050.931
KALEBackbone=Qwen3-32B-thinking2026.010.8854
SFTBackbone=Qwen3-32B-thinking2026.010.8457
VanillaBackbone=Qwen3-32B-thinking2026.010.8148
PaLM 2-Lprompting=1-shot2023.050.77
PaLM 2-Mprompting=1-shot2023.050.719
PaLMprompting=1-shot2023.050.693
PaLM 2-Sprompting=1-shot2023.050.689
GPT-3Setting=Zero-Shot2020.050.584
GPT-3Setting=Few-Shot2020.050.581
GPT-3Setting=One-Shot2020.050.574
INTRADOCChunk Length (L)=8K2024.020.4556
INTRADOCChunk Length (L)=2K2024.020.4496
BM25ChunkChunk Length (L)=2K2024.020.4492
BM25ChunkChunk Length (L)=8K2024.020.4414
UNIChunkChunk Length (L)=2K2024.020.4352
MIXChunkChunk Length (L)=2K2024.020.4277
UNIChunkChunk Length (L)=8K2024.020.4164
MIXChunkChunk Length (L)=8K2024.020.4157
+49-class Div.Class Divisions=49-class, Training Tokens=150B, Model Size=15B-A1.5B2026.020.3466
+3-class Div.Class Divisions=3-class, Training Tokens=150B, Model Size=15B-A1.5B2026.020.3389
Baseline MoEClass Divisions=None, Training Tokens=150B, Model Size=15B-A1.5B2026.020.335
FineWeb-EduModel Backbone=GPT-2 XL2026.020.2681
DCLM-FastTextModel Backbone=GPT-2 XL2026.020.2653
GREATSModel Backbone=GPT-2 XL2026.020.2604
OPUSModel Backbone=GPT-2 XL2026.020.2577
DSIRModel Backbone=GPT-2 XL2026.020.2507
QuRatingModel Backbone=GPT-2 XL2026.020.2479
RandomModel Backbone=GPT-2 XL2026.020.2458
PPLModel Backbone=GPT-2 XL2026.020.2437
UltraFinewebModel Backbone=GPT-2 XL2026.020.2326