Reading Comprehension on RACE-m
0.931AccuracyFine-tuned SOTA
Evaluation Results
| Method | Links | |
|---|---|---|
| Fine-tuned SOTASetting=Fine-tuned2020.05 | 0.931 | |
| KALEBackbone=Qwen3-32B-thinking2026.01 | 0.8854 | |
| SFTBackbone=Qwen3-32B-thinking2026.01 | 0.8457 | |
| VanillaBackbone=Qwen3-32B-thinking2026.01 | 0.8148 | |
| PaLM 2-Lprompting=1-shot2023.05 | 0.77 | |
| PaLM 2-Mprompting=1-shot2023.05 | 0.719 | |
| PaLMprompting=1-shot2023.05 | 0.693 | |
| PaLM 2-Sprompting=1-shot2023.05 | 0.689 | |
| GPT-3Setting=Zero-Shot2020.05 | 0.584 | |
| GPT-3Setting=Few-Shot2020.05 | 0.581 | |
| GPT-3Setting=One-Shot2020.05 | 0.574 | |
| INTRADOCChunk Length (L)=8K2024.02 | 0.4556 | |
| INTRADOCChunk Length (L)=2K2024.02 | 0.4496 | |
| BM25ChunkChunk Length (L)=2K2024.02 | 0.4492 | |
| BM25ChunkChunk Length (L)=8K2024.02 | 0.4414 | |
| UNIChunkChunk Length (L)=2K2024.02 | 0.4352 | |
| MIXChunkChunk Length (L)=2K2024.02 | 0.4277 | |
| UNIChunkChunk Length (L)=8K2024.02 | 0.4164 | |
| MIXChunkChunk Length (L)=8K2024.02 | 0.4157 | |
| +49-class Div.Class Divisions=49-class, Training Tokens=150B, Model Size=15B-A1.5B2026.02 | 0.3466 | |
| +3-class Div.Class Divisions=3-class, Training Tokens=150B, Model Size=15B-A1.5B2026.02 | 0.3389 | |
| Baseline MoEClass Divisions=None, Training Tokens=150B, Model Size=15B-A1.5B2026.02 | 0.335 | |
| FineWeb-EduModel Backbone=GPT-2 XL2026.02 | 0.2681 | |
| DCLM-FastTextModel Backbone=GPT-2 XL2026.02 | 0.2653 | |
| GREATSModel Backbone=GPT-2 XL2026.02 | 0.2604 | |
| OPUSModel Backbone=GPT-2 XL2026.02 | 0.2577 | |
| DSIRModel Backbone=GPT-2 XL2026.02 | 0.2507 | |
| QuRatingModel Backbone=GPT-2 XL2026.02 | 0.2479 | |
| RandomModel Backbone=GPT-2 XL2026.02 | 0.2458 | |
| PPLModel Backbone=GPT-2 XL2026.02 | 0.2437 | |
| UltraFinewebModel Backbone=GPT-2 XL2026.02 | 0.2326 |