List your company
ResearchFormerly WizWand

Reading Comprehension on SciQ

93.7AccuracyLLaMA2-7B
28.59645.49862.479.302Oct 10, 2023Mar 10, 2024Aug 9, 2024Jan 8, 2025Jun 9, 2025Nov 8, 2025Apr 9, 2026
Updated 6mo ago

Evaluation results

MethodLinks
LLaMA2-7B# tokens for training=2T, Shot number=0, Training data source=Different from RedPajama2023.1093.7
Open-LLaMA-3B-v2# tokens for training=1T, Shot number=0, Training data source=RedPajama2023.1091.8
Open-LLaMA-3B-v1# tokens for training=1T, Shot number=0, Training data source=RedPajama2023.1091.3
Sheared-LLaMA-2.7B# tokens for training=50B, Shot number=0, Training data source=RedPajama2023.1090.8
INCITE-Base-3B# tokens for training=800B, Shot number=0, Training data source=RedPajama2023.1090.7
TinyLlama-1.1B# tokens for training=3T, Shot number=0, Training data source=RedPajama2023.1088.9
Pythia-2.8B# tokens for training=300B, Shot number=0, Training data source=RedPajama2023.1088.3
Sheared-LLaMA-1.3B# tokens for training=50B, Shot number=0, Training data source=RedPajama2023.1087.3
Pythia-1.4B# tokens for training=300B, Shot number=0, Training data source=RedPajama2023.1086.4
OPT-2.7B# tokens for training=300B, Shot number=0, Training data source=Different from RedPajama2023.1085.8
OPT-1.3B# tokens for training=300B, Shot number=0, Training data source=Different from RedPajama2023.1084.3
DoGraphBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0466.1
RegMixBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0462.8
DoReMiBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0461.6
UniformBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0460.3
DOGEBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0460.1
Dynamic Loss-BasedBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0460
Data Mixing LawBackbone=LLaMA-1.1B, Pre-training Dataset=The Pile2026.0458.6
Dynamic Loss-BasedModel=GPT-2 Small, Training Data=SlimPajama2026.0453.8
DoGraphModel=GPT-2 Small, Training Data=SlimPajama2026.0453.8
DoGraphBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0453.5
Dynamic Loss-BasedBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0453.3
DoReMiBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0453.3
DOGEBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0452.8
DoReMiModel=GPT-2 Small, Training Data=SlimPajama2026.0452.4
UniformBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0449
RegMixBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0446.6
Data Mixing LawBackbone=GPT-2 Medium, Pre-training Dataset=SlimPajama2026.0445.2
Data Mixing LawModel=GPT-2 Small, Training Data=SlimPajama2026.0439.6
RegMixModel=GPT-2 Small, Training Data=SlimPajama2026.0438.7
UniformModel=GPT-2 Small, Training Data=SlimPajama2026.0432.6
DOGEModel=GPT-2 Small, Training Data=SlimPajama2026.0431.1