List your company
ResearchFormerly WizWand

Story Reasoning on XStoryCloze

71AccuracyBETR
47.464853.574959.68565.7951Nov 15, 2023Apr 10, 2024Sep 4, 2024Jan 30, 2025Jun 26, 2025Nov 20, 2025Apr 17, 2026
Updated 5mo ago

Evaluation results

MethodLinks
BETRTarget Setting=Single-Target2026.0471
NAG_Llama-3.2-3BTarget Setting=Single-Target, Backbone=Llama-3.2-3B2026.0470.8
NAG_SmolLM3-3BTarget Setting=Single-Target, Backbone=SmolLM3-3B2026.0470.5
NAG_Qwen3-1.7BTarget Setting=Single-Target, Backbone=Qwen3-1.7B2026.0470
NAG_Llama-3.2-3BTarget Setting=Multi-Target, Backbone=Llama-3.2-3B2026.0469.8
BETRTarget Setting=Multi-Target2026.0469.5
NAG_Qwen3-1.7BTarget Setting=Multi-Target, Backbone=Qwen3-1.7B2026.0469.3
NAG_SmolLM3-3BTarget Setting=Multi-Target, Backbone=SmolLM3-3B2026.0469.2
Random2026.0467.1
FineWeb-Edu2026.0465.9
Ours-SFTalignment=SFT2025.0761.86
Ours-Base2025.0760.8
Ours-Base-32kcontext-length=32k2025.0760.6
Gemma 2Number of Parameters=1B, Evaluation Protocol=Zero-shot2024.0858.5
FTBase Model=LLaMA-7B, Training Context=Alpaca-En2023.1157.6
FTBase Model=LLaMA-7B, Training Context=Alpaca-X2023.1157.2
LoRABase Model=LLaMA-7B, Training Context=Alpaca-En2023.1157
LoRABase Model=LLaMA-7B, Training Context=Alpaca-X2023.1157
DMoEModel Scale=1.7B, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Zero-shot2025.0656.6
BLOOM + Continued Pre-trainingModel Scale=1.7B, Training Strategy=Continued Pre-training, Evaluation Protocol=Zero-shot2025.0656.4
Branch-Train-MixModel Scale=1.7B, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Zero-shot2025.0656.4
Branch-Train-MixModel Scale=1.7B, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Few-shot (4-shot)2025.0656.2
XGLMNumber of Parameters=1.7B, Evaluation Protocol=Zero-shot2024.0856.2
DMoEModel Scale=1.7B, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Few-shot (4-shot)2025.0656.1
Parrot-7BBase Model=Parrot-7B2023.1156.1
BLOOMModel Scale=1.7B, Training Strategy=Base, Evaluation Protocol=Zero-shot2025.0656
BLOOM + Continued Pre-trainingModel Scale=1.7B, Training Strategy=Continued Pre-training, Evaluation Protocol=Few-shot (4-shot)2025.0656
Embed FTBase Model=LLaMA-7B, Training Context=Alpaca-En2023.1155.9
Embed FTBase Model=LLaMA-7B, Training Context=Alpaca-X2023.1155.9
LoRABase Model=LLaMA-7B, Training Context=Bilingual2023.1155.9
Embed FTBase Model=LLaMA-7B, Training Context=Bilingual2023.1155.9
FTBase Model=LLaMA-7B, Training Context=Bilingual2023.1155.6
BLOOMModel Scale=1.7B, Training Strategy=Base, Evaluation Protocol=Few-shot (4-shot)2025.0655.5
LLaMA-7BBase Model=LLaMA-7B2023.1155.5
LLaMA 3Number of Parameters=1B, Evaluation Protocol=Zero-shot2024.0855.1
DMoEModel Scale=560M, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Zero-shot2025.0654.1
BLOOM + Continued Pre-trainingModel Scale=560M, Training Strategy=Continued Pre-training, Evaluation Protocol=Zero-shot2025.0653.8
Branch-Train-MixModel Scale=560M, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Zero-shot2025.0653.8
BLOOMModel Scale=560M, Training Strategy=Base, Evaluation Protocol=Zero-shot2025.0653.5
DMoEModel Scale=560M, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Few-shot (4-shot)2025.0653.4
Branch-Train-MixModel Scale=560M, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Few-shot (4-shot)2025.0653.1
XGLMNumber of Parameters=564M, Evaluation Protocol=Zero-shot2024.0853.1
BLOOM + Continued Pre-trainingModel Scale=560M, Training Strategy=Continued Pre-training, Evaluation Protocol=Few-shot (4-shot)2025.0652.7
Gemma 2Number of Parameters=270M, Evaluation Protocol=Zero-shot2024.0852.7
BLOOMModel Scale=560M, Training Strategy=Base, Evaluation Protocol=Few-shot (4-shot)2025.0652.6
BLOOMNumber of Parameters=560M, Evaluation Protocol=Zero-shot2024.0852.6
GoldfishNumber of Parameters=124M, Evaluation Protocol=Zero-shot2024.0852.3
Chance2024.0850
Tibetan-Alpaca-7B2025.0749.97
Tibetan-Llama2-7B2025.0749.37
Yak-Llama2-7B2025.0748.37