Story completion on StoryCloze
99.47AccuracyLlama-3.3-70B-Instruct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama-3.3-70B-InstructShots=02026.04 | 99.47 | — | — | |
| Qwen3-14BShots=02026.04 | 98.81 | — | — | |
| SecGPT-14BShots=02026.04 | 97.55 | — | — | |
| T5(3B) + PE w/ ROE (ORC.)Backbone=T5 (3B), Expert type=Oracle Retrieval-of-Expert, Additional Parameters=100M2023.02 | 97.47 | — | — | |
| XekRung-8BShots=02026.04 | 97.42 | — | — | |
| Qwen3-8BShots=02026.04 | 97.09 | — | — | |
| T5(3B) + Cos PEBackbone=T5 (3B), Expert type=Prompt Expert trained on COSMOS-QA, Additional Parameters=100M2023.02 | 97.06 | — | — | |
| Qwen3.5-9BShots=02026.04 | 97.02 | — | — | |
| LATModel=microsoft/deberta-xxlarge-v2-mnli2023.10 | 97 | — | — | |
| FLIPPEDModel Size=11B, Evaluation Protocol=Zero-shot2022.10 | 96.12 | — | — | |
| FLIPPEDModel Size=3B, Evaluation Protocol=Zero-shot2022.10 | 95.88 | — | — | |
| T0-11BParams=11B2022.10 | 94.7 | 4.7 | — | |
| KiC-LargeParams=0.77B2022.10 | 94.4 | 1.2 | — | |
| DirectModel Size=3B, Evaluation Protocol=Zero-shot2022.10 | 94.24 | — | — | |
| T0-11BParameters=11B2023.02 | 92.4 | — | — | |
| T0Model Size=11B, Evaluation Protocol=Zero-shot2022.10 | 92.4 | — | — | |
| SOTA2020.05 | 91.8 | — | — | |
| Spherical SteeringZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 89.08 | — | — | |
| GPT-3setting=few-shot, model_size=175B2020.05 | 87.7 | — | — | |
| TrajectoryTrain Dataset=ARC-E, Model=Qwen2.5-14b2026.03 | 87.28 | — | — | |
| T5(3B) + PE w/ ROEBackbone=T5 (3B), Expert type=Retrieval-of-Expert, Additional Parameters=100M2023.02 | 86.33 | — | — | |
| CCSModel=microsoft/deberta-xxlarge-v2-mnli2023.10 | 86 | — | — | |
| Our Trained ModelModel Size=52B, Data type=FP16, Zero-shot=true2023.05 | 85.87 | — | — | |
| Our Trained ModelModel Size=52B, Data type=W8A8, Zero-shot=true2023.05 | 85.5 | — | — | |
| T0-3BParams=3B2022.10 | 85.1 | 3.2 | — | |
| T0-3BParameters=3B2023.02 | 84.91 | — | — | |
| GPT-3setting=one-shot, model_size=175B2020.05 | 84.7 | — | — | |
| Few-shot AccuracyModel=Qwen2.5-14b2026.03 | 84.4 | — | — | |
| ASR + Qwen2.5-7BSystem Type=Cascaded topline, ASR=Whisper-v3-Large2025.10 | 84.2 | — | 0.8 | |
| T0-LargeParams=0.77B2022.10 | 84.1 | 1.8 | — | |
| T0Model Size=3B, Evaluation Protocol=Zero-shot2022.10 | 84.03 | — | — | |
| GPT-3setting=zero-shot, model_size=175B2020.05 | 83.2 | — | — | |
| GPT-3 (175B)Parameters=175B2023.02 | 83.2 | — | — | |
| GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2022.10 | 83.2 | — | — | |
| ASR + Qwen2.5-3BSystem Type=Cascaded topline, ASR=Whisper-v3-Large2025.10 | 82.7 | — | 0.2 | |
| KiC-BaseParams=0.22B2022.10 | 82.6 | 0.8 | — | |
| Our Trained ModelModel Size=52B, Data type=W4, Zero-shot=true2023.05 | 82.56 | — | — | |
| TrajectoryTrain Dataset=OpenQA, Model=Qwen2.5-14b2026.03 | 82.34 | — | — | |
| Our Trained ModelModel Size=13B, Data type=W8A8, Zero-shot=true2023.05 | 82.2 | — | — | |
| Our Trained ModelModel Size=13B, Data type=FP16, Zero-shot=true2023.05 | 82.11 | — | — | |
| TrajectoryTrain Dataset=ARC-C, Model=Qwen2.5-14b2026.03 | 81.72 | — | — | |
| Our Trained ModelModel Size=6B, Data type=FP16, Zero-shot=true2023.05 | 81.5 | — | — | |
| SALAD-7BTraining Stage=Stage I2025.10 | 81.5 | — | 3.5 | |
| SALAD-7BTraining Stage=Stage II2025.10 | 81.5 | — | 3.5 | |
| Our Trained ModelModel Size=6B, Data type=W8A8, Zero-shot=true2023.05 | 81 | — | — | |
| T0-BaseParams=0.22B2022.10 | 80.6 | 1.3 | — | |
| Zero-shot AccuracyModel=Qwen2.5-14b2026.03 | 80.6 | — | — | |
| Qwen2.5-Omni-7BSystem Type=End-to-end system2025.10 | 80.1 | — | 4.9 | |
| CAAZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 79.02 | — | — | |
| GLM-4-Voice-9BSystem Type=End-to-end system2025.10 | 78.2 | — | 20.6 | |
| Our Trained ModelModel Size=13B, Data type=W4, Zero-shot=true2023.05 | 77.4 | — | — | |
| Our Trained ModelModel Size=6B, Data type=W4, Zero-shot=true2023.05 | 77.21 | — | — | |
| LLM-CASTarget LLM=LLaMA2-7B-CHAT2025.12 | 76.04 | — | — | |
| SALAD-3BTraining Stage=Stage II2025.10 | 75.8 | — | 7.1 | |
| SADI-HEADZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 75.72 | — | — | |
| SALAD-3BTraining Stage=Stage I2025.10 | 75.5 | — | 7.4 | |
| ITIZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 75.12 | — | — | |
| BaselineZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 74.72 | — | — | |
| CAATarget LLM=LLaMA2-7B-CHAT2025.12 | 74.65 | — | — | |
| Qwen2-Audio-7BSystem Type=End-to-end system2025.10 | 71.9 | — | 9 | |
| Our Trained ModelModel Size=410M, Data type=FP16, Zero-shot=true2023.05 | 70.5 | — | — | |
| Linear ProbeTrain Dataset=ARC-C, Model=Qwen2.5-14b2026.03 | 69.94 | — | — | |
| Our Trained ModelModel Size=410M, Data type=W8A8, Zero-shot=true2023.05 | 69.7 | — | — | |
| DiVA-Llama3.1-8BSystem Type=End-to-end system2025.10 | 68.6 | — | 19.7 | |
| ITITarget LLM=LLaMA2-7B-CHAT2025.12 | 68.5 | — | — | |
| SADITarget LLM=LLaMA2-7B-CHAT2025.12 | 67.57 | — | — | |
| Our Trained ModelModel Size=410M, Data type=W4, Zero-shot=true2023.05 | 67.47 | — | — | |
| BaselineTarget LLM=LLaMA2-7B-CHAT2025.12 | 65.06 | — | — | |
| Linear ProbeTrain Dataset=ARC-E, Model=Qwen2.5-14b2026.03 | 62.44 | — | — | |
| ChannelModel Size=3B, Evaluation Protocol=Zero-shot2022.10 | 57.84 | — | — | |
| KiC-SmallParams=0.06B2022.10 | 57.7 | 1.7 | — | |
| OPTParams=30B2022.10 | 55.3 | 0.3 | — | |
| GPT-NeoParams=2.7B2022.10 | 54.6 | 1.4 | — | |
| GPT-NeoXParams=20B2022.10 | 53.5 | 1 | — | |
| GPT-JParams=6B2022.10 | 53.3 | 1.1 | — | |
| Linear ProbeTrain Dataset=OpenQA, Model=Qwen2.5-14b2026.03 | 52.86 | — | — | |
| BERTParams=0.34B2022.10 | 50.1 | 0.1 | — | |
| Random2025.10 | 50 | — | — | |
| RoBERTaParams=0.35B2022.10 | 48.5 | 0.4 | — | |
| Llama-Primus-Reasoning-8BShots=02026.04 | 38.45 | — | — |