Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

In-Context Learning research benchmarks · SOTA2 Research

In-Context Learning

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

NLP Task Suite (Capitalize, Country-Capital, Present-Past, Singular-Plural, Person-Sport, AG News) (test)

99.9Capitalize

20

Feb 27, 2026

12 Downstream Classification Tasks

53Accuracy

15

Apr 8, 2026

9-dataset Average (SST-5, MNLI, CMSQA, HellaSwag, GeoQ, NL2Bash, Break, MTOP, SMCalFlow) (test)

68.08Accuracy

15

Feb 26, 2026

Garg ICL d=20 2022 (held-out tasks)

Transformer (continuous attn)

0.989R2

10

Jul 3, 2026

Garg ICL (d=10) (held-out tasks)

Transformer (continuous attn)

0.996R2

9

Jul 3, 2026

Fineweb-Edu 16.8B tokens

Spectra-AdEMAMix

36.86ARC-c Accuracy

8

Mar 17, 2026

58.4Accuracy

6

Apr 3, 2026

79.5Accuracy

6

Apr 3, 2026

Llama3-8B Scenario 5 ICL prompts

82.8Accuracy

3

Apr 13, 2026

Llama3-8B Scenario 4: More layers & Pos. (P={-5,...}, L={0,4,...})

46.38Accuracy

3

Apr 13, 2026

Llama3-8B Scenario 3: More layers (L={0,4,8,...})

80.43Accuracy

3

Apr 13, 2026

Llama3-8B Scenario 2: More Pos. (P={-5,...,-1})

78.18Accuracy

3

Apr 13, 2026

Llama3-8B Scenario 1: Diff. Pos. (P={4})

74.1Accuracy

3

Apr 13, 2026

Llama3-8B Baseline (P={-1}, L={14})

78.65Accuracy

3

Apr 13, 2026

MMLongBench iNat

22Accuracy

2

Jun 23, 2026

MMLongBench Food

34Accuracy

2

Jun 23, 2026

MMLongBench Cars

0.264Accuracy

2

Jun 23, 2026

Popular datasets

Llama3 MMLongBench Llama3-8B Llama Fineweb-edu

Follow for updates

Join SOTA2 Contact