BEAM
Benchmarks
TaskDatasetSOTA resultTrendResults
BEAM 100K tokens
5Abstention Rate
18
BEAM 10M tokens
5Abstention Rate
12
BEAM 1M tokens
22.1Abstention
12
BEAM 500K tokens
82.8Abstention Rate
12
BEAM 100K (test)
59Pass@0.5 Rate
10
BEAM 1M (test)
52.57Pass@0.5
9
BEAM 500K (test)
57Pass@0.5
9