Psychometric Predictive Power on SAP ROI benchmark leaderboard · SOTA2 Research

Psychometric Predictive Power on SAP ROI

14.1MVRRsurprisal update

Evaluation Results

Method	Links
surprisal update2026.04		14.1	13.8	24.5	1.7	0.8
surprisal update2026.04		13.5	11.9	20.1	1.9	1.1
surprisal update2026.04		11.1	6.6	17.8	2.6	1.8
surprisal update2026.04		11	7	20	1.7	1.7
surprisal update2026.04		10.4	9.8	10.8	0.3	1.8
surprisal update2026.04		9.1	6.6	11.1	0.3	2.2
surprisal update2026.04		8.8	11.9	14.8	0.8	0.9
surprisal update2026.04		8.7	6.9	26.3	1.5	1.9
surprisal update2026.04		8.7	8.5	6.2	0.7	1.7
surprisal update2026.04		8.6	10.8	15	0.6	1.5
surprisal update2026.04		7.8	8.2	20.9	2.1	1.9
surprisal update2026.04		7.7	5.1	15.3	1.8	1.1
surprisal update2026.04		7.1	8.3	19	1.3	1.8
surprisal update2026.04		7	10.3	27	1.8	2.2
OT-30b2026.04		5.9	7.4	7.1	0.6	3.3
surprisal update2026.04		5.5	7.3	28.1	1.6	2.1
OT-13b2026.04		4.6	2.8	8.2	0.7	2.9
surprisal update2026.04		3.7	7.3	13.8	0.3	1.2
PT-1b2026.04		3.2	1.9	19.6	2.1	11.7
surprisal update2026.04		3.1	3.7	10.5	2	1.4
OT-6.7b2026.04		2.9	4	8.9	0.7	2.4
PT-410m2026.04		2.9	1.7	2.4	1.2	9.2
surprisal update2026.04		2.6	1.5	8.7	0.3	0.6
G2-sm2026.04		2.5	0.7	9.7	0.6	2.6
G2-lg2026.04		2.2	0.8	16	0.1	7.4
G2-xl2026.04		2.2	0.3	15.5	0.4	4.3
G2-md2026.04		2	5.1	15	0.2	4.9
PT-70m2026.04		2	1.4	0.8	3.3	7
OT-2.7b2026.04		1.9	1.2	10.8	0.6	3.7
OT-66b2026.04		1.7	1.2	12.1	1.2	3.5
surprisal update2026.04		1.6	1.6	0.9	0.7	4.2
OT-1.3b2026.04		1.6	1.4	13.3	0.5	4
PT-2.8b2026.04		0.7	3.3	12.7	3.7	5
PT-160m2026.04		0.5	2.6	1.8	3.8	9.7
PT-1.4b2026.04		0.3	1.9	5.9	1.8	6.2
PT-12b2026.04		0.3	1.3	8	2.3	5.5
OT-125m2026.04		0.2	2.1	8.5	2.4	6
PT-6.9b2026.04		0.2	2.2	8.8	2.2	6.6