Quantitative evaluation of LLM feedback against human gold standards on 50 SOC analysis reports (test) benchmark leaderboard · SOTA2 Research

Quantitative evaluation of LLM feedback against human gold standards on 50 SOC analysis reports (test)

0.66Spearman Correlation (ρ)MESSALA

Evaluation Results

Method	Links
MESSALA2026.01		0.66	0.53	0.66	0.89
Method 12026.01		0.65	0.54	0.64	1.24
Method 32026.01		0.65	0.51	0.63	0.91
MESSALA2026.01		0.64	0.53	0.64	0.88
Method 12026.01		0.63	0.53	0.61	1.27
MESSALA2026.01		0.6	0.49	0.59	1.03
Method 22026.01		0.6	0.5	0.57	1.07
Method 42026.01		0.6	0.5	0.6	0.97
MESSALA2026.01		0.59	0.46	0.59	1.13
Method 42026.01		0.58	0.45	0.58	1.04
MESSALA2026.01		0.58	0.46	0.56	1.02
Method 42026.01		0.57	0.44	0.56	0.93
Method 12026.01		0.56	0.44	0.55	1.27
Method 32026.01		0.56	0.43	0.54	1.16
Method 12026.01		0.56	0.44	0.55	1.01
Method 42026.01		0.55	0.45	0.53	1.01
Method 12026.01		0.54	0.46	0.54	1.16
Method 22026.01		0.54	0.42	0.55	1.23
Method 22026.01		0.52	0.4	0.51	0.97
Method 22026.01		0.51	0.42	0.51	1.05
Method 32026.01		0.5	0.39	0.49	1
Method 32026.01		0.49	0.4	0.49	1.27
Method 32026.01		0.49	0.4	0.49	1.3
Method 42026.01		0.46	0.36	0.44	1.34
Method 22026.01		0.41	0.33	0.39	1.56
MESSALA2026.01		0.38	0.3	0.35	1.06
Method 22026.01		0.36	0.3	0.36	1.12
Method 42026.01		0.3	0.23	0.28	1.17
Method 12026.01		0.28	0.23	0.28	1.52
Method 32026.01		0.22	0.17	0.23	1.34