Patient-level agreement on Closed-Weight (CW) benchmark cohort
0.831Quadratic Weighted Kappa (kw)Assessors (A1, A2) vs consolidated output
Evaluation results
| Method | Links | |
|---|---|---|
| Assessors (A1, A2) vs consolidated outputContext=CW internal pipeline agreement, N=2002026.07 | 0.831 | |
| CW MOSAICComparison Reference=OW MOSAIC, N=2002026.07 | 0.773 | |
| CW MOSAICComparison Reference=Cooper GT, N=2002026.07 | 0.702 | |
| CW MOSAICComparison Reference=Young GT, N=2002026.07 | 0.671 | |
| GPT-4o vs DeepSeekContext=CW inter-assessor, N=2002026.07 | 0.621 |