Question Answering on WMDP Cyber QA benchmark leaderboard · SOTA2 Research

Question Answering on WMDP Cyber QA

44.3Default AccuracyBase model

Evaluation Results

Method	Links
Base model2025.01		44.3	—	—	—	—
Base model2025.01		44.3	—	—	—	—
RMU2025.01		29.1	27.7	38.3	28.5	40.5
RMU2025.01		29.1	—	39.5	28.6	33.4
RSV2025.01		28.9	27.9	41	31.7	39.8
RSV2025.01		28.9	—	34.5	28.8	30.6
Adaptive RMU2025.01		28.7	27.1	38.1	29.4	39.4
Adaptive RMU2025.01		28.7	—	38.6	28.4	34.2
RMU2025.01		28.6	28.6	39.8	30.4	40.6
RMU2025.01		28.6	—	39.7	29.9	35.5
RSV2025.01		28.2	27.5	40	31.8	39.5
RSV2025.01		28.2	—	35.9	29.3	31.7
Adaptive RMU2025.01		27.9	28.4	39.6	32.6	41.3
Adaptive RMU2025.01		27.9	—	41.4	32.5	35.4
SimNPO+KL2025.01		27.1	27.4	42.1	26.2	39.9
SimNPO+KL2025.01		27.1	—	43	26.3	37.4
NPO+MSE2025.01		27	27	42.9	27.6	35.7
DPO+KL2025.01		27	26.6	40.3	32.3	38.3
NPO+MSE2025.01		27	—	41.2	27	26.1
DPO+KL2025.01		27	—	37.3	26.9	37.7
SimNPO+MSE2025.01		26.7	26.8	41.3	27.3	37
SimNPO+MSE2025.01		26.7	—	41.9	26.6	27.5
SimNPO+MSE2025.01		26	27.4	42.8	27.5	37.3
SimNPO+MSE2025.01		26	—	41.8	26.1	36.8
NPO+KL2025.01		25.8	27.1	43.3	28.1	39.7
NPO+KL2025.01		25.8	—	42.5	27.2	39.9
SimNPO+KL2025.01		25.7	27.5	42.3	26	39.8
SimNPO+KL2025.01		25.7	—	42.4	28	41.4
NPO+KL2025.01		25.6	27.5	42.8	29.8	40.6
NPO+KL2025.01		25.6	—	42.5	27.2	40.5
NPO+MSE2025.01		25.4	27.5	42.5	25.3	36.3
NPO+MSE2025.01		25.4	—	41.3	27	29.2
DPO+MSE2025.01		25.2	28	36.4	27.4	36.1
DPO+MSE2025.01		25.2	—	25.3	27	26.2
DPO+KL2025.01		25.1	27.2	40.5	28.9	35.3
DPO+KL2025.01		25.1	—	38	28.4	35.3
DPO+MSE2025.01		23.9	26.6	40.3	26	37.4
DPO+MSE2025.01		23.9	—	36.9	26.4	32.4