List your company
ResearchFormerly WizWand

Question Answering on TruthfulQA (Primary Truthfulness Metric)

58.4TruthfulQAGRPO
16.90427.67738.4549.223Oct 1, 2025Nov 1, 2025Dec 2, 2025Jan 2, 2026Feb 2, 2026Mar 5, 2026Apr 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
GRPOBackbone=Qwen3-8B, Scenario=Strategic Deception2026.0358.4
CoT MonitorBackbone=Llama-3.1-8B, Scenario=Strategic Deception2026.0357.98
GRPOBackbone=Llama-3.1-8B, Scenario=Strategic Deception2026.0356.42
SARBackbone=Qwen3-8B, Scenario=Strategic Deception2026.0356.11
CoT MonitorBackbone=Qwen3-8B, Scenario=Strategic Deception2026.0355.77
SARBackbone=Qwen3-8B, Scenario=Sycophancy2026.0354.87
GRPOBackbone=Llama-3.1-8B, Scenario=Sycophancy2026.0354.74
CoT MonitorBackbone=Llama-3.1-8B, Scenario=Sycophancy2026.0354.56
Qwen3-8BScenario=Strategic Deception2026.0354.44
Qwen3-8BScenario=Sycophancy2026.0354.44
CoT MonitorBackbone=Qwen3-8B, Scenario=Sycophancy2026.0354.38
SARBackbone=Llama-3.1-8B, Scenario=Sycophancy2026.0354.23
GRPOBackbone=Qwen3-8B, Scenario=Sycophancy2026.0354.22
Llama-3.1-8BScenario=Strategic Deception2026.0354.05
Llama-3.1-8BScenario=Sycophancy2026.0354.05
SARBackbone=Llama-3.1-8B, Scenario=Strategic Deception2026.0352.42
Base modelRuntime (min)=N/A2026.0430.2
POMERuntime (min)=10.62026.0429.2
MuonRuntime (min)=12.02026.0429
SIFTRuntime (min)=26.42026.0429
AdamWRuntime (min)=8.12026.0428.9
BLURRuntime (min)=8.62026.0427.1
GradDiff w/ HybridAlgorithm=GradDiff, Optimizer=Hybrid2025.1024.1
NPO w/ HybridAlgorithm=NPO, Optimizer=Hybrid2025.1023.8
NPO w/ signAdamAlgorithm=NPO, Optimizer=signAdam2025.1023.6
NPO w/ SAMAlgorithm=NPO, Optimizer=SAM2025.1023.6
NPO w/ RSAlgorithm=NPO, Optimizer=RS2025.1023.4
NPOAlgorithm=NPO2025.1023.3
GradDiffAlgorithm=GradDiff2025.1022.4
NPO w/ signSGDAlgorithm=NPO, Optimizer=signSGD2025.1022.2
GradDiff w/ RSAlgorithm=GradDiff, Optimizer=RS2025.1022.2
GradDiff w/ signSGDAlgorithm=GradDiff, Optimizer=signSGD2025.1021.9
Pre-unlearn2025.1021.4
GradDiff w/ SAMAlgorithm=GradDiff, Optimizer=SAM2025.1021.4
GradDiff w/ signAdamAlgorithm=GradDiff, Optimizer=signAdam2025.1020.8
GradDiff w/ ZOAlgorithm=GradDiff, Optimizer=ZO2025.1020.3
NPO w/ ZOAlgorithm=NPO, Optimizer=ZO2025.1018.5