Counterfact
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Counterfact (test)
99.86RwA
12
CounterFact
60Top-1 Accuracy
12
CounterFact
98.1Efficacy
10
CounterFact full (10K sequential edits) (test)
94.45Efficacy
10
CounterFact
100Efficacy
10
CounterFact T = 10000
86Efficacy
9
CounterFact (test)
54Top-1 Accuracy
9
COUNTERFACT 7,500-record GPT-2 XL (test)
89.2Score
9
COUNTERFACT
99.75Efficacy
8
CounterFact
64.6Rel (QA Context)
8
counterfact
0.84AUROC
8
Counterfact (val)
1Relational Score
8
Counterfact (first 150 edits)
98.67DI Score
8
CounterFact LLaMA2-7B
100Accuracy (Acc)
6
CounterFact GPT2-XL
100Accuracy
6
CounterFact 15000 (test)
91.22Efficacy
6
CounterFact
100Attack Performance
6
COUNTERFACT
88.2S
6
CounterFact (indices 0-5000)
99.08SEKA
5
CounterFact (test)
96.2Reliability
5
CounterFact (150 random samples)
100Efficacy Score
5
COUNTERFACT 2,000-record GPT-J (test)
91.5Score (S)
5
CounterFact
99.59Similarity
5
CounterFact
90.69BLEU
4
CounterFact
92.6Precision
4