List your company
ResearchFormerly WizWand

Long-Paragraph Factuality on FACTOR (test)

78.12Factual AccuracySLED
4.446423.573242.761.8268Nov 1, 2024
Updated 8mo ago

Evaluation results

MethodLinks
SLEDBackbone=Gemma-3-27B-PT, Decoding=SLED2024.1178.12
SLEDBackbone=Gemma-3-12B-PT, Decoding=SLED2024.1174.25
SLEDBackbone=Gemma-3-27B-IT, Decoding=SLED2024.1173.75
Gemma-3-27B-PT (Greedy)Backbone=Gemma-3-27B-PT, Decoding=Greedy2024.1172.04
SLEDBackbone=Gemma-3-12B-IT, Decoding=SLED2024.1170.81
SLEDBackbone=Gemma-3-4B-PT, Decoding=SLED2024.1169.94
Gemma-3-12B-PT (Greedy)Backbone=Gemma-3-12B-PT, Decoding=Greedy2024.1166.67
SLEDBackbone=Gemma-3-4B-IT, Decoding=SLED2024.1166.15
Gemma-3-27B-IT (Greedy)Backbone=Gemma-3-27B-IT, Decoding=Greedy2024.1166
SLEDBackbone=Gemma-3-1B-PT, Decoding=SLED2024.1163.29
Gemma-3-12B-IT (Greedy)Backbone=Gemma-3-12B-IT, Decoding=Greedy2024.1159.12
Gemma-3-4B-PT (Greedy)Backbone=Gemma-3-4B-PT, Decoding=Greedy2024.1158.78
DoLaBackbone=Gemma-3-27B-PT, Decoding=DoLa2024.1155.88
SLEDBackbone=Gemma-3-1B-IT, Decoding=SLED2024.1155.04
Gemma-3-4B-IT (Greedy)Backbone=Gemma-3-4B-IT, Decoding=Greedy2024.1150.17
Gemma-3-1B-PT (Greedy)Backbone=Gemma-3-1B-PT, Decoding=Greedy2024.1147.83
DoLaBackbone=Gemma-3-27B-IT, Decoding=DoLa2024.1147.43
Gemma-3-1B-IT (Greedy)Backbone=Gemma-3-1B-IT, Decoding=Greedy2024.1137.17
DoLaBackbone=Gemma-3-1B-IT, Decoding=DoLa2024.1116.5
DoLaBackbone=Gemma-3-1B-PT, Decoding=DoLa2024.1112.63
DoLaBackbone=Gemma-3-12B-IT, Decoding=DoLa2024.1111.06
DoLaBackbone=Gemma-3-4B-IT, Decoding=DoLa2024.1110.79
DoLaBackbone=Gemma-3-4B-PT, Decoding=DoLa2024.119.25
DoLaBackbone=Gemma-3-12B-PT, Decoding=DoLa2024.117.28