Long-Paragraph Factuality on FACTOR (test)
78.12Factual AccuracySLED
Evaluation Results
| Method | Links | |
|---|---|---|
| SLEDBackbone=Gemma-3-27B-PT, Decoding=SLED2024.11 | 78.12 | |
| SLEDBackbone=Gemma-3-12B-PT, Decoding=SLED2024.11 | 74.25 | |
| SLEDBackbone=Gemma-3-27B-IT, Decoding=SLED2024.11 | 73.75 | |
| Gemma-3-27B-PT (Greedy)Backbone=Gemma-3-27B-PT, Decoding=Greedy2024.11 | 72.04 | |
| SLEDBackbone=Gemma-3-12B-IT, Decoding=SLED2024.11 | 70.81 | |
| SLEDBackbone=Gemma-3-4B-PT, Decoding=SLED2024.11 | 69.94 | |
| Gemma-3-12B-PT (Greedy)Backbone=Gemma-3-12B-PT, Decoding=Greedy2024.11 | 66.67 | |
| SLEDBackbone=Gemma-3-4B-IT, Decoding=SLED2024.11 | 66.15 | |
| Gemma-3-27B-IT (Greedy)Backbone=Gemma-3-27B-IT, Decoding=Greedy2024.11 | 66 | |
| SLEDBackbone=Gemma-3-1B-PT, Decoding=SLED2024.11 | 63.29 | |
| Gemma-3-12B-IT (Greedy)Backbone=Gemma-3-12B-IT, Decoding=Greedy2024.11 | 59.12 | |
| Gemma-3-4B-PT (Greedy)Backbone=Gemma-3-4B-PT, Decoding=Greedy2024.11 | 58.78 | |
| DoLaBackbone=Gemma-3-27B-PT, Decoding=DoLa2024.11 | 55.88 | |
| SLEDBackbone=Gemma-3-1B-IT, Decoding=SLED2024.11 | 55.04 | |
| Gemma-3-4B-IT (Greedy)Backbone=Gemma-3-4B-IT, Decoding=Greedy2024.11 | 50.17 | |
| Gemma-3-1B-PT (Greedy)Backbone=Gemma-3-1B-PT, Decoding=Greedy2024.11 | 47.83 | |
| DoLaBackbone=Gemma-3-27B-IT, Decoding=DoLa2024.11 | 47.43 | |
| Gemma-3-1B-IT (Greedy)Backbone=Gemma-3-1B-IT, Decoding=Greedy2024.11 | 37.17 | |
| DoLaBackbone=Gemma-3-1B-IT, Decoding=DoLa2024.11 | 16.5 | |
| DoLaBackbone=Gemma-3-1B-PT, Decoding=DoLa2024.11 | 12.63 | |
| DoLaBackbone=Gemma-3-12B-IT, Decoding=DoLa2024.11 | 11.06 | |
| DoLaBackbone=Gemma-3-4B-IT, Decoding=DoLa2024.11 | 10.79 | |
| DoLaBackbone=Gemma-3-4B-PT, Decoding=DoLa2024.11 | 9.25 | |
| DoLaBackbone=Gemma-3-12B-PT, Decoding=DoLa2024.11 | 7.28 |