Calibration on UNK-QA 1 (val)
72.4AUCLikelihood
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LikelihoodModel=Flamingo 3B, accuracy=12.5%, triggering rate=48.0%2024.06 | 72.4 | 3.8 | 48.6 | 18.4 | 5.2 | |
| AVG BLEUModel=Flamingo 3B, accuracy=12.5%, triggering rate=48.0%2024.06 | 71.9 | 9.9 | 50.3 | 17.4 | 2.4 | |
| RepetitionsModel=Flamingo 3B, accuracy=12.5%, triggering rate=48.0%2024.06 | 69.1 | 24.8 | 32.3 | 3.5 | 0 | |
| DiversityModel=Flamingo 3B, accuracy=12.5%, triggering rate=48.0%2024.06 | 69 | 23.6 | 37.8 | 3.5 | 0 | |
| RepetitionsModel=LLaVA 13B, accuracy=20%, triggering rate=65%2024.06 | 61.4 | 51.3 | — | 21.3 | 0 | |
| AVG BLEUModel=LLaVA 13B, accuracy=20%, triggering rate=65%2024.06 | 61 | 27.3 | — | 26.4 | 1.3 | |
| DiversityModel=LLaVA 13B, accuracy=20%, triggering rate=65%2024.06 | 60.7 | 52.4 | — | 21.3 | 0 | |
| LikelihoodModel=LLaVA 13B, accuracy=20%, triggering rate=65%2024.06 | 60.1 | 49.5 | — | 22.1 | 1.3 |