Image Captioning on COCO Captioning (32-shot Delta)
81.7CIDErFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Backbone=OpenFlamingo-9B, Precision=FP16, Shots=32-shot2023.06 | 81.7 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=32-shot2023.06 | 80.53 | -1.17 | |
| FP16Backbone=OpenFlamingo-9B, Precision=FP16, Shots=16-shot2023.06 | 79.74 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=16-shot2023.06 | 78.23 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=32-shot2023.06 | 77.13 | -4.57 | |
| FP16Backbone=OpenFlamingo-9B, Precision=FP16, Shots=8-shot2023.06 | 76.95 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=16-shot2023.06 | 74.98 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=32-shot2023.06 | 74.98 | -6.72 | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=8-shot2023.06 | 74.75 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=32-shot2023.06 | 74.47 | -7.23 | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=16-shot2023.06 | 74.09 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=16-shot2023.06 | 72.86 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=8-shot2023.06 | 72.53 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=8-shot2023.06 | 72.46 | — | |
| FP16Backbone=OpenFlamingo-9B, Precision=FP16, Shots=4-shot2023.06 | 72.18 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=4-shot2023.06 | 71.02 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=8-shot2023.06 | 68.79 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=4-shot2023.06 | 68.07 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=4-shot2023.06 | 67.68 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=32-shot2023.06 | 64.79 | -16.91 | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=32-shot2023.06 | 64.77 | -16.93 | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=4-shot2023.06 | 64.73 | — | |
| FP16Backbone=OpenFlamingo-9B, Precision=FP16, Shots=0-shot2023.06 | 63.73 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=16-shot2023.06 | 63.21 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=0-shot2023.06 | 62.57 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=16-shot2023.06 | 60.54 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=8-shot2023.06 | 60.46 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=0-shot2023.06 | 60.24 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT4, Group size=128, Shots=0-shot2023.06 | 59.72 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=8-shot2023.06 | 56.55 | — | |
| AWQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=0-shot2023.06 | 56.33 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=4-shot2023.06 | 55.13 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=4-shot2023.06 | 50.77 | — | |
| RTNBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=0-shot2023.06 | 46.07 | — | |
| GPTQBackbone=OpenFlamingo-9B, Precision=INT3, Group size=128, Shots=0-shot2023.06 | 29.84 | — |