Uncertainty estimation on CoQA (test)
77.3AUROCSAR
Evaluation Results
| Method | Links | |
|---|---|---|
| SARModel=OPT-13b, Rouge-L threshold=0.32023.07 | 77.3 | |
| SARModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 76.8 | |
| SARModel=OPT-30b, Rouge-L threshold=0.32023.07 | 76.7 | |
| TOKENSARModel=OPT-13b, Rouge-L threshold=0.32023.07 | 75.2 | |
| SEModel=OPT-13b, Rouge-L threshold=0.32023.07 | 75.1 | |
| SEModel=OPT-30b, Rouge-L threshold=0.32023.07 | 75.1 | |
| LN-PEModel=OPT-13b, Rouge-L threshold=0.32023.07 | 75 | |
| TOKENSARModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 74.6 | |
| TOKENSARModel=OPT-30b, Rouge-L threshold=0.32023.07 | 74.6 | |
| LN-PEModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 74.5 | |
| SARModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 74.2 | |
| LN-PEModel=OPT-30b, Rouge-L threshold=0.32023.07 | 74.2 | |
| SEModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 74.1 | |
| LN-PEModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 71.9 | |
| TOKENSARModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 71.9 | |
| SEModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 71.2 | |
| SENTSARModel=OPT-30b, Rouge-L threshold=0.32023.07 | 69.8 | |
| SENTSARModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 69.6 | |
| SENTSARModel=OPT-13b, Rouge-L threshold=0.32023.07 | 69 | |
| SENTSARModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 68.9 | |
| SARModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 68.6 | |
| LN-PEModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 67.3 | |
| SEModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 67.2 | |
| TOKENSARModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 67.2 | |
| PEModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 67.1 | |
| PEModel=OPT-30b, Rouge-L threshold=0.32023.07 | 67.1 | |
| PEModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 66.6 | |
| PEModel=OPT-13b, Rouge-L threshold=0.32023.07 | 66.6 | |
| SARModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 66.5 | |
| LN-PEModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 65.3 | |
| TOKENSARModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 65.3 | |
| SEModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 65.2 | |
| PEModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 64.6 | |
| SENTSARModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 63.5 | |
| PEModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 61.7 | |
| SENTSARModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 61 | |
| LSModel=OPT-6.7b, Rouge-L threshold=0.32023.07 | 58.8 | |
| LSModel=OPT-13b, Rouge-L threshold=0.32023.07 | 58.8 | |
| LSModel=OPT-2.7b, Rouge-L threshold=0.32023.07 | 57.3 | |
| LSModel=OPT-30b, Rouge-L threshold=0.32023.07 | 55 | |
| LSModel=LLaMA-13b, Rouge-L threshold=0.32023.07 | 52.2 | |
| LSModel=LLaMA-7b, Rouge-L threshold=0.32023.07 | 51.1 |