Factuality and Reasoning on GPT4All
0.6586HellaSwag AccuracyFullFT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| FullFTBackbone=Llama-2 13B, SFT=true2024.06 | 0.6586 | 0.404 | 0.7466 | 0.5222 | 0.8093 | 0.8373 | 0.79 | 0.6811 | |
| FullFTBackbone=Llama-2 7B, SFT=true2024.06 | 0.6279 | 0.4 | 0.7482 | 0.4932 | 0.7917 | 0.8401 | 0.778 | 0.6684 | |
| SATBackbone=Llama-2 13B, SFT=true2024.06 | 0.6229 | 0.416 | 0.7885 | 0.5657 | 0.8312 | 0.8682 | 0.8118 | 0.7006 | |
| (IA)³Backbone=Llama-2 13B, SFT=true2024.06 | 0.6162 | 0.366 | 0.7711 | 0.5316 | 0.8228 | 0.8511 | 0.7998 | 0.6798 | |
| OriginalBackbone=Llama-2 13B, SFT=false2024.06 | 0.6127 | 0.372 | 0.7711 | 0.5282 | 0.8203 | 0.837 | 0.7938 | 0.6764 | |
| SpIELBackbone=Llama-2 13B, SFT=true2024.06 | 0.6115 | 0.384 | 0.7845 | 0.5427 | 0.8295 | 0.8645 | 0.7998 | 0.6881 | |
| LoRABackbone=Llama-2 13B, SFT=true2024.06 | 0.6101 | 0.4 | 0.7861 | 0.5205 | 0.8207 | 0.8606 | 0.802 | 0.6857 | |
| SATBackbone=Llama-2 7B, SFT=true2024.06 | 0.5971 | 0.402 | 0.7601 | 0.5256 | 0.8152 | 0.8385 | 0.7971 | 0.6765 | |
| SpIELBackbone=Llama-2 7B, SFT=true2024.06 | 0.5874 | 0.374 | 0.753 | 0.523 | 0.8144 | 0.8437 | 0.7938 | 0.6699 | |
| (IA)³Backbone=Llama-2 7B, SFT=true2024.06 | 0.5846 | 0.364 | 0.7435 | 0.4846 | 0.7963 | 0.8052 | 0.7927 | 0.653 | |
| OriginalBackbone=Llama-2 7B, SFT=false2024.06 | 0.583 | 0.364 | 0.738 | 0.4923 | 0.7929 | 0.7936 | 0.7878 | 0.6502 | |
| LoRABackbone=Llama-2 7B, SFT=true2024.06 | 0.5806 | 0.378 | 0.753 | 0.5009 | 0.7988 | 0.8327 | 0.7943 | 0.6626 |