Multiple-choice task on COSE_KRE
60.23AccuracyOpin
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| OpinBackbone=LLaMA-3-8B-Instruct2024.06 | 60.23 | 30.11 | |
| IRCAN-CADBackbone=LLaMA-2-13B-Chat2024.06 | 57.77 | 22.09 | |
| CADBackbone=LLaMA-2-13B-Chat2024.06 | 56.96 | 22.59 | |
| CADBackbone=LLaMA-3-8B-Instruct2024.06 | 52.86 | 35.52 | |
| Based_on_FormattedBackbone=LLaMA-3-8B-Instruct2024.06 | 51.55 | 37.81 | |
| IRCAN-CADBackbone=LLaMA-3-8B-Instruct2024.06 | 51.55 | 31.42 | |
| Based_onBackbone=LLaMA-2-13B-Chat2024.06 | 50.57 | 29.46 | |
| ITI (Probe Weight Direction)Backbone=LLaMA-2-13B-Chat2024.06 | 50.41 | 25.37 | |
| Utilizing_FormattedBackbone=LLaMA-3-8B-Instruct2024.06 | 50.08 | 40.1 | |
| OriginalBackbone=LLaMA-2-13B-Chat2024.06 | 49.75 | 29.13 | |
| IRCANBackbone=LLaMA-2-13B-Chat2024.06 | 49.26 | 30.11 | |
| IRCAN-CADBackbone=LLaMA-2-7B-Chat2024.06 | 48.61 | 19.48 | |
| IRCANBackbone=LLaMA-2-7B-Chat2024.06 | 45.01 | 24.88 | |
| OpinBackbone=LLaMA-2-13B-Chat2024.06 | 43.21 | 22.91 | |
| Based_onBackbone=LLaMA-3-8B-Instruct2024.06 | 42.88 | 45.34 | |
| IRCANBackbone=LLaMA-3-8B-Instruct2024.06 | 42.72 | 37.64 | |
| IRCAN-CADBackbone=Gemma-2B-it2024.06 | 41.9 | 17.35 | |
| CADBackbone=LLaMA-2-7B-Chat2024.06 | 41.57 | 19.8 | |
| Utilizing_FormattedBackbone=LLaMA-2-13B-Chat2024.06 | 41.57 | 21.93 | |
| Based_on_FormattedBackbone=LLaMA-2-13B-Chat2024.06 | 41.24 | 23.57 | |
| ITI (Probe Weight Direction)Backbone=LLaMA-3-8B-Instruct2024.06 | 40.75 | 45.01 | |
| OriginalBackbone=LLaMA-3-8B-Instruct2024.06 | 39.93 | 47.79 | |
| IRCANBackbone=Gemma-2B-it2024.06 | 39.12 | 18.99 | |
| Utilizing_FormattedBackbone=Gemma-2B-it2024.06 | 38.95 | 22.26 | |
| ITI (Mass Mean Shift)Backbone=LLaMA-3-8B-Instruct2024.06 | 38.95 | 43.04 | |
| Based_on_FormattedBackbone=Gemma-2B-it2024.06 | 38.46 | 22.42 | |
| CADBackbone=Gemma-2B-it2024.06 | 37.97 | 19.64 | |
| ITI (Probe Weight Direction)Backbone=LLaMA-2-7B-Chat2024.06 | 37.32 | 17.51 | |
| OriginalBackbone=LLaMA-2-7B-Chat2024.06 | 36.66 | 23.4 | |
| OpinBackbone=Gemma-2B-it2024.06 | 35.19 | 19.97 | |
| OpinBackbone=LLaMA-2-7B-Chat2024.06 | 35.19 | 17.35 | |
| OriginalBackbone=Gemma-2B-it2024.06 | 35.02 | 21.28 | |
| Based_onBackbone=Gemma-2B-it2024.06 | 34.7 | 22.42 | |
| Based_onBackbone=LLaMA-2-7B-Chat2024.06 | 33.22 | 20.29 | |
| Utilizing_FormattedBackbone=LLaMA-2-7B-Chat2024.06 | 33.06 | 18 | |
| Based_on_FormattedBackbone=LLaMA-2-7B-Chat2024.06 | 32.41 | 18.49 | |
| ITI (Probe Weight Direction)Backbone=Gemma-2B-it2024.06 | 31.59 | 23.57 | |
| ITI (Mass Mean Shift)Backbone=Gemma-2B-it2024.06 | 29.46 | 23.73 | |
| ITI (Mass Mean Shift)Backbone=LLaMA-2-7B-Chat2024.06 | 26.35 | 18.66 | |
| ITI (Mass Mean Shift)Backbone=LLaMA-2-13B-Chat2024.06 | 25.2 | 19.15 |