Stance Detection on SEM 16
89.56HCLlama
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LlamaCategory=SMART-D (Ours)2026.06 | 89.56 | — | 75.4 | 73.29 | 84.74 | 91.52 | 82.9 | |
| MistralCategory=SMART-D (Ours)2026.06 | 89.33 | — | 78.44 | 77.67 | 75.21 | 83.13 | 80.76 | |
| LlamaCategory=Baseline (Single-Model)2026.06 | 89.19 | — | 77.03 | 70.43 | 77.3 | 87.22 | 80.23 | |
| GeminiCategory=Baseline (Single-Model)2026.06 | 89.18 | — | 79.55 | 69.46 | 76.71 | 85.58 | 80.1 | |
| MistralCategory=Baseline (Single-Model)2026.06 | 88.09 | — | 80.4 | 76.46 | 69.17 | 71.77 | 77.18 | |
| GeminiCategory=SMART-D (Ours)2026.06 | 86.5 | — | 81 | 71.13 | 87.75 | 88.16 | 82.77 | |
| KIRP*Model Category=Our_Model, Evaluation Setting=0-shot2026.06 | 85.06 | 82.28 | 83.17 | 84.21 | 85.5 | 81.74 | 84.05 | |
| DeepSeek-V4 ToC*Model Category=LLM-based, Evaluation Setting=0-shot2026.06 | 83.28 | 83.27 | 82.73 | 84.57 | 84.39 | 80.03 | 83.91 | |
| MSMEModel=GPT-4o, Category=MSME (ours)2025.12 | 81.9 | — | 75.5 | 71.9 | 80.3 | 76.2 | 77.2 | |
| GPT-4oCategory=MSME2026.06 | 81.9 | — | 75.5 | 71.9 | 80.3 | 76.2 | 77.2 | |
| COLAZero-shot=true2023.10 | 81.7 | 68.5 | 63.4 | 71 | 70.8 | 65.5 | — | |
| COLACategory=Zero-shot Method2023.10 | 81.7 | 68.5 | 63.4 | 71 | 70.8 | 67.5 | — | |
| COLACategory=Zero-shot based on LLMs2025.12 | 81.7 | — | 63.4 | 71 | 70.8 | 65.5 | 70.5 | |
| COLACategory=Zero-shot (LLMs)2026.06 | 81.7 | — | 63.4 | 71 | 70.8 | 65.5 | 70.5 | |
| DeepSeek-V4 CoT*Model Category=LLM-based, Evaluation Setting=0-shot2026.06 | 81.22 | 80.32 | 79.58 | 84.89 | 82.2 | 78.58 | 81.74 | |
| MSMEModel=GPT-3.5, Category=MSME (ours)2025.12 | 81.1 | — | 72.5 | 69.9 | 75.2 | 74.9 | 74.7 | |
| GPT-3.5Category=MSME2026.06 | 81.1 | — | 72.5 | 69.9 | 75.2 | 74.9 | 74.7 | |
| MSMEModel=DeepSeek-r1, Category=MSME (ours)2025.12 | 80.6 | — | 74.8 | 73.5 | 81.5 | 78.5 | 77.8 | |
| DeepSeek-r1Category=MSME2026.06 | 80.6 | — | 74.8 | 73.5 | 81.5 | 78.5 | 77.8 | |
| GPT-3.5-ToC*Model Category=LLM-based, Evaluation Setting=0-shot2026.06 | 80.26 | 76.58 | 81.44 | 78.25 | 77.44 | 75.26 | 79.89 | |
| MSMEModel=QwQ-32b, Category=MSME (ours)2025.12 | 76.9 | — | 76.3 | 68.4 | 79.5 | 77.1 | 75.6 | |
| QwQ-32BCategory=MSME2026.06 | 76.9 | — | 76.3 | 68.4 | 79.5 | 77.1 | 75.6 | |
| BKELCategory=Zero-shot based on LLMs2025.12 | 76.5 | — | 63.1 | 64.2 | 71.5 | 66 | 68.3 | |
| BKELCategory=Zero-shot (LLMs)2026.06 | 76.5 | — | 63.1 | 64.2 | 71.5 | 66 | 68.3 | |
| GPT-3.5-CoT*Model Category=LLM-based, Evaluation Setting=0-shot2026.06 | 75.08 | 74.32 | 69.89 | 75.24 | 77.4 | 73.55 | 72.07 | |
| TPDGCategory=In-target Labeled Data Dependent Methods2023.10 | 73.4 | 63 | 67.3 | 74.7 | 64.7 | 42.3 | — | |
| TPDGCategory=In-target2025.12 | 73.4 | — | 67.3 | 74.7 | 64.7 | 42.3 | 64.5 | |
| GPT-3.5Zero-shot=true, Chain-of-Thought=true2023.10 | 70.9 | 63.3 | 47.7 | 53.4 | 13.3 | 34 | — | |
| CoTCategory=Zero-shot based on LLMs2025.12 | 70.7 | — | 62.4 | 61.9 | 64.1 | 55.7 | 63 | |
| CoTCategory=Zero-shot (LLMs)2026.06 | 70.7 | — | 62.4 | 61.9 | 64.1 | 55.7 | 63.2 | |
| GPT-3.5Zero-shot=true2023.10 | 68.7 | 62.5 | 44.7 | 51.5 | 9.1 | 31.1 | — | |
| Stance ReasonerCategory=Zero-shot based on LLMs2025.12 | 67.7 | — | 73.9 | 60.3 | 69.7 | 62.5 | 66.8 | |
| Stance ReasonerCategory=Zero-shot (LLMs)2026.06 | 67.7 | — | 73.9 | 60.3 | 69.7 | 62.5 | 66.8 | |
| BaseCategory=Zero-shot based on LLMs2025.12 | 65 | — | 62.3 | 60.8 | 58.3 | 51.1 | 59.5 | |
| Base (GPT-3.5)Category=Zero-shot (LLMs)2026.06 | 65 | — | 62.3 | 60.8 | 58.3 | 51.1 | 59.5 | |
| DS-ESDModel Category=LLM-based, Evaluation Setting=100% training data2026.06 | 63.82 | 65.43 | 69.9 | 71.25 | 69.45 | 66.68 | 68.55 | |
| KASD-GPT*Model Category=LLM-based, Evaluation Setting=0-shot2026.06 | 61.57 | 64.25 | 66.37 | 67.76 | 63.88 | 68.45 | 65.77 | |
| BERTCategory=In-target Labeled Data Dependent Methods2023.10 | 61.3 | 57.9 | 59 | 63.1 | 60.7 | 38.8 | — | |
| BERTCategory=In-target2025.12 | 61.3 | — | 59 | 63.1 | 60.7 | 38.8 | 56.6 | |
| ASGCNCategory=In-target Labeled Data Dependent Methods2023.10 | 61 | 58.7 | 58.7 | 63.2 | 59.5 | 40.6 | — | |
| ASGCNCategory=In-target2025.12 | 61 | — | 58.7 | 63.2 | 59.5 | 40.6 | 56.6 | |
| CrossNetCategory=In-target Labeled Data Dependent Methods2023.10 | 60.2 | 60.2 | 55.7 | 61.3 | 56.4 | 40.1 | — | |
| CrossNetCategory=In-target2025.12 | 60.2 | — | 55.7 | 61.3 | 56.4 | 40.1 | 54.7 | |
| TGA NetCategory=Zero-shot2025.12 | 60.2 | — | 61.2 | 55.7 | 56.1 | 52.9 | 57.2 | |
| ATT-LSTMCategory=In-target Labeled Data Dependent Methods2023.10 | 59.8 | 55.3 | 55.3 | 62.6 | 55.9 | 39.2 | — | |
| BiCondCategory=In-target Labeled Data Dependent Methods2023.10 | 56.1 | 59 | 52.9 | 61.2 | 55.3 | 35.6 | — | |
| JointCLZero-shot=true2023.10 | 54.8 | 50.5 | 53.8 | 49.5 | 54.5 | 39.7 | — | |
| JointCLCategory=Zero-shot2025.12 | 54.8 | — | 53.8 | 49.5 | 54.5 | 39.7 | 50.5 | |
| ANEK*Model Category=BERT-based, Evaluation Setting=0-shot2026.06 | 54.7 | 50.26 | 55.21 | 48.77 | 54.14 | 39.2 | 47.67 | |
| JointCL*Model Category=CL-based, Evaluation Setting=0-shot2026.06 | 54.67 | 50.5 | 53.28 | 48.98 | 54.45 | 40.7 | 50.78 | |
| PT-HCLZero-shot=true2023.10 | 54.5 | 50.1 | 54.6 | 50.9 | 56.5 | 38.9 | — | |
| PT-HCL*Model Category=CL-based, Evaluation Setting=0-shot2026.06 | 54.35 | 49.98 | 54.6 | 48.79 | 54.78 | 51.24 | 53.21 | |
| MTFF*Model Category=CL-based, Evaluation Setting=0-shot2026.06 | 51.26 | 48.32 | 49.45 | 53.08 | 46.84 | 52.14 | 49.28 | |
| TOADZero-shot=true2023.10 | 51.2 | 49.5 | 54.1 | 46.2 | 46.1 | 30.9 | — | |
| TOADCategory=Zero-shot2025.12 | 51.2 | — | 54.1 | 46.2 | 46.1 | 30.9 | 45.7 | |
| RoBERTa-EnrichedModel Category=BERT-based, Evaluation Setting=100% training data2026.06 | 50.87 | 47.24 | 48.93 | 56.25 | 55.24 | 49.68 | 51.57 | |
| TGA-Net-ft*Model Category=BERT-based, Evaluation Setting=0-shot2026.06 | 50.24 | 41.72 | 45.78 | 47.83 | 52.74 | 37.25 | 44.48 | |
| TGA-Net*Model Category=BERT-based, Evaluation Setting=0-shot2026.06 | 50.21 | 40.68 | 46.64 | 45.17 | 52.67 | 36.66 | 42.78 | |
| VTNModel Category=EK-based, Evaluation Setting=100% training data2026.06 | 50.1 | 49.25 | 47.94 | 51.88 | 49.35 | 47.67 | 49.24 | |
| BERT-GCNZero-shot=true2023.10 | 50 | 42.3 | 44.3 | 44.2 | 53.6 | 35.5 | — | |
| BERT-GCNCategory=Zero-shot2025.12 | 50 | — | 44.3 | 44.2 | 53.6 | 35.5 | 45.5 | |
| DTCL*Model Category=CL-based, Evaluation Setting=0-shot2026.06 | 49.8 | 50.15 | 51.48 | 50.03 | 49.07 | 50.52 | 50.14 | |
| MT-DNNModel Category=EK-based, Evaluation Setting=100% training data2026.06 | 49.58 | 45.4 | 53.45 | 52.38 | 56.35 | 55.03 | 53.17 | |
| TGA NetZero-shot=true2023.10 | 49.3 | 40.7 | 46.6 | 45.2 | 52.7 | 36.6 | — | |
| STCCModel Category=CL-based, Evaluation Setting=100% training data2026.06 | 48.9 | 46.24 | 51.25 | 47.93 | 52.25 | 49.01 | 49.57 | |
| CKE-Net*Model Category=EK-based, Evaluation Setting=0-shot2026.06 | 47.24 | 46.79 | 50 | 49.28 | 51.15 | 51.04 | 49.27 | |
| BERTweetModel Category=BERT-based, Evaluation Setting=100% training data2026.06 | 46.37 | 52.75 | 48.09 | 50.01 | 45.57 | 47.87 | 47.26 | |
| TANModel Category=BiLSTM-based, Evaluation Setting=100% training data2026.06 | 43.67 | 40.01 | 39.91 | 41.1 | 42.28 | 28.74 | 37.65 | |
| CrossNetModel Category=BiLSTM-based, Evaluation Setting=100% training data2026.06 | 40.25 | 35.58 | 41.7 | 37.77 | 39.65 | 22.75 | 36.89 | |
| BiCondModel Category=BiLSTM-based, Evaluation Setting=100% training data2026.06 | 33.54 | 30.47 | 38.23 | 34.42 | 31.11 | 16.83 | 29.74 | |
| ToCCategory=Zero-shot based on LLMs2025.12 | — | — | — | — | — | — | 69.4 | |
| ToCCategory=Zero-shot (LLMs)2026.06 | — | — | — | — | — | — | 69.4 |