Multi-class classification on PROMISE original (10-fold cross-validation)
100F1 Score (A)DeepSeek-V3
Evaluation Results
| Method | Links | ||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V3Model Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 100 | 94 | 100 | 100 | 92 | 94 | 93 | 91 | 89 | 90 | 91 | 91 | 91 | 83 | 100 | 91 | 86 | 86 | 86 | 100 | 93 | 96 | 89 | 0.89 | 89 | 95 | 89 | 92 | 93 | 0.81 | 87 | — | — | 92 | |
| PRC-RoBERTa-LModel Category=BERT variants, Evaluation Protocol=10-fold cross-validation2025.09 | 100 | 93 | — | — | — | — | 100 | — | — | 96 | — | — | 93 | — | — | 94 | — | — | 94 | — | — | 96 | — | — | 91 | — | — | 98 | — | — | 97 | — | — | 96 | |
| GPT-3.5-turboModel Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 94 | 89 | 91 | 95 | 100 | 60 | 75 | 83 | 95 | 89 | 86 | 79 | 83 | 71 | 59 | 65 | 83 | 73 | 78 | 70 | 74 | 72 | 76 | 0.62 | 68 | 90 | 71 | 80 | 92 | 69 | 79 | 82 | — | 82 | |
| Mistral-7B-ins.Model Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 93 | 90 | 91 | 95 | 100 | 60 | 75 | 83 | 95 | 89 | 86 | 79 | 82 | 71 | 59 | 65 | 83 | 75 | 78 | 70 | 74 | 72 | 76 | 0.62 | 68 | 90 | 71 | 80 | 92 | 69 | 79 | — | — | 81 | |
| PRC-BERT-BModel Category=BERT variants, Evaluation Protocol=10-fold cross-validation2025.09 | 93 | 91 | — | — | — | — | 74 | — | — | 68 | — | — | 90 | — | — | 86 | — | — | 93 | — | — | 91 | — | — | 87 | — | — | 98 | — | — | 94 | — | — | 91 | |
| NB (DP)Model Category=ML algorithms, Evaluation Protocol=10-fold cross-validation, Data Processing=Manual feature selection (DP)2025.09 | 90 | — | 90 | 90 | 90 | 97 | 93 | 100 | 75 | 86 | 94 | 94 | 94 | 82 | 90 | 86 | 91 | 78 | 84 | 100 | 90 | 95 | 83 | 0.83 | 83 | 100 | 97 | 98 | 77 | 0.97 | 86 | — | — | 90 | |
| LLaMA-3.1-8B-ins.Model Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 81 | 94 | 73 | 90 | 100 | 70 | 82 | 94 | 91 | 92 | 76 | 89 | 82 | 86 | 35 | 50 | 81 | 81 | 81 | 82 | 91 | 86 | 77 | 0.81 | 79 | 83 | 88 | 85 | 92 | 0.82 | 87 | — | — | 86 | |
| NoR-BERT-BModel Category=BERT variants, Evaluation Protocol=10-fold cross-validation2025.09 | 79 | 90 | 77 | 81 | 60 | 30 | 40 | 91 | 77 | 83 | 80 | 74 | 77 | 70 | 41 | 52 | 83 | 84 | 83 | 94 | 87 | 90 | 64 | 0.67 | 65 | 78 | 92 | 85 | 78 | 0.85 | 81 | — | — | 80 | |
| NoR-BERT-LModel Category=BERT variants, Evaluation Protocol=10-fold cross-validation2025.09 | 78 | 90 | 80 | 76 | 60 | 60 | 60 | 91 | 77 | 83 | 81 | 79 | 80 | 62 | 47 | 53 | 78 | 84 | 81 | 92 | 87 | 90 | 76 | 0.76 | 76 | 90 | 92 | 91 | 83 | 0.88 | 86 | — | — | 82 | |
| LDA (DP)Model Category=ML algorithms, Evaluation Protocol=10-fold cross-validation, Data Processing=Manual feature selection (DP)2025.09 | 74 | — | 60 | 95 | 2 | 10 | 3 | 20 | 47 | 28 | 85 | 60 | 70 | 52 | 70 | 60 | 70 | 35 | 47 | 95 | 70 | 81 | 57 | 0.81 | 70 | 87 | 87 | 87 | 76 | 0.61 | 68 | — | — | 62 | |
| GPT-4oModel Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 68 | 92 | 51 | 100 | 100 | 40 | 57 | 90 | 83 | 87 | 73 | 79 | 76 | 40 | 59 | 48 | 55 | 39 | 45 | 83 | 56 | 67 | 69 | 43 | 53 | 88 | 80 | 84 | 78 | 0.87 | 82 | 76 | — | 76 | |
| Gemma-3-4BModel Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 67 | 91 | 100 | 50 | 90 | 90 | 90 | 87 | 88 | 88 | 0 | 0 | 0 | 40 | 80 | 53 | 86 | 86 | 86 | 82 | 100 | 90 | 100 | 0.5 | 67 | 100 | 100 | 100 | 93 | 0.72 | 81 | — | — | 77 | |
| LLaMA-3.2-3B-ins.Model Category=Instruct-LLMs, Evaluation Protocol=10-fold cross-validation2025.09 | 55 | 64 | 50 | 62 | 50 | 10 | 17 | 91 | 83 | 85 | 82 | 61 | 70 | 0 | 0 | 0 | 71 | 51 | 61 | 83 | 37 | 51 | 100 | 0.38 | 55 | 76 | 77 | 77 | 90 | 0.69 | 78 | — | — | 71 |