FR/NFR Classification on PROMISE original (10-fold cross val)
94FR PrecisionGemma-3-4B
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma-3-4BModel Category=Instruct-LLMs2025.09 | 94 | 88 | 91 | 92 | 96 | 94 | 93 | |
| DeepSeek-V3Model Category=Instruct-LLMs2025.09 | 93 | 94 | 94 | 96 | 95 | 95 | 95 | |
| LLaMA-3.1-8B-instructModel Category=Instruct-LLMs2025.09 | 93 | 94 | 94 | 95 | 96 | 96 | 95 | |
| PRC-RoBERTa-largeModel Category=BERT variants2025.09 | 92 | 94 | 93 | 95 | 96 | 95 | 94 | |
| NoR-BERT-largeModel Category=BERT variants2025.09 | 92 | 88 | 90 | 92 | 95 | 93 | 92 | |
| SVM (hand-picked features)Model Category=ML algorithms, Feature Selection=Manual2025.09 | 92 | 93 | 93 | 93 | 92 | 92 | 92 | |
| PRC-BERT-baseModel Category=BERT variants2025.09 | 91 | 92 | 91 | 91 | 96 | 93 | 92 | |
| C4.5 DT (processed data)Model Category=ML algorithms, Data Processing=Manual feature selection2025.09 | 90 | 97 | 93 | 98 | 93 | 95 | 94 | |
| NoR-BERT-baseModel Category=BERT variants2025.09 | 89 | 88 | 90 | 92 | 95 | 93 | 92 | |
| SVM (autom. features)Model Category=ML algorithms, Feature Selection=Automatic2025.09 | 88 | 87 | 87 | 87 | 88 | 87 | 87 | |
| GPT-4oModel Category=Instruct-LLMs2025.09 | 85 | 99 | 92 | 99 | 88 | 93 | 93 | |
| GPT-3.5-turboModel Category=Instruct-LLMs2025.09 | 84 | 95 | 89 | 96 | 87 | 91 | 91 | |
| Mistral-7B-instructModel Category=Instruct-LLMs2025.09 | 84 | 98 | 90 | 98 | 87 | 92 | 91 | |
| C4.5 DT (unprocessed data)Model Category=ML algorithms, Data Processing=Unprocessed2025.09 | 84 | 93 | 88 | 95 | 88 | 91 | 90 | |
| LLaMA-3.2-3B-instructModel Category=Instruct-LLMs2025.09 | 48 | 98 | 64 | 93 | 24 | 38 | 48 |