# FR/NFR Classification on PROMISE original (10-fold cross val)

Source: [FR/NFR Classification on PROMISE original (10-fold cross val)](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T06:49:22.013Z

Evaluation of Functional Requirement (FR) and Non-Functional Requirement (NFR) classification performance on the PROMISE original dataset using 10-fold cross-validation.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | FR/NFR Classification |
| Dataset (including any stated split) | PROMISE original (10-fold cross val) |
| Metric | FR Precision |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [FR Precision](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=FR%20Precision>): Higher is better; unit: PERCENT.

- [FR Recall](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=FR%20Recall>): Higher is better; unit: PERCENT.

- [FR F1 Score](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=FR%20F1%20Score>): Higher is better; unit: PERCENT.

- [NFR Precision](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=NFR%20Precision>): Higher is better; unit: PERCENT.

- [NFR Recall](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=NFR%20Recall>): Higher is better; unit: PERCENT.

- [NFR F1 Score](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=NFR%20F1%20Score>): Higher is better; unit: PERCENT.

- [Overall F1 Score](<https://www.sota2.com/research/sota/fr-nfr-classification-on-promise-original-10-fold-cross-val.md?metric=Overall%20F1%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | FR Precision | Result date | Report paper |
| --- | --- | --- | --- |
| Gemma-3-4B | 94 | Not recorded | 68fa7b661e209131cea88313 |
| DeepSeek-V3 | 93 | Not recorded | 68fa7b661e209131cea88313 |
| LLaMA-3.1-8B-instruct | 93 | Not recorded | 68fa7b661e209131cea88313 |
| PRC-RoBERTa-large | 92 | Not recorded | 68fa7b661e209131cea88313 |
| NoR-BERT-large | 92 | Not recorded | 68fa7b661e209131cea88313 |
| SVM (hand-picked features) | 92 | Not recorded | 68fa7b661e209131cea88313 |
| PRC-BERT-base | 91 | Not recorded | 68fa7b661e209131cea88313 |
| C4.5 DT (processed data) | 90 | Not recorded | 68fa7b661e209131cea88313 |
| NoR-BERT-base | 89 | Not recorded | 68fa7b661e209131cea88313 |
| SVM (autom. features) | 88 | Not recorded | 68fa7b661e209131cea88313 |
| GPT-4o | 85 | Not recorded | 68fa7b661e209131cea88313 |
| GPT-3.5-turbo | 84 | Not recorded | 68fa7b661e209131cea88313 |
| Mistral-7B-instruct | 84 | Not recorded | 68fa7b661e209131cea88313 |
| C4.5 DT (unprocessed data) | 84 | Not recorded | 68fa7b661e209131cea88313 |
| LLaMA-3.2-3B-instruct | 48 | Not recorded | 68fa7b661e209131cea88313 |

### Gemma-3-4B

FR Precision: 94. Result ID: 6a211feefbc373592c1c093b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

### DeepSeek-V3

FR Precision: 93. Result ID: 6a211feefbc373592c1c093a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

### LLaMA-3.1-8B-instruct

FR Precision: 93. Result ID: 6a211feefbc373592c1c093d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

### PRC-RoBERTa-large

FR Precision: 92. Result ID: 6a211feefbc373592c1c093f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"BERT variants"}. This record does not assert independent reproduction.

### NoR-BERT-large

FR Precision: 92. Result ID: 6a211feefbc373592c1c0941.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"BERT variants"}. This record does not assert independent reproduction.

### SVM (hand-picked features)

FR Precision: 92. Result ID: 6a211feefbc373592c1c0944.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"ML algorithms","Feature Selection":"Manual"}. This record does not assert independent reproduction.

### PRC-BERT-base

FR Precision: 91. Result ID: 6a211feefbc373592c1c0940.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"BERT variants"}. This record does not assert independent reproduction.

### C4.5 DT (processed data)

FR Precision: 90. Result ID: 6a211feefbc373592c1c0946.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63831e209131ce9ff19b.md>)

Recorded evaluation configuration: {"Model Category":"ML algorithms","Data Processing":"Manual feature selection"}. This record does not assert independent reproduction.

### NoR-BERT-base

FR Precision: 89. Result ID: 6a211feefbc373592c1c0942.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"BERT variants"}. This record does not assert independent reproduction.

### SVM (autom. features)

FR Precision: 88. Result ID: 6a211feefbc373592c1c0943.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"ML algorithms","Feature Selection":"Automatic"}. This record does not assert independent reproduction.

### GPT-4o

FR Precision: 85. Result ID: 6a211feefbc373592c1c0938.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

### GPT-3.5-turbo

FR Precision: 84. Result ID: 6a211feefbc373592c1c0939.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

### Mistral-7B-instruct

FR Precision: 84. Result ID: 6a211feefbc373592c1c093c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

### C4.5 DT (unprocessed data)

FR Precision: 84. Result ID: 6a211feefbc373592c1c0945.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63831e209131ce9ff19b.md>)

Recorded evaluation configuration: {"Model Category":"ML algorithms","Data Processing":"Unprocessed"}. This record does not assert independent reproduction.

### LLaMA-3.2-3B-instruct

FR Precision: 48. Result ID: 6a211feefbc373592c1c093e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b661e209131cea88313.md>)

Recorded evaluation configuration: {"Model Category":"Instruct-LLMs"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
