# Article Generation on FreshWiki 2024-08-01 (test)

Source: [Article Generation on FreshWiki 2024-08-01 (test)](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:47:16.762Z

Evaluation of article generation quality on the specific test split of FreshWiki (dated 2024-08-01), utilizing a comprehensive set of metrics including subjective quality ratings (Interest, Organization, Relevance), factual grounding (Claims, Entity Recall), and standard overlap scores (ROUGE).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Article Generation |
| Dataset (including any stated split) | FreshWiki 2024-08-01 (test) |
| Metric | Interest |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Interest](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Interest>): Higher is better; unit: SCALAR.

- [Organization](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Organization>): Higher is better; unit: SCALAR.

- [Relevance](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Relevance>): Higher is better; unit: SCALAR.

- [Coverage](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Coverage>): Higher is better; unit: SCALAR.

- [ROUGE-1](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=ROUGE-1>): Higher is better; unit: PERCENT.

- [ROUGE-L](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=ROUGE-L>): Higher is better; unit: PERCENT.

- [Entity Recall](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Entity%20Recall>): Higher is better; unit: PERCENT.

- [Precision](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Precision>): Higher is better; unit: PERCENT.

- [Claims](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Claims>): Higher is better; unit: SCALAR.

- [F1@300](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=F1%40300>): Higher is better; unit: PERCENT.

- [Diversity](<https://www.sota2.com/research/sota/article-generation-on-freshwiki-2024-08-01-test.md?metric=Diversity>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Interest | Result date | Report paper |
| --- | --- | --- | --- |
| RAPID | 4.38 | Not recorded | 68fa77371e209131cea71ad7 |
| RAPID | 4.37 | Not recorded | 68fa77371e209131cea71ad7 |
| RAPID (w/o plan) | 4.35 | Not recorded | 68fa77371e209131cea71ad7 |
| Co-STORM | 4.31 | Not recorded | 68fa77371e209131cea71ad7 |
| STORM | 4.23 | Not recorded | 68fa77371e209131cea71ad7 |
| RAG | 4.18 | Not recorded | 68fa77371e209131cea71ad7 |
| STORM | 4.16 | Not recorded | 68fa77371e209131cea71ad7 |
| RAG | 4.03 | Not recorded | 68fa77371e209131cea71ad7 |
| RAPID | 3.96 | Not recorded | 68fa77371e209131cea71ad7 |
| Co-STORM | 3.92 | Not recorded | 68fa77371e209131cea71ad7 |
| STORM | 3.9 | Not recorded | 68fa77371e209131cea71ad7 |
| Co-STORM | 3.88 | Not recorded | 68fa77371e209131cea71ad7 |
| oRAG | 3.68 | Not recorded | 68fa77371e209131cea71ad7 |
| RAG | 3.66 | Not recorded | 68fa77371e209131cea71ad7 |
| oRAG | 3.46 | Not recorded | 68fa77371e209131cea71ad7 |
| oRAG | 3.32 | Not recorded | 68fa77371e209131cea71ad7 |

### RAPID

Interest: 4.38. Result ID: 699477743a6c3148491d7110.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Implementation](<https://github.com/stanfordnlp/dspy>)

Recorded evaluation configuration: {"Foundation Model":"GPT-4o"}. This record does not assert independent reproduction.

### RAPID

Interest: 4.37. Result ID: 699477743a6c3148491d710b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Implementation](<https://github.com/stanfordnlp/dspy>)

Recorded evaluation configuration: {"Foundation Model":"DeepSeek-v3"}. This record does not assert independent reproduction.

### RAPID (w/o plan)

Interest: 4.35. Result ID: 699477743a6c3148491d7111.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Implementation](<https://github.com/stanfordnlp/dspy>)

Recorded evaluation configuration: {"Foundation Model":"GPT-4o","Ablation":"without plan"}. This record does not assert independent reproduction.

### Co-STORM

Interest: 4.31. Result ID: 699477743a6c3148491d710f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa753c1e209131cea66a77.md>)

Recorded evaluation configuration: {"Foundation Model":"GPT-4o"}. This record does not assert independent reproduction.

### STORM

Interest: 4.23. Result ID: 699477743a6c3148491d7109.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72cb1e209131cea58790.md>)

Recorded evaluation configuration: {"Foundation Model":"DeepSeek-v3"}. This record does not assert independent reproduction.

### RAG

Interest: 4.18. Result ID: 699477743a6c3148491d710c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Foundation Model":"GPT-4o"}. This record does not assert independent reproduction.

### STORM

Interest: 4.16. Result ID: 699477743a6c3148491d710e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72cb1e209131cea58790.md>)

Recorded evaluation configuration: {"Foundation Model":"GPT-4o"}. This record does not assert independent reproduction.

### RAG

Interest: 4.03. Result ID: 699477743a6c3148491d7107.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Foundation Model":"DeepSeek-v3"}. This record does not assert independent reproduction.

### RAPID

Interest: 3.96. Result ID: 699477743a6c3148491d7106.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Implementation](<https://github.com/stanfordnlp/dspy>)

Recorded evaluation configuration: {"Foundation Model":"Qwen-Max"}. This record does not assert independent reproduction.

### Co-STORM

Interest: 3.92. Result ID: 699477743a6c3148491d7105.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa753c1e209131cea66a77.md>)

Recorded evaluation configuration: {"Foundation Model":"Qwen-Max"}. This record does not assert independent reproduction.

### STORM

Interest: 3.9. Result ID: 699477743a6c3148491d7104.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72cb1e209131cea58790.md>)

Recorded evaluation configuration: {"Foundation Model":"Qwen-Max"}. This record does not assert independent reproduction.

### Co-STORM

Interest: 3.88. Result ID: 699477743a6c3148491d710a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa753c1e209131cea66a77.md>)

Recorded evaluation configuration: {"Foundation Model":"DeepSeek-v3"}. This record does not assert independent reproduction.

### oRAG

Interest: 3.68. Result ID: 699477743a6c3148491d7108.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

Recorded evaluation configuration: {"Foundation Model":"DeepSeek-v3"}. This record does not assert independent reproduction.

### RAG

Interest: 3.66. Result ID: 699477743a6c3148491d7102.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Foundation Model":"Qwen-Max"}. This record does not assert independent reproduction.

### oRAG

Interest: 3.46. Result ID: 699477743a6c3148491d710d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

Recorded evaluation configuration: {"Foundation Model":"GPT-4o"}. This record does not assert independent reproduction.

### oRAG

Interest: 3.32. Result ID: 699477743a6c3148491d7103.

[Reporting paper](<https://www.sota2.com/research/paper/68fa77371e209131cea71ad7.md>)

Recorded evaluation configuration: {"Foundation Model":"Qwen-Max"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
