Near-duplicate identification on Full C4 (365M documents)
27,207,144Number of DuplicatesSEDD
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SEDD2025.01 | 27,207,144 | 24,533,637 | 97 | |
| GPU baseline (NeMo Curator)2025.01 | 25,307,187 | — | — |
Loading the SOTA2 catalog…
| Method | Links | |||
|---|---|---|---|---|
| SEDD2025.01 | 27,207,144 | 24,533,637 | 97 | |
| GPU baseline (NeMo Curator)2025.01 | 25,307,187 | — | — |