AI-generated text detection on Long-form QA 9K pooled generations corpus
100Detection Accuracy (at 1% FPR)SP (Retrieval over 9K)
Evaluation results
| Method | Links | |
|---|---|---|
| SP (Retrieval over 9K)Generator Model=GPT2-XL, Query Transformation=Original2023.03 | 100 | |
| SP (Retrieval over 9K)Generator Model=OPT-13B, Query Transformation=Original2023.03 | 100 | |
| SP (Retrieval over 9K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 9K)Generator Model=GPT2-XL, Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 9K)Generator Model=OPT-13B, Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 9K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 9K)Generator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 98.5 | |
| BM25 (Retrieval over 9K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60L2023.03 | 98.5 | |
| BM25 (Retrieval over 9K)Generator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 98.3 | |
| BM25 (Retrieval over 9K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60 L,O2023.03 | 96 | |
| BM25 (Retrieval over 9K)Generator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 95.2 | |
| BM25 (Retrieval over 9K)Generator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 94.4 | |
| SP (Retrieval over 9K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60L2023.03 | 93.8 | |
| SP (Retrieval over 9K)Generator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 89.6 | |
| SP (Retrieval over 9K)Generator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 88.9 | |
| SP (Retrieval over 9K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60 L,O2023.03 | 84.6 | |
| SP (Retrieval over 9K)Generator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 76.4 | |
| SP (Retrieval over 9K)Generator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 75.4 |