AI-generated text detection on Long-form QA 3K generations corpus
100Detection Accuracy (1% FPR)Watermark
Evaluation results
| Method | Links | |
|---|---|---|
| WatermarkGenerator Model=GPT2-XL, Query Transformation=Original2023.03 | 100 | |
| WatermarkGenerator Model=OPT-13B, Query Transformation=Original2023.03 | 100 | |
| SP (Retrieval over 3K)Generator Model=GPT2-XL, Query Transformation=Original2023.03 | 100 | |
| SP (Retrieval over 3K)Generator Model=OPT-13B, Query Transformation=Original2023.03 | 100 | |
| SP (Retrieval over 3K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 3K)Generator Model=GPT2-XL, Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 3K)Generator Model=OPT-13B, Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 3K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=Original2023.03 | 100 | |
| BM25 (Retrieval over 3K)Generator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 99.3 | |
| BM25 (Retrieval over 3K)Generator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 99.2 | |
| BM25 (Retrieval over 3K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60L2023.03 | 98.6 | |
| BM25 (Retrieval over 3K)Generator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 97.8 | |
| BM25 (Retrieval over 3K)Generator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 97.3 | |
| BM25 (Retrieval over 3K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60 L,O2023.03 | 96.2 | |
| SP (Retrieval over 3K)Generator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 95.6 | |
| SP (Retrieval over 3K)Generator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 94.8 | |
| SP (Retrieval over 3K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60L2023.03 | 94.2 | |
| SP (Retrieval over 3K)Generator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 87.7 | |
| SP (Retrieval over 3K)Generator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 85.3 | |
| SP (Retrieval over 3K)Generator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60 L,O2023.03 | 85.1 | |
| DetectGPTGenerator Model=GPT2-XL, Query Transformation=Original2023.03 | 74.9 | |
| WatermarkGenerator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 71.1 | |
| WatermarkGenerator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 65.5 | |
| OpenAIGenerator Model=GPT2-XL, Query Transformation=Original2023.03 | 59.2 | |
| WatermarkGenerator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 55.8 | |
| WatermarkGenerator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 51.4 | |
| OpenAIGenerator Model=GPT-3.5 (davinci-003), Query Transformation=Original2023.03 | 40.5 | |
| OpenAIGenerator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60L2023.03 | 40.1 | |
| OpenAIGenerator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60 L,O2023.03 | 38.1 | |
| OpenAIGenerator Model=OPT-13B, Query Transformation=Original2023.03 | 33.5 | |
| OpenAIGenerator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 32.7 | |
| OpenAIGenerator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 31.3 | |
| DetectGPTGenerator Model=OPT-13B, Query Transformation=Original2023.03 | 29.8 | |
| OpenAIGenerator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 21.6 | |
| OpenAIGenerator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 21.6 | |
| DetectGPTGenerator Model=GPT2-XL, Query Transformation=+ 60L2023.03 | 15.8 | |
| DetectGPTGenerator Model=GPT2-XL, Query Transformation=+ 60 L,O2023.03 | 7.6 | |
| DetectGPTGenerator Model=OPT-13B, Query Transformation=+ 60L2023.03 | 3.2 | |
| DetectGPTGenerator Model=OPT-13B, Query Transformation=+ 60 L,O2023.03 | 1.5 | |
| DetectGPTGenerator Model=GPT-3.5 (davinci-003), Query Transformation=Original2023.03 | 1 | |
| DetectGPTGenerator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60L2023.03 | 0 | |
| DetectGPTGenerator Model=GPT-3.5 (davinci-003), Query Transformation=+ 60 L,O2023.03 | 0 |