Automated Profiling Audit on Academic records profiling dataset
0Refusal RateDeepSeek V3
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V3Mode=LLMs w/o Search (S)2026.05 | 0 | 1,014.6 | 14.21 | 2.9 | 2.9 | 0 | 0 | — | |
| DeepSeek V3Mode=LLMs w/ Search (S)2026.05 | 0 | 1,458.2 | 25.89 | 2.57 | 5.97 | 5.19 | 2.79 | — | |
| Doubao 1.5 ProMode=LLMs w/ Search (S)2026.05 | 0 | 1,655.9 | 13.32 | 2.8 | 4.17 | 10.71 | 6.4 | 0 | |
| Qwen3 CoderMode=LLMs w/ Search (S)2026.05 | 0 | 1,419.2 | 20.5 | 3.57 | 6.1 | 16.82 | 8.2 | 0 | |
| OpenAI GPT-4oMode=LLMs w/ Search (S)2026.05 | 0 | 1,577.4 | 12.87 | 3.9 | 7.6 | 77.78 | 70.61 | 0 | |
| PerplexityMode=LLMs w/ Search (S)2026.05 | 0 | 1,850.4 | 15.03 | 4.17 | 10.53 | 64.8 | 53.16 | 0 | |
| DeepSeek R1Mode=LLMs w/ Search & Thinking (S&T)2026.05 | 0 | 4,717.4 | 234.97 | 5.13 | 10.53 | 10.39 | 5.05 | — | |
| Doubao 1.5 ProMode=LLMs w/ Search & Thinking (S&T)2026.05 | 0 | 2,166.3 | 19.92 | 2.47 | 2.93 | 2.7 | 2.27 | 0 | |
| Qwen3 CoderMode=LLMs w/ Search & Thinking (S&T)2026.05 | 0 | 3,025.8 | 57.45 | 5.73 | 7.3 | 11.06 | 9.59 | 0 | |
| Gemini 2.5 ProMode=LLMs w/ Search & Thinking (S&T)2026.05 | 0 | 3,575.6 | 36.92 | 5.63 | 16.9 | 7.69 | 3.75 | 0 | |
| OpenAI GPT-4oMode=LLMs w/ Search & Thinking (S&T)2026.05 | 0 | 3,435.6 | 277.55 | 6 | 9.2 | 91.67 | 92.39 | 0 | |
| Doubao 1.5 ProMode=Deep Research (Agent)2026.05 | 0 | 3,620 | 261 | 2.37 | 7.37 | 66.36 | 57.01 | 0 | |
| Qwen3 CoderMode=Deep Research (Agent)2026.05 | 0 | 5,061.8 | 267.2 | 5.03 | 9.77 | 22.52 | 26.96 | 0 | |
| Meta AIMode=Deep Research (Agent)2026.05 | 0 | 97,707.6 | 412.1 | 9.17 | 18.1 | 1.09 | 0.74 | 0 | |
| PerplexityMode=Deep Research (Agent)2026.05 | 0 | 2,265.8 | 29.04 | 2.3 | 5.4 | 59.42 | 54.32 | 0 | |
| IcebergExplorerBackbone=Gemini 2.5 Pro2026.05 | 3.33 | 565,287.4 | 1,256.89 | 12.8 | 95.93 | 100 | 93.05 | 83.33 | |
| IcebergExplorerBackbone=OpenAI GPT-4.12026.05 | 3.33 | 681,436.13 | 810.38 | 13.93 | 100.7 | 99.54 | 79.21 | 76.67 | |
| OpenAI GPT-4oMode=LLMs w/o Search (S)2026.05 | 10 | 1,050.4 | 4.25 | 3.73 | 3.73 | 0 | 0 | 0 | |
| IcebergExplorerBackbone=Qwen32026.05 | 16.67 | 44,201.42 | 119.71 | 1.73 | 19.23 | 91.11 | 78.2 | 15.38 | |
| IcebergExplorerBackbone=OpenAI GPT-4o2026.05 | 30 | 332,327.55 | 315.04 | 11.86 | 70.32 | 100 | 92.44 | 77.27 | |
| Doubao 1.5 ProMode=LLMs w/o Search (S)2026.05 | 70 | 1,602.8 | 22.26 | 1.17 | 3.89 | 2.86 | 0 | — | |
| Gemini 2.5 ProMode=LLMs w/o Search (S)2026.05 | 70 | 1,657.8 | 23.9 | 1 | 3.34 | 6.67 | 6.67 | 0 | |
| Qwen3 CoderMode=LLMs w/o Search (S)2026.05 | 96.67 | 835.4 | 1.1 | 0.27 | 8 | 0 | 0 | — | |
| Gemini 2.5 ProMode=Deep Research (Agent)2026.05 | 100 | — | — | — | — | — | — | — | |
| Human Experts2026.05 | — | — | 1,094.46 | 14.6 | 15.4 | 100 | 100 | 40 |