Geolocation Inference on Location-related Privacy Leakage Dataset
99.1VRRNon-Expert Human (MTurk)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Non-Expert Human (MTurk)Type=Human Baseline2025.04 | 99.1 | 140.08 | 37.22 | 6.01 | 1,309.73 | |
| GPT-4.1Prediction Setting=Top 3, MLRM=false2025.04 | 96.88 | 14.06 | 4.29 | 19.42 | 1,916.55 | |
| GEMINI 2.5 PROPrediction Setting=Top 3, MLRM=true2025.04 | 95.07 | 9.92 | 2.98 | 21.97 | 1,987.16 | |
| LLAMA 4 MAVERICKPrediction Setting=Top 3, MLRM=false2025.04 | 91.85 | 174.82 | 28.49 | 7.05 | 1,253.85 | |
| LLAMA 4 MAVERICKPrediction Setting=Top 1, MLRM=false2025.04 | 88.77 | 166.61 | 30.86 | 7.49 | 1,219.01 | |
| OPENAI O3Prediction Setting=Top 3, MLRM=true2025.04 | 87.95 | 7.44 | 2.73 | 20.09 | 1,912.77 | |
| QVQ-MAXPrediction Setting=Top 3, MLRM=true2025.04 | 84.8 | 32.92 | 16.15 | 9.69 | 1,455.18 | |
| GEMINI 2.5 PROPrediction Setting=Top 1, MLRM=true2025.04 | 84.53 | 14.75 | 4.63 | 19.73 | 1,701.61 | |
| GPT-4.1Prediction Setting=Top 1, MLRM=false2025.04 | 83.48 | 15.24 | 6.07 | 13.84 | 1,647.29 | |
| OPENAI O3Prediction Setting=Top 1, MLRM=true2025.04 | 80.8 | 13.56 | 5.46 | 14.73 | 1,628.5 | |
| GPT-5Prediction Setting=Top 1, MLRM=true2025.04 | 78.41 | 11.26 | 4.35 | 17.4 | 1,633.87 | |
| GPT-5Prediction Setting=Top 3, MLRM=true2025.04 | 74.23 | 6.69 | 2.15 | 22.03 | 1,688.66 | |
| OPENAI O4-MINIPrediction Setting=Top 3, MLRM=true2025.04 | 71.88 | 11.2 | 4.31 | 16.96 | 1,515.72 | |
| QVQ-MAXPrediction Setting=Top 1, MLRM=true2025.04 | 66.74 | 121.06 | 24.02 | 9.25 | 1,025.05 | |
| OPENAI O4-MINIPrediction Setting=Top 1, MLRM=true2025.04 | 53.79 | 15.64 | 7.04 | 12.05 | 1,105.84 | |
| CLAUDE OPUS 4Prediction Setting=Top 3, MLRM=true2025.04 | 40.75 | 20.33 | 5.49 | 9.03 | 859.03 | |
| CLAUDE OPUS 4Prediction Setting=Top 3, MLRM=false2025.04 | 39.65 | 21.92 | 9.16 | 7.27 | 804.2 | |
| LLAMA 4 SCOUTPrediction Setting=Top 1, MLRM=false2025.04 | 34.36 | 129.16 | 26.32 | 3.52 | 565.58 | |
| LLAMA 4 SCOUTPrediction Setting=Top 3, MLRM=false2025.04 | 32.38 | 33.6 | 14.46 | 4.63 | 627.2 | |
| CLAUDE SONNET 4Prediction Setting=Top 3, MLRM=false2025.04 | 27.31 | 92.15 | 8.99 | 6.17 | 516 | |
| CLAUDE OPUS 4Prediction Setting=Top 1, MLRM=false2025.04 | 24.01 | 145.06 | 30.04 | 5.29 | 401.24 | |
| CLAUDE SONNET 4Prediction Setting=Top 1, MLRM=false2025.04 | 23.35 | 92.68 | 9.62 | 4.85 | 444.71 | |
| CLAUDE OPUS 4Prediction Setting=Top 1, MLRM=true2025.04 | 15.64 | 108.52 | 3.36 | 4.85 | 328.11 | |
| GPT-4OPrediction Setting=Top 3, MLRM=false2025.04 | 13.84 | 1.24 | 0.27 | 7.14 | 432.47 | |
| GPT-4OPrediction Setting=Top 1, MLRM=false2025.04 | 12.95 | 2.01 | 0.4 | 6.03 | 389.83 | |
| CLAUDE SONNET 4Prediction Setting=Top 3, MLRM=true2025.04 | 12.11 | 21.34 | 0.62 | 4.85 | 317 | |
| CLAUDE SONNET 4Prediction Setting=Top 1, MLRM=true2025.04 | 9.47 | 4.8 | 1 | 3.3 | 265.25 |