Privacy appropriateness classification on Education
84.8F1 (Area)ARIEL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ARIELModel=GPT-52025.12 | 84.8 | 89.2 | 1,603 | |
| ARIELModel=Gemini 2.5 Pro2025.12 | 84.1 | 88.2 | 1,681 | |
| ARIELModel=Claude Sonnet 4.52025.12 | 84 | 88.4 | 1,608 | |
| ARIELModel=Gemma 3 4B2025.12 | 80.4 | 86.1 | 1,644 | |
| ICL (w Undet)Model=Gemini 2.5 Pro2025.12 | 75.5 | 87.2 | 2,325 | |
| ICLModel=Claude Sonnet 4.52025.12 | 75.4 | 82.8 | 3,020 | |
| ICL (w Undet)Model=Claude Sonnet 4.52025.12 | 75 | 82.8 | 2,950 | |
| ICL (w Undet)Model=GPT-52025.12 | 74.4 | 83.2 | 2,961 | |
| ICLModel=GPT-52025.12 | 74 | 82.9 | 3,020 | |
| ICLModel=Gemini 2.5 Pro2025.12 | 69.9 | 80.8 | 3,020 | |
| zero-shotModel=Gemma 3 4B2025.12 | 64.1 | 18 | — | |
| ICL (w Undet)Model=Gemma 3 4B2025.12 | 63.4 | 78.8 | 3,020 | |
| zero-shotModel=GPT-52025.12 | 62.9 | 55.6 | — | |
| privacy normsModel=Claude Sonnet 4.52025.12 | 62.3 | 62.5 | — | |
| ICLModel=Gemma 3 4B2025.12 | 61.7 | 78.7 | 3,020 | |
| zero-shotModel=Claude Sonnet 4.52025.12 | 61.4 | 63 | — | |
| zero-shotModel=Gemini 2.5 Pro2025.12 | 60.1 | 61.7 | — | |
| privacy normsModel=GPT-52025.12 | 51.2 | 69.2 | — | |
| privacy normsModel=Gemma 3 4B2025.12 | 46.7 | 66.8 | — | |
| privacy normsModel=Gemini 2.5 Pro2025.12 | 38.5 | 71.6 | — |