Label Classification on TRDL21 (Experiment 2)
0.376KappaRandom Forest
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Random ForestLLM=Deepseek-v3, CoT=w/o CoT, Confidence features=Included2026.01 | 0.376 | 0.623 | 51.5 | |
| Random ForestLLM=Deepseek-v3, CoT=w/ CoT, Confidence features=Included2026.01 | 0.372 | 0.623 | 51.5 | |
| Random ForestLLM=GPT-4o-mini, CoT=w/ CoT, Confidence features=Included2026.01 | 0.371 | 0.626 | 51.4 | |
| Random ForestLLM=GPT-4o-mini, CoT=w/o CoT, Confidence features=Included2026.01 | 0.364 | 0.622 | 50.8 | |
| Random Forest w/o conf.LLM=Deepseek-v3, CoT=w/ CoT, Confidence features=Excluded2026.01 | 0.364 | 0.613 | 50.6 | |
| Random Forest w/o conf.LLM=Deepseek-v3, CoT=w/o CoT, Confidence features=Excluded2026.01 | 0.363 | 0.614 | 50.3 | |
| OracleLLM=Deepseek-v3, CoT=w/ CoT2026.01 | 0.354 | 0.631 | 50.3 | |
| OracleLLM=Deepseek-v3, CoT=w/o CoT2026.01 | 0.351 | 0.604 | 49.8 | |
| Random Forest w/o conf.LLM=GPT-4o-mini, CoT=w/ CoT, Confidence features=Excluded2026.01 | 0.347 | 0.601 | 49.7 | |
| OracleLLM=GPT-4o-mini, CoT=w/ CoT2026.01 | 0.343 | 0.604 | 49.8 | |
| XGBoostLLM=Deepseek-v3, CoT=w/o CoT2026.01 | 0.338 | 0.591 | 46.9 | |
| XGBoostLLM=Deepseek-v3, CoT=w/ CoT2026.01 | 0.334 | 0.582 | 48 | |
| LGBMLLM=Deepseek-v3, CoT=w/ CoT2026.01 | 0.333 | 0.582 | 48.1 | |
| LGBMLLM=Deepseek-v3, CoT=w/o CoT2026.01 | 0.333 | 0.589 | 46.6 | |
| XGBoostLLM=GPT-4o-mini, CoT=w/ CoT2026.01 | 0.326 | 0.572 | 47.6 | |
| LGBMLLM=GPT-4o-mini, CoT=w/ CoT2026.01 | 0.322 | 0.569 | 47.2 | |
| Random Forest w/o conf.LLM=GPT-4o-mini, CoT=w/o CoT, Confidence features=Excluded2026.01 | 0.321 | 0.582 | 46.9 | |
| Ordinal Logistic RegressionLLM=Deepseek-v3, CoT=w/ CoT2026.01 | 0.32 | 0.598 | 40.9 | |
| XGBoostLLM=GPT-4o-mini, CoT=w/o CoT2026.01 | 0.319 | 0.565 | 47.3 | |
| LGBMLLM=GPT-4o-mini, CoT=w/o CoT2026.01 | 0.318 | 0.566 | 47.1 | |
| OracleLLM=GPT-4o-mini, CoT=w/o CoT2026.01 | 0.311 | 0.573 | 47.7 | |
| Ordinal Logistic RegressionLLM=Deepseek-v3, CoT=w/o CoT2026.01 | 0.307 | 0.607 | 39.8 | |
| Ordinal Logistic RegressionLLM=GPT-4o-mini, CoT=w/ CoT2026.01 | 0.287 | 0.565 | 39.1 | |
| Ordinal Logistic RegressionLLM=GPT-4o-mini, CoT=w/o CoT2026.01 | 0.277 | 0.579 | 38.3 | |
| Random ForestLLM=Llama-3-8B, CoT=w/o CoT, Confidence features=Included2026.01 | 0.267 | 0.478 | 43.1 | |
| Random ForestLLM=Llama-3-8B, CoT=w/ CoT, Confidence features=Included2026.01 | 0.262 | 0.494 | 41.7 | |
| XGBoostLLM=Llama-3-8B, CoT=w/ CoT2026.01 | 0.261 | 0.464 | 42.9 | |
| LGBMLLM=Llama-3-8B, CoT=w/o CoT2026.01 | 0.26 | 0.437 | 42.8 | |
| LGBMLLM=Llama-3-8B, CoT=w/ CoT2026.01 | 0.259 | 0.463 | 42.7 | |
| XGBoostLLM=Llama-3-8B, CoT=w/o CoT2026.01 | 0.259 | 0.433 | 42.8 | |
| Random Forest w/o conf.LLM=Llama-3-8B, CoT=w/ CoT, Confidence features=Excluded2026.01 | 0.246 | 0.471 | 38.8 | |
| Ordinal Logistic RegressionLLM=Llama-3-8B, CoT=w/ CoT2026.01 | 0.244 | 0.499 | 37.8 | |
| Ordinal Logistic RegressionLLM=Llama-3-8B, CoT=w/o CoT2026.01 | 0.243 | 0.495 | 36.9 | |
| Random Forest w/o conf.LLM=Llama-3-8B, CoT=w/o CoT, Confidence features=Excluded2026.01 | 0.224 | 0.387 | 37.9 | |
| OracleLLM=Llama-3-8B, CoT=w/ CoT2026.01 | 0.114 | 0.385 | 27.3 | |
| OracleLLM=Llama-3-8B, CoT=w/o CoT2026.01 | 0.113 | 0.34 | 25.1 |