Salience of information assessment on DementiaBank manual transcripts
2.09Mean Score (Control Group)LLaMA
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LLaMAModel Version=Llama 3 70B, Transcript type=Manual2026.06 | 2.09 | 2.44 | 0.17 | 0.35 | 0.22 | 0.15 | 0.37 | |
| 4o-miniModel Version=GPT-4o-mini, Transcript type=Manual2026.06 | 1.87 | 2.08 | 0.11 | 0.5 | 0.38 | 0.19 | 0.1 | |
| LLaMAModel Version=Llama 3 70B, Transcript type=Manual2026.06 | 1.78 | 2.11 | 0.19 | 0.35 | 0.19 | 0.12 | 0.41 | |
| 4o-miniModel Version=GPT-4o-mini, Transcript type=Manual2026.06 | 1.69 | 1.98 | 0.17 | 0.52 | 0.44 | 0.31 | 0.15 | |
| 4oModel Version=GPT-4o, Transcript type=Manual2026.06 | 0.96 | 1.72 | 0.79 | 0.96 | 0.6 | 0.48 | 0.36 | |
| ClaudeModel Version=Claude 3.5 Sonnet, Transcript type=Manual2026.06 | 0.6 | 1.57 | 1.61 | 1.15 | 0.65 | 0.55 | 0.45 | |
| 4oModel Version=GPT-4o, Transcript type=Manual2026.06 | 0.51 | 1.42 | 1.78 | 0.95 | 0.6 | 0.53 | 0.43 | |
| ClaudeModel Version=Claude 3.5 Sonnet, Transcript type=Manual2026.06 | 0.37 | 1.27 | 2.43 | 0.88 | 0.62 | 0.41 | 0.52 |