Model Helpfulness Evaluation on Just-Eval (test)
4.96Helpfulness ScoreSAGE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SAGEModel=GPT-4o2025.05 | 4.96 | 5 | 4.95 | 4.53 | 4.63 | 4.81 | |
| No DefenseModel=GPT-4o2025.05 | 4.95 | 4.95 | 4.81 | 4.51 | 4.8 | 4.8 | |
| Self-ReminderModel=GPT-4o2025.05 | 4.94 | 4.99 | 4.92 | 4.4 | 4.91 | 4.83 | |
| Self-ReminderModel=Claude-3.5-Sonnet2025.05 | 4.92 | 4.97 | 4.87 | 4.06 | 4.59 | 4.68 | |
| SAGEModel=GPT-4o-mini2025.05 | 4.9 | 4.97 | 4.9 | 4.26 | 4.19 | 4.64 | |
| No DefenseModel=GPT-4o-mini2025.05 | 4.87 | 4.94 | 4.71 | 4.14 | 4.59 | 4.65 | |
| SAGEModel=Claude-3.5-Sonnet2025.05 | 4.87 | 4.97 | 4.89 | 3.94 | 4.52 | 4.64 | |
| No DefenseBackbone=Gemma-2-9B-IT2025.05 | 4.87 | 4.93 | 4.6 | 4.33 | 4.79 | 4.7 | |
| Self-ReminderModel=GPT-4o-mini2025.05 | 4.86 | 4.97 | 4.76 | 3.97 | 4.67 | 4.65 | |
| Goal PrioritizationModel=Claude-3.5-Sonnet2025.05 | 4.85 | 4.97 | 4.8 | 3.94 | 4.64 | 4.64 | |
| ICDModel=GPT-4o-mini2025.05 | 4.83 | 4.97 | 4.73 | 3.93 | 4.47 | 4.59 | |
| Self-ExaminationModel=GPT-4o2025.05 | 4.83 | 4.86 | 4.71 | 4.4 | 4.69 | 4.7 | |
| No DefenseModel=Claude-3.5-Sonnet2025.05 | 4.83 | 4.86 | 4.71 | 4.4 | 4.69 | 4.7 | |
| Goal PrioritizationModel=GPT-4o2025.05 | 4.79 | 4.98 | 4.89 | 3.88 | 4.56 | 4.62 | |
| No DefenseBackbone=Llama-3.1-8B-Instruct2025.05 | 4.79 | 4.92 | 4.49 | 4.07 | 4.53 | 4.56 | |
| IAModel=Claude-3.5-Sonnet2025.05 | 4.77 | 4.91 | 4.69 | 3.74 | 4.16 | 4.45 | |
| No DefenseBackbone=Qwen2.5-7B-Instruct2025.05 | 4.77 | 4.91 | 4.57 | 4.1 | 4.55 | 4.58 | |
| Self-ReminderBackbone=Qwen2.5-7B-Instruct2025.05 | 4.77 | 4.91 | 4.54 | 3.94 | 4.55 | 4.54 | |
| Self-ReminderBackbone=Gemma-2-9B-IT2025.05 | 4.75 | 4.89 | 4.67 | 3.92 | 4.6 | 4.57 | |
| ICDBackbone=Gemma-2-9B-IT2025.05 | 4.75 | 4.91 | 4.63 | 4.07 | 4.63 | 4.6 | |
| Self-ExaminationModel=Claude-3.5-Sonnet2025.05 | 4.73 | 4.87 | 4.66 | 3.9 | 4.33 | 4.5 | |
| Self-ExaminationBackbone=Qwen2.5-7B-Instruct2025.05 | 4.72 | 4.88 | 4.5 | 4.05 | 4.51 | 4.53 | |
| Self-ExaminationModel=GPT-4o-mini2025.05 | 4.7 | 4.84 | 4.6 | 4.02 | 4.45 | 4.52 | |
| ICDBackbone=Qwen2.5-7B-Instruct2025.05 | 4.7 | 4.86 | 4.44 | 3.94 | 4.41 | 4.47 | |
| SAGEBackbone=Llama-3.1-8B-Instruct2025.05 | 4.7 | 4.79 | 4.74 | 3.97 | 4.01 | 4.44 | |
| SAGEBackbone=Gemma-2-9B-IT2025.05 | 4.69 | 4.81 | 4.68 | 4.04 | 3.9 | 4.42 | |
| Goal PrioritizationModel=GPT-4o-mini2025.05 | 4.68 | 4.93 | 4.69 | 3.74 | 4.36 | 4.48 | |
| Self-ReminderBackbone=Llama-3.1-8B-Instruct2025.05 | 4.62 | 4.82 | 4.49 | 3.98 | 4.59 | 4.5 | |
| IAModel=GPT-4o-mini2025.05 | 4.56 | 4.94 | 4.64 | 3.48 | 3.88 | 4.3 | |
| Self-ExaminationBackbone=Gemma-2-9B-IT2025.05 | 4.51 | 4.67 | 4.42 | 4 | 4.41 | 4.4 | |
| IABackbone=Gemma-2-9B-IT2025.05 | 4.51 | 4.76 | 4.55 | 3.53 | 4.1 | 4.29 | |
| SAGEBackbone=Qwen2.5-7B-Instruct2025.05 | 4.51 | 4.81 | 4.58 | 3.78 | 4.29 | 4.39 | |
| IABackbone=Llama-3.1-8B-Instruct2025.05 | 4.51 | 4.76 | 4.35 | 3.66 | 4.08 | 4.27 | |
| Self-ExaminationBackbone=Llama-3.1-8B-Instruct2025.05 | 4.46 | 4.67 | 4.32 | 3.8 | 4.15 | 4.28 | |
| ICDModel=Claude-3.5-Sonnet2025.05 | 4.38 | 4.69 | 4.68 | 3.46 | 4.02 | 4.25 | |
| IAModel=GPT-4o2025.05 | 4.23 | 4.92 | 4.69 | 3.14 | 3.56 | 4.11 | |
| IABackbone=Qwen2.5-7B-Instruct2025.05 | 4.13 | 4.69 | 4.37 | 3.28 | 3.75 | 4.04 | |
| Goal PrioritizationBackbone=Llama-3.1-8B-Instruct2025.05 | 4.12 | 4.71 | 4.28 | 3.33 | 3.96 | 4.08 | |
| Goal PrioritizationBackbone=Qwen2.5-7B-Instruct2025.05 | 3.91 | 4.65 | 4.3 | 3.01 | 3.71 | 3.92 | |
| ICDBackbone=Llama-3.1-8B-Instruct2025.05 | 3.81 | 4.26 | 4.07 | 3.13 | 3.54 | 3.76 | |
| Goal PrioritizationBackbone=Gemma-2-9B-IT2025.05 | 3.79 | 4.55 | 4.58 | 3.1 | 3.59 | 3.92 | |
| ICDModel=GPT-4o2025.05 | 3.21 | 3.69 | 3.96 | 2.86 | 3.12 | 3.37 |