IHEval
Benchmarks
TaskDatasetSOTA resultTrendResults
IHEval
86.7PLA
21
IHEval Conflict Multi-turn
51.74Accuracy
16
IHEval Conflict Single-turn
52.13Accuracy
16
IHEval Aligned Multi-turn
88.8Accuracy
16
IHEval Aligned Single-turn
84.8Accuracy
16
IHEval
100Language Detection (Reference)
12
IHEval Overall Tool Use v1 (All)
69.9Average Accuracy
12
IHEval v1 (Conflict)
80Accuracy
12
IHEval v1 (Aligned)
83Accuracy
12
IHEval v1 (Reference)
94Accuracy
12
IHEval v1 (Conflict)
39.8Accuracy
12
IHEval Aligned v1
55.9Accuracy
12
IHEval v1 (Reference)
86Accuracy
12
IHEval Single-Turn
88.5Accuracy (Reference)
12
IHEval Multi-Turn
89.8Accuracy (Reference)
12
IHEval Average 1.0
66.9Average Accuracy
12
IHEval Prompt Hijacking Conflict 1.0
45Accuracy
12
IHEval Prompt Hijacking - Alignment 1.0
82.5Accuracy
12
IHEval Prompt Hijacking 1.0 (Reference)
97.5Accuracy
12
IHEval Prompt Extraction - Conflict 1.0
59.6Accuracy
12
IHEval Prompt Extraction Alignment 1.0
83.7Accuracy
12
IHEval Prompt Extraction 1.0 (Reference)
96.9Accuracy
12
IHEval Tool-use
0FPR
6
IHEval Rule-following
0.01FPR
6
IHEval
88.1Accuracy
4