ACRE
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ACRE Systematicity
67.44Query Accuracy
10
ACRE Compositionality
69.04Query Accuracy
10
ACRE (I.I.D.)
66.29Query Accuracy
10
ACRE 1.0 (systematicity)
93.9Accuracy
8
ACRE compositionality 1.0
93.27Accuracy
8
ACRE I.I.D. 1.0
92.98Accuracy
7
ACRE
93Success Rate
4
ACRE (Systematic)
93.9Overall Score
3
ACRE Compositional
91.76All
3