Geometry Problem Solving on Geometry3K 1.0 (test)
90.9Overall ScoreHuman Expert
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Human Expertevaluator=Expert Human2021.05 | 90.9 | 89.9 | 92 | 93.9 | 66.7 | 95.9 | 92.2 | 90.5 | 89.9 | 92.3 | |
| Inter-GPSground_truth_literals=true2021.05 | 78.3 | 83.1 | 77.9 | 62.3 | 75 | 86.4 | 83.3 | 77.6 | 61.5 | 70.4 | |
| Inter-GPSliterals=extracted from diagrams and text2021.05 | 57.5 | 59.1 | 61.7 | 30.2 | 50 | 59.3 | 66 | 52.4 | 45.5 | 48.1 | |
| Humanevaluator=Average Human2021.05 | 56.9 | 53.7 | 59.3 | 57.7 | 42.9 | 46.7 | 53.8 | 68.7 | 61.7 | 58.3 | |
| FILM-BARTlanguage_model=BART2021.05 | 33 | 32.1 | 33 | 35.8 | 50 | 34.6 | 32.6 | 37.1 | 30.1 | 37 | |
| FILM-BERTlanguage_model=BERT2021.05 | 32.8 | 32.9 | 33.3 | 30.2 | 25 | 32.1 | 32.3 | 32.2 | 34.3 | 33.3 | |
| FiLMbackbone=Feature Interleaved Linear Modulation2021.05 | 31.7 | 28.7 | 32.7 | 39.6 | 33.3 | 33.3 | 29.2 | 33.6 | 30.8 | 29.6 | |
| RelNetbackbone=Relational Network2021.05 | 29.6 | 26.2 | 34 | 20.8 | 41.7 | 29.6 | 33.7 | 25.2 | 28 | 25.9 | |
| I-onlyinput=Image only2021.05 | 27 | 26.2 | 28.4 | 24.5 | 16.7 | 24.7 | 26.7 | 30.1 | 30.1 | 25.9 | |
| Q+Iinput=Question Text and Image2021.05 | 26.7 | 26.2 | 26.7 | 28.3 | 25 | 21 | 28.1 | 32.2 | 21 | 25.9 | |
| Q-onlyinput=Question Text only2021.05 | 25.3 | 29.5 | 21.5 | 28.3 | 33.3 | 21 | 26 | 25.9 | 25.2 | 22.2 | |
| Randombaseline_type=Random Selection2021.05 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 |