Text-to-Cypher on CypherBench (test)
85.3EX (%)Gemini 3.1 Pro
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 3.1 ProProtocol=One-shot, CYGNET=false2026.06 | 85.3 | — | 0.6 | |
| Gemini 3.1 ProProtocol=Corrected, CYGNET=true2026.06 | 85.2 | 0.01 | — | |
| Gemini 3 FlashProtocol=Corrected, CYGNET=true2026.06 | 85.2 | 1.35 | — | |
| Gemini 3 FlashProtocol=One-shot, CYGNET=false2026.06 | 83.8 | — | 3.9 | |
| Gemma 4 31BProtocol=Corrected, CYGNET=true2026.06 | 77.7 | 0.16 | — | |
| Gemma 4 31BProtocol=One-shot, CYGNET=false2026.06 | 77.6 | — | 4.8 | |
| Gemma 4 26BProtocol=Corrected, CYGNET=true2026.06 | 64.4 | 0.04 | — | |
| Gemma 4 26BProtocol=One-shot, CYGNET=false2026.06 | 64.3 | — | 19.3 | |
| Gemini 2.5 Flash LiteProtocol=Corrected, CYGNET=true2026.06 | 44.5 | 0.65 | — | |
| Gemini 2.5 Flash LiteProtocol=One-shot, CYGNET=false2026.06 | 43.8 | — | 7.1 |