Natural Language to Code Generation on UCLID5 Textbooks 33 benchmarks
0Semantic Score 1GPT3t
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT3tTraining=Fine-tuned2024.06 | 0 | — | — | 0 | 0 | 0 | |
| GPT4tPrompting=One-shot, Chain-of-Thought=true2024.06 | 0 | — | 0 | 0 | — | 0 | |
| GPT4tPrompting=Three-shot, Chain-of-Thought=true2024.06 | 0 | — | 0 | 0 | — | — | |
| GPT3tPrompting=Three-shot, Chain-of-Thought=true2024.06 | 0 | — | 0 | 0 | 0 | — | |
| GPT4tPrompting=Three-shot, Chain-of-Thought=false2024.06 | 0 | — | 0 | 0 | — | — | |
| GPT3tPrompting=Three-shot, Chain-of-Thought=false2024.06 | 0 | — | 0 | — | 0 | — |