Semantic Parsing on MTOP
50.43AccuracyEPR-KMeans
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EPR-KMeansModel=Llama3-70B, Shots=1282025.06 | 50.43 | — | |
| CLGModel=Llama3-70B, Shots=1282025.06 | 47.74 | — | |
| EPR-KMeansModel=Qwen2.5-72B, Shots=1282025.06 | 47.53 | — | |
| CLGModel=Qwen2.5-72B, Shots=1282025.06 | 47.07 | — | |
| Best-of-NModel=Qwen2.5-72B, Shots=1282025.06 | 46.89 | — | |
| RandomModel=Qwen2.5-72B, Shots=1282025.06 | 45.32 | — | |
| BGE-KMeansModel=Llama3-70B, Shots=1282025.06 | 44.79 | — | |
| RandomModel=Llama3-70B, Shots=1282025.06 | 44.66 | — | |
| BGE-KMeansModel=Qwen2.5-72B, Shots=1282025.06 | 43.71 | — | |
| Best-of-NModel=Llama3-70B, Shots=1282025.06 | 43.4 | — | |
| Latent-BayesianModel=Llama3-70B, Shots=1282025.06 | 41.21 | — | |
| BM25-MajorModel=Qwen2.5-72B, Shots=1282025.06 | 16.24 | — | |
| BM25-MajorModel=Llama3-70B, Shots=1282025.06 | 8.41 | — | |
| Latent-BayesianModel=Qwen2.5-72B, Shots=1282025.06 | 5.23 | — | |
| Pasupat et al. (2021)Extra Pre-training=false2022.01 | — | 86.36 | |
| T5-3BBackbone=T5-3B2022.01 | — | 86.78 | |
| T5-baseBackbone=T5-base2022.01 | — | 85.49 | |
| T5-largeBackbone=T5-large2022.01 | — | 86.17 |