# Semantic Parsing on SParC-DI (test)

Source: [Semantic Parsing on SParC-DI (test)](<https://www.sota2.com/research/sota/semantic-parsing-on-sparc-di-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:59:10.627Z

Evaluation of Semantic Parsing performance on the test split of the SParC-DI dataset, measured by Query accuracy and Interaction accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Semantic Parsing |
| Dataset (including any stated split) | SParC-DI (test) |
| Metric | Query Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Query Accuracy](<https://www.sota2.com/research/sota/semantic-parsing-on-sparc-di-test.md?metric=Query%20Accuracy>): Higher is better; unit: PERCENT.

- [Interaction Accuracy](<https://www.sota2.com/research/sota/semantic-parsing-on-sparc-di-test.md?metric=Interaction%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Query Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| Liu et al. (2020) | 57.1 | Not recorded | 68fa6b101e209131cea2bdd9 |
| MemCE+Liu et al. (2020) | 55.7 | Not recorded | 68fa6b101e209131cea2bdd9 |
| MemCE+CDS2S | 41.5 | Not recorded | 68fa6b101e209131cea2bdd9 |
| MemCE+CDS2S | 41.3 | Not recorded | 68fa6b101e209131cea2bdd9 |
| CDS2S | 39.5 | Not recorded | 68fa6b101e209131cea2bdd9 |
| CDS2S | 38.7 | Not recorded | 68fa6b101e209131cea2bdd9 |

### Liu et al. (2020)

Query Accuracy: 57.1. Result ID: 698e4f61ffc3901de09c85e4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67ba1e209131cea1828c.md>)

Recorded evaluation configuration: {"Enc-Dec":"HE-Grammar"}. This record does not assert independent reproduction.

### MemCE+Liu et al. (2020)

Query Accuracy: 55.7. Result ID: 698e4f61ffc3901de09c85e7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

Recorded evaluation configuration: {"Enc-Dec":"Mem-Grammar"}. This record does not assert independent reproduction.

### MemCE+CDS2S

Query Accuracy: 41.5. Result ID: 698e4f61ffc3901de09c85e6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

Recorded evaluation configuration: {"Enc-Dec":"Mem-SnipCopy"}. This record does not assert independent reproduction.

### MemCE+CDS2S

Query Accuracy: 41.3. Result ID: 698e4f61ffc3901de09c85e5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

Recorded evaluation configuration: {"Enc-Dec":"Mem-LSTM"}. This record does not assert independent reproduction.

### CDS2S

Query Accuracy: 39.5. Result ID: 698e4f61ffc3901de09c85e2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66541e209131cea0f82f.md>)

[Implementation](<https://github.com/ryanzhumich/sparc_atis_pytorch>)

Recorded evaluation configuration: {"Enc-Dec":"HE-LSTM"}. This record does not assert independent reproduction.

### CDS2S

Query Accuracy: 38.7. Result ID: 698e4f61ffc3901de09c85e3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6b101e209131cea2bdd9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66541e209131cea0f82f.md>)

[Implementation](<https://github.com/ryanzhumich/sparc_atis_pytorch>)

Recorded evaluation configuration: {"Enc-Dec":"HE-SnipCopy"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.