Semantic Retrieval on AirDialog v1 (test)
0Avg Cost per QueryMIPS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MIPSCondition=book2024.06 | 0 | 93 | |
| MIPSCondition=no_flight2024.06 | 0 | 86.7 | |
| MIPSCondition=no_reservation2024.06 | 0 | 96.5 | |
| MIPSCondition=cancel2024.06 | 0 | 6.6 | |
| UDFCondition=book2024.06 | 0.01 | — | |
| UDFCondition=no_flight2024.06 | 0.01 | — | |
| UDFCondition=no_reservation2024.06 | 0.01 | — | |
| UDFCondition=cancel2024.06 | 0.01 | — | |
| UQECondition=book2024.06 | 0.01 | — | |
| UQECondition=no_flight2024.06 | 0.01 | — | |
| UQECondition=no_reservation2024.06 | 0.01 | — | |
| UQECondition=cancel2024.06 | 0.01 | — | |
| lc-gpt-4-turboCondition=book2024.06 | 0.43 | — | |
| lc-gpt-4-turboCondition=no_flight2024.06 | 0.43 | — | |
| lc-gpt-4-turboCondition=no_reservation2024.06 | 0.43 | — | |
| lc-gpt-4-turboCondition=cancel2024.06 | 0.43 | — | |
| lc-claude-3-opusCondition=book2024.06 | 0.74 | — | |
| lc-claude-3-opusCondition=no_flight2024.06 | 0.74 | — | |
| lc-claude-3-opusCondition=no_reservation2024.06 | 0.74 | — | |
| lc-claude-3-opusCondition=cancel2024.06 | 0.74 | — |