# 'Tis but Thy Name: Semantic Question Answering Evaluation with 11M Names for 1M Entities

Source: ['Tis but Thy Name: Semantic Question Answering Evaluation with 11M Names for 1M Entities](<https://www.sota2.com/research/paper/68fa6bdd1e209131cea30b4f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:59:05.045Z

Authors: Albert Huang

Published: 2022-02-28T07:12:39.000Z

## Abstract

Classic lexical-matching-based QA metrics are slowly being phased out because they punish succinct or informative outputs just because those answers were not provided as ground truth. Recently proposed neural metrics can evaluate semantic similarity but were trained on small textual similarity datasets grafted from foreign domains. We introduce the Wiki Entity Similarity (WES) dataset, an 11M example, domain targeted, semantic entity similarity dataset that is generated from link texts in Wikipedia. WES is tailored to QA evaluation: the examples are entities and phrases and grouped into semantic clusters to simulate multiple ground-truth labels. Human annotators consistently agree with WES labels, and a basic cross encoder metric is better than four classic metrics at predicting human judgments of correctness.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2202.13581>)

[PDF](<https://arxiv.org/pdf/2202.13581.pdf>)