synthetic probes
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
8192-token synthetic probes Weighted average
50.6Accuracy
4
8192-token synthetic probes Semantic paraphrase+distractor
16.7Accuracy
4
synthetic probes Semantic paraphrase 8192-token
50Accuracy
4
Synthetic probes 8192-token Semantic exact
52.1Accuracy
4
8192-token synthetic probes Bridge-window
59.9Accuracy
4