Spy
Benchmarks
TaskDatasetSOTA resultTrendResults
SPY Legal Questions
93.2Name
6
SPY (Synthetic PII Yesterday)
52.2Legal Precision
5
SPY 2025Q2 τ=14d (Whole sample)
9.49IVRMSE
5
SPY 2020Q1 τ=14d (Whole sample)
5.53IVRMSE
5
SPY 2020Q2 τ=56d
4.36IVRMSE
5
SPY 2020Q1 τ=56d (Whole sample)
1.23IVRMSE
5
SPY ATM 2020Q1
91Shortfall Probability
5
SPY Moneyness > 1.03 28d maturity 2025Q2
4.04IVRMSE
5
SPY 2020Q1, Moneyness > 1.03, 28d maturity
2.02IVRMSE
5
SPY 2025Q2, Moneyness > 1, 28d maturity
3.97IVRMSE
5
SPY Moneyness < 1, 28d maturity 2025Q2
7.55IVRMSE
5
SPY 2020Q1, Moneyness < 1, 28d maturity
1.61IVRMSE
5
SPY Whole sample 28d maturity 2025Q2
7.34IVRMSE
5
SPY 2020Q1 Whole sample 28d maturity
1.76IVRMSE
5
SPY ARM64 d=24 1.5.0 (test)
0.036Inference Time (s)
4
Spy
70Accuracy
3
SPY K/F=1.03 2025Q2
14.535ES 5%
2
SPY ATM 2025Q2
17.711ES 5%
2
SPY K/F=1.03 2020Q1
6.187Expected Shortfall (5%)
2
SPY 2025Q2 K/F=1.03 (τ = 56d)
15.437ES5%
1
SPY ATM (τ = 56d) 2025Q2
22.015ES 5%
1
SPY K/F=1.03 (τ = 56d) 2020Q1
8.002ES (5%)
1
SPY ATM (τ = 56d) 2020Q1
9.963Exposure Shortfall (5%)
1