List your company

ResearchFormerly WizWand

Code

Benchmarks

TaskDatasetSOTA resultTrendResults

Code Generation

Code

86.9ASR

7

Language Modeling

Code

5.5Perplexity

6

Speculative Decoding

Code

2.38Speedup

6

Code Generation

Code (test)

36.5Accuracy

6

Misaligned Task Learning

Code In-domain

56.67Misalignment

6

Emergent Misalignment Measurement

Code

0Misalignment

6

Critique Quality Evaluation

Code

67.5Win Rate

6

Code Generation

Code

87.7MTP Acceptance Rate

5

ECG Classification

CODE 15%

96.6Macro AUROC

5

Code

2Coverage Loss

5

Language Modeling

Code 24B tokens

0.699Cross-Entropy Loss

5

Code Generation and Reasoning

Code Domain Average

73.7Accuracy

4

CODE (test)

3.62Turns

4

Code Generation

CODE

99F1 Score

4

Code

1,694.26Average Tokens per Sample

3

Code Generation

Code

183.54Throughput (token/s)

3

Representation coupling evaluation via mean coupling diagnostic

Code

98.6Mean Coupling

2

Zero-shot transfer attack

Code

95.9Attack Success Rate (ASR)

2