List your company

ResearchFormerly WizWand

GITHUB

Benchmarks

TaskDatasetSOTA resultTrendResults

Language Modeling

GitHub (test)

2.42Perplexity

113

Membership Inference Attack

GitHub Pythia

1ROC AUC

36

Membership Inference

GitHub Pythia (train)

95.6TPR@1%FPR

36

Membership Inference Attack

GitHub

0.876AUC

32

Semi-supervised graph classification

GITHUB 10-fold cross-validation

0.7Accuracy

21

Graph Classification

GITHUB

71.06Accuracy

18

JSON generation

github medium

78.47Schema Adherence

14

Language Modeling

GitHub (val)

1.83Perplexity

13

Language Modeling

GitHub tokens (test)

0.976Bits Per Token (BPT)

11

github

0.757AUC

6

Tokenization efficiency

GitHub

688Token Count

6

Autonomous Task Completion

GitHub

84Success Rate

6

PyTorch-to-JAX Translation

Github dataset Extrinsic Evaluation (100 samples)

0.41CodeBLEU

2

Token Efficiency

GitHub Events

968JSON Compact Token Count

1

Graph Explanation

Github (test)

65.01FID

1

Website Navigation

GitHub (test)

—Primary metric

0