List your company

ResearchFormerly WizWand

Llama3

Benchmarks

TaskDatasetSOTA resultTrendResults

Multi-bit LLM Watermarking

LLaMA3-8B-Base Max 256 Tokens

1AUC

20

Multi-bit LLM Watermarking

LLaMA3-8B-Base Max 128 Tokens

1AUC

20

Jailbreak attack

Llama3-8b

0Average ASR

16

Jailbreak Attack

llama3-8b pretrained v1

0ASR

13

Defending against gradient-based attacks

Llama3 AutoDAN Attack (test)

10.57ASR

10

Llama3

0.794Entropy (bit/token)

9

Style Manipulation Attack

Llama3-8B-Inst

1.481Score

6

LLM Jailbreaking

Llama3-CB

70Success Rate First (SRF)

6

Training Throughput

Llama3 8B (train)

2,320.47Throughput (128K SeqLen)

5

Training Memory Usage Profiling

Llama3-8B 8×H100s

21.1Peak Memory Usage (128K)

5

Model Merging for Safety and Utility

LLaMA3-8B-Instruct

12.4HS Score

4

LLaMA3-8B

27Averaged Quantization Time (s)

4

In-Context Learning

Llama3-8B Scenario 5 ICL prompts

82.8Accuracy

3

In-Context Learning

Llama3-8B Scenario 2: More Pos. (P={-5,...,-1})

78.18Accuracy

3

In-Context Learning

Llama3-8B Baseline (P={-1}, L={14})

78.65Accuracy

3

Training Efficiency

Llama3-8x70B Coarse-grained

41.6MFU

3

Model Compression

Llama3-1b

0.077Energy Consumed (kWh)

2