Differential Transformer (Diff) implementation on New-feature tasks
38.2Session DurationPithTrain
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PithTrainFramework=PithTrain2026.05 | 38.2 | 27.6 | 47 | 69,500 | 25,400 | |
| Megatron-LMFramework=Megatron-LM2026.05 | 47.1 | 33.7 | 125 | 118,700 | 57,100 | |
| TorchTitanFramework=TorchTitan2026.05 | 49.6 | 40.3 | 58 | 103,200 | 36,000 |