# Supervised Fine-Tuning on SFT (val)

Source: [Supervised Fine-Tuning on SFT (val)](<https://www.sota2.com/research/sota/supervised-fine-tuning-on-sft-val>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-03T06:31:51.036Z

Tracking performance metrics during Supervised Fine-Tuning evaluation on the SFT validation set, including validation loss, performance delta (Δ), and optimizer state memory usage.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Supervised Fine-Tuning |
| Dataset (including any stated split) | SFT (val) |
| Metric | Validation Loss |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Validation Loss](<https://www.sota2.com/research/sota/supervised-fine-tuning-on-sft-val.md?metric=Validation%20Loss>): Lower is better; unit: SCALAR.

- [Delta (Δ)](<https://www.sota2.com/research/sota/supervised-fine-tuning-on-sft-val.md?metric=Delta%20(%CE%94)>): Higher is better; unit: SCALAR.

- [Optimizer State Memory (MB)](<https://www.sota2.com/research/sota/supervised-fine-tuning-on-sft-val.md?metric=Optimizer%20State%20Memory%20(MB)>): Lower is better; unit: SIZE.

## Recorded results

| Method | Validation Loss | Result date | Report paper |
| --- | --- | --- | --- |
| Ember | 1.3519 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 1.3527 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| Ember | 1.4306 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 1.4309 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| Ember | 1.5888 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 1.59 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| Ember | 1.743 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 1.7467 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| Ember | 2.0381 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 2.0456 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| Ember | 2.3578 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 2.364 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| AdamW | 2.4782 | Not recorded | 6a4741e5143ebdcacacc5a03 |
| Ember | 2.5136 | Not recorded | 6a4741e5143ebdcacacc5a03 |

### Ember

Validation Loss: 1.3519. Result ID: 6a47573a60b88bb33991ddf2.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"pythia-2.8b","Params (M)":"2800","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 1.3527. Result ID: 6a47573a60b88bb33991ddf1.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"pythia-2.8b","Params (M)":"2800","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### Ember

Validation Loss: 1.4306. Result ID: 6a47573a60b88bb33991ddf0.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"pythia-1.4b","Params (M)":"1400","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 1.4309. Result ID: 6a47573a60b88bb33991ddef.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"pythia-1.4b","Params (M)":"1400","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### Ember

Validation Loss: 1.5888. Result ID: 6a47573a60b88bb33991ddec.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"gpt2-large","Params (M)":"774","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 1.59. Result ID: 6a47573a60b88bb33991ddeb.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"gpt2-large","Params (M)":"774","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### Ember

Validation Loss: 1.743. Result ID: 6a47573a60b88bb33991ddea.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"gpt2-medium","Params (M)":"350","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 1.7467. Result ID: 6a47573a60b88bb33991dde9.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"gpt2-medium","Params (M)":"350","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### Ember

Validation Loss: 2.0381. Result ID: 6a47573a60b88bb33991dde8.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"gpt2","Params (M)":"124","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 2.0456. Result ID: 6a47573a60b88bb33991dde7.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"gpt2","Params (M)":"124","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### Ember

Validation Loss: 2.3578. Result ID: 6a47573a60b88bb33991dde6.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"distilgpt2","Params (M)":"82","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 2.364. Result ID: 6a47573a60b88bb33991dde5.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"distilgpt2","Params (M)":"82","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### AdamW

Validation Loss: 2.4782. Result ID: 6a47573a60b88bb33991dded.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"pythia-160m","Params (M)":"160","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

### Ember

Validation Loss: 2.5136. Result ID: 6a47573a60b88bb33991ddee.

[Reporting paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a4741e5143ebdcacacc5a03.md>)

Recorded evaluation configuration: {"Model":"pythia-160m","Params (M)":"160","Step":"250","Optimizer State Precision":"fp32"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
