# Model-Human Comparison on model-vs-human (test) Source: [Model-Human Comparison on model-vs-human (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:42:12.666Z Evaluation of model performance via direct comparison against human performance or judgments on the model-vs-human test set, focusing on metrics like shape bias, OOD accuracy, and error consistency. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Model-Human Comparison | | Dataset (including any stated split) | model-vs-human (test) | | Metric | Error Consistency | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Error Consistency](): Higher is better; unit: SCALAR. - [Shape Bias](): Higher is better; unit: PERCENT. - [OOD Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Error Consistency | Result date | Report paper | | --- | --- | --- | --- | | Humans (avg) | 0.43 | Not recorded | 6993f5fa9ebc499c974099f4 | | OpenCLIP ViT-H-14 Fourier-filtered | 0.38 | Not recorded | 6993f5fa9ebc499c974099f4 | | OpenCLIP ViT-H-14 Blurred | 0.37 | Not recorded | 6993f5fa9ebc499c974099f4 | | OpenCLIP ViT-H-14 Resized (64x64) | 0.35 | Not recorded | 6993f5fa9ebc499c974099f4 | | Imagen | 0.31 | Not recorded | 68fa5f5d1e209131ce9e94cb | | Imagen | 0.31 | Not recorded | 6993f5fa9ebc499c974099f4 | | CLIP | 0.28 | Not recorded | 68fa5f5d1e209131ce9e94cb | | OpenCLIP ViT-H-14 | 0.28 | Not recorded | 6993f5fa9ebc499c974099f4 | | Stable Diffusion | 0.26 | Not recorded | 68fa5f5d1e209131ce9e94cb | | CLIP | 0.26 | Not recorded | 68fa5f5d1e209131ce9e94cb | | ViT-22B-384 | 0.26 | Not recorded | 68fa5f5d1e209131ce9e94cb | | ViT-22B-384 | 0.26 | Not recorded | 6993f5fa9ebc499c974099f4 | | RN-50 | 0.24 | Not recorded | 68fa5f5d1e209131ce9e94cb | | Parti | 0.23 | Not recorded | 68fa5f5d1e209131ce9e94cb | | ViT-L | 0.21 | Not recorded | 68fa5f5d1e209131ce9e94cb | | RN-50 | 0.21 | Not recorded | 68fa5f5d1e209131ce9e94cb | | RN-50 | 0.18 | Not recorded | 68fa5f5d1e209131ce9e94cb | ### Humans (avg) Error Consistency: 0.43. Result ID: 699a810a5698665ec40ab7f3. [Reporting paper]() Recorded evaluation configuration: {"Subject":"Human"}. This record does not assert independent reproduction. ### OpenCLIP ViT-H-14 Fourier-filtered Error Consistency: 0.38. Result ID: 699a810a5698665ec40ab7f9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-H-14","Input Transformation":"Fourier-filtered"}. This record does not assert independent reproduction. ### OpenCLIP ViT-H-14 Blurred Error Consistency: 0.37. Result ID: 699a810a5698665ec40ab7f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-H-14","Input Transformation":"Blurred (sigma = 2.5)"}. This record does not assert independent reproduction. ### OpenCLIP ViT-H-14 Resized (64x64) Error Consistency: 0.35. Result ID: 699a810a5698665ec40ab7f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-H-14","Input Transformation":"Resized (64x64)"}. This record does not assert independent reproduction. ### Imagen Error Consistency: 0.31. Result ID: 698bc5377a967eddb7713aae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"zero-shot","prompts":"1 prompt"}. This record does not assert independent reproduction. ### Imagen Error Consistency: 0.31. Result ID: 699a810a5698665ec40ab7f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Imagen"}. This record does not assert independent reproduction. ### CLIP Error Consistency: 0.28. Result ID: 698bc5377a967eddb7713ab2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"model type":"zero-shot","prompts":"80 prompts"}. This record does not assert independent reproduction. ### OpenCLIP ViT-H-14 Error Consistency: 0.28. Result ID: 699a810a5698665ec40ab7f5. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"ViT-H-14"}. This record does not assert independent reproduction. ### Stable Diffusion Error Consistency: 0.26. Result ID: 698bc5377a967eddb7713aaf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"zero-shot","prompts":"1 prompt"}. This record does not assert independent reproduction. ### CLIP Error Consistency: 0.26. Result ID: 698bc5377a967eddb7713ab1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"model type":"zero-shot","prompts":"1 prompt"}. This record does not assert independent reproduction. ### ViT-22B-384 Error Consistency: 0.26. Result ID: 698bc5377a967eddb7713ab3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"discriminative","training\_data":"4B images"}. This record does not assert independent reproduction. ### ViT-22B-384 Error Consistency: 0.26. Result ID: 699a810a5698665ec40ab7f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"ViT-22B","Resolution":"384"}. This record does not assert independent reproduction. ### RN-50 Error Consistency: 0.24. Result ID: 698bc5377a967eddb7713ab6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"discriminative","augmentation":"diffusion noise"}. This record does not assert independent reproduction. ### Parti Error Consistency: 0.23. Result ID: 698bc5377a967eddb7713ab0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"zero-shot","prompts":"1 prompt"}. This record does not assert independent reproduction. ### ViT-L Error Consistency: 0.21. Result ID: 698bc5377a967eddb7713ab4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"model type":"discriminative","training\_data":"IN-21K"}. This record does not assert independent reproduction. ### RN-50 Error Consistency: 0.21. Result ID: 698bc5377a967eddb7713ab5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"discriminative","training\_data":"IN-1K"}. This record does not assert independent reproduction. ### RN-50 Error Consistency: 0.18. Result ID: 698bc5377a967eddb7713ab7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"model type":"discriminative","mode":"train+eval w/ diffusion noise"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.