# IED

Source: [IED](<https://www.sota2.com/research/dataset/ied>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Benchmarks

Page 1 of 10. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/dataset/ied.md?page=2>): Follow this link to discover more pages.

- [Dynamic Multi-objective Optimization on DF and FDA Benchmark Suites (DF1-DF14, FDA1-FDA5) Modified](<https://www.sota2.com/research/sota/dynamic-multi-objective-optimization-on-df-and-fda-benchmark-suites-df1-df14-fda1-fda5-modified.md>): Dynamic multi-objective optimization; DF and FDA benchmark suites (DF1-DF14, FDA1-FDA5) MIGD values Modified.

- [Software Engineering Issue Resolution on SWE-bench Verified](<https://www.sota2.com/research/sota/software-engineering-issue-resolution-on-swe-bench-verified.md>): Software Engineering Issue Resolution; SWE-bench Verified.

- [Question Answering on SimpleQA Verified](<https://www.sota2.com/research/sota/question-answering-on-simpleqa-verified.md>): Question Answering; SimpleQA Verified.

- [Agentic Coding on SWE-bench Verified](<https://www.sota2.com/research/sota/agentic-coding-on-swe-bench-verified.md>): Agentic Coding; SWE-bench Verified.

- [Software Engineering Task Resolution on SWE-bench Verified](<https://www.sota2.com/research/sota/software-engineering-task-resolution-on-swe-bench-verified.md>): Software Engineering Task Resolution; SWE-bench Verified.

- [Concept Accuracy on Broden simplified (test)](<https://www.sota2.com/research/sota/concept-accuracy-on-broden-simplified-test.md>): Concept Accuracy; Broden simplified (test).

- [Program Repair on Real misspecified programs neutral briefs](<https://www.sota2.com/research/sota/program-repair-on-real-misspecified-programs-neutral-briefs.md>): Program Repair; Real misspecified programs neutral briefs.

- [Point Cloud Classification on STPCTLS (5-fold stratified cross-validation)](<https://www.sota2.com/research/sota/point-cloud-classification-on-stpctls-5-fold-stratified-cross-validation.md>): Point Cloud Classification; STPCTLS (5-fold stratified cross-validation).

- [Embodied AI on EmbodiedBench EB-Habitat](<https://www.sota2.com/research/sota/embodied-ai-on-embodiedbench-eb-habitat.md>): Embodied AI; EmbodiedBench EB-Habitat.

- [Software Engineering on SWE-bench Verified (Success Rate (%))](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-success-rate-percent.md>): Software Engineering; SWE-bench Verified.

- [Regression on fried](<https://www.sota2.com/research/sota/regression-on-fried.md>): Regression; fried.

- [Software Engineering on SWE-bench Verified (Accuracy)](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-accuracy.md>): Software Engineering; SWE-bench Verified.

- [OS GUI Agentic Task Execution on OSWorld 361 tasks (Verified)](<https://www.sota2.com/research/sota/os-gui-agentic-task-execution-on-osworld-361-tasks-verified.md>): OS GUI Agentic Task Execution; OSWorld 361 tasks (Verified).

- [Debugging on OR-Debug-Bench (450 stratified samples)](<https://www.sota2.com/research/sota/debugging-on-or-debug-bench-450-stratified-samples.md>): Debugging; OR-Debug-Bench (450 stratified samples).

- [Textual Explanation on ImageNet-1K misclassified cases](<https://www.sota2.com/research/sota/textual-explanation-on-imagenet-1k-misclassified-cases.md>): Textual Explanation; ImageNet-1K misclassified cases.

- [Multimodal Continual Instruction Tuning on UCIT](<https://www.sota2.com/research/sota/multimodal-continual-instruction-tuning-on-ucit.md>): Multimodal Continual Instruction Tuning; UCIT (Unified Continual Instruction Tuning).

- [Perceptual Quality Assessment on TIEdit 1.0 (test)](<https://www.sota2.com/research/sota/perceptual-quality-assessment-on-tiedit-1-0-test.md>): Perceptual Quality Assessment; TIEdit 1.0 (test).

- [Reward Modeling on Unified Feedback (UF)](<https://www.sota2.com/research/sota/reward-modeling-on-unified-feedback-uf.md>): Reward Modeling; Unified Feedback (UF).

- [GUI Automation on OSWorld Verified (test)](<https://www.sota2.com/research/sota/gui-automation-on-osworld-verified-test.md>): GUI Automation; OSWorld Verified (test).

- [Automated Software Engineering on SWE-bench Verified](<https://www.sota2.com/research/sota/automated-software-engineering-on-swe-bench-verified.md>): Automated Software Engineering; SWE-bench Verified.

- [Image Editing on IEdit](<https://www.sota2.com/research/sota/image-editing-on-iedit.md>): Image Editing; IEdit.

- [Issue Resolution on SWE-bench Verified (test)](<https://www.sota2.com/research/sota/issue-resolution-on-swe-bench-verified-test.md>): Issue Resolution; SWE-bench Verified (test).

- [Regression on Friedman 10](<https://www.sota2.com/research/sota/regression-on-friedman-10.md>): Regression; Friedman 10.

- [Regression on Friedman 5](<https://www.sota2.com/research/sota/regression-on-friedman-5.md>): Regression; Friedman 5.

- [Adversarial Attack on JailbreakBench 50% stratified per-category sample (48 requests)](<https://www.sota2.com/research/sota/adversarial-attack-on-jailbreakbench-50-percent-stratified-per-category-sample-48-requests.md>): Adversarial Attack; JailbreakBench 50% stratified per-category sample (48 requests).

- [Deception Detection on Varied Deception](<https://www.sota2.com/research/sota/deception-detection-on-varied-deception.md>): Deception Detection; Varied Deception.

- [Software Engineering on SWE-bench verified (All)](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-all.md>): Software Engineering; SWE-bench verified (All).

- [Jailbreak Detection on Jailbreak data (70/30 stratified)](<https://www.sota2.com/research/sota/jailbreak-detection-on-jailbreak-data-70-30-stratified.md>): Jailbreak Detection; Jailbreak data (70/30 stratified).

- [GUI Navigation on OSWorld Verified](<https://www.sota2.com/research/sota/gui-navigation-on-osworld-verified.md>): GUI Navigation; OSWorld Verified.

- [Software Engineering on SWE-bench Verified (resolution rate)](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-resolution-rate.md>): Software Engineering; SWE-bench Verified.

- [Fluid Dynamics Simulation on Navier-Stokes (NS) nu=10^-5 at 64x64 unified-protocol (test)](<https://www.sota2.com/research/sota/fluid-dynamics-simulation-on-navier-stokes-ns-nu-10-5-at-64x64-unified-protocol-test.md>): Fluid Dynamics Simulation; Navier-Stokes (NS) nu=10^-5 at 64x64 unified-protocol (test).

- [Misalignment Detection on Agent-SafetyBench underspecified queries n=148](<https://www.sota2.com/research/sota/misalignment-detection-on-agent-safetybench-underspecified-queries-n-148.md>): Misalignment Detection; Agent-SafetyBench underspecified queries n=148.

- [Coding on Coding Real-data 20% verified](<https://www.sota2.com/research/sota/coding-on-coding-real-data-20-percent-verified.md>): Coding; Coding Real-data 20% verified.

- [Attributing Multimodal Foundation Model Errors on ImageNet misclassified samples (val)](<https://www.sota2.com/research/sota/attributing-multimodal-foundation-model-errors-on-imagenet-misclassified-samples-val.md>): Attributing Multimodal Foundation Model Errors; ImageNet misclassified samples (val).

- [Machine Reading Comprehension on CMRC Simplified Chinese 2018 (Challenge)](<https://www.sota2.com/research/sota/machine-reading-comprehension-on-cmrc-simplified-chinese-2018-challenge.md>): Machine Reading Comprehension; CMRC Simplified Chinese 2018 (Challenge).

- [Classification Attribution on ImageNet (misclassified samples)](<https://www.sota2.com/research/sota/classification-attribution-on-imagenet-misclassified-samples.md>): Classification attribution; ImageNet (misclassified samples).

- [Code Generation on TACO Verified](<https://www.sota2.com/research/sota/code-generation-on-taco-verified.md>): Code Generation; TACO Verified.

- [Machine Reading Comprehension on CMRC Simplified Chinese 2018 (test)](<https://www.sota2.com/research/sota/machine-reading-comprehension-on-cmrc-simplified-chinese-2018-test.md>): Machine Reading Comprehension; CMRC Simplified Chinese 2018 (test).

- [Predicting Artistic Success on WikiArt-Seq2Rank (stratified split subset)](<https://www.sota2.com/research/sota/predicting-artistic-success-on-wikiart-seq2rank-stratified-split-subset.md>): Predicting Artistic Success; WikiArt-Seq2Rank (stratified split subset).

- [Function-level Code Localization on SWE-bench Verified (Lite)](<https://www.sota2.com/research/sota/function-level-code-localization-on-swe-bench-verified-lite.md>): Function-level Code Localization; SWE-bench Verified (Lite).

- [File-level Code Localization on SWE-bench Verified Lite](<https://www.sota2.com/research/sota/file-level-code-localization-on-swe-bench-verified-lite.md>): File-level Code Localization; SWE-bench Verified Lite.

- [Uncertainty Quantification on Modified Shallow Water (MSW) Model QPEns Reference (full assimilation window)](<https://www.sota2.com/research/sota/uncertainty-quantification-on-modified-shallow-water-msw-model-qpens-reference-full-assimilation-window.md>): Uncertainty Quantification; Modified Shallow Water (MSW) Model QPEns Reference (full assimilation window).

- [Spreadsheet Task Execution on SpreadsheetBench Verified (Held-out)](<https://www.sota2.com/research/sota/spreadsheet-task-execution-on-spreadsheetbench-verified-held-out.md>): Spreadsheet Task Execution; SpreadsheetBench Verified (Held-out).

- [Multi-label Classification on 13 Multi-label Tabular Datasets (stratified 5-fold CV)](<https://www.sota2.com/research/sota/multi-label-classification-on-13-multi-label-tabular-datasets-stratified-5-fold-cv.md>): Multi-label Classification; 13 Multi-label Tabular Datasets (cal500, birds, enron, scene, yeast, Corel5k, rcv1subset1, rcv1subset2, rcv1subset3, bibtex, yahoo-Arts, yahoo-Business, mediamill) stratified five-fold cross-validation.

- [Embodied AI task execution on EmbodiedBench EB-ALFRED](<https://www.sota2.com/research/sota/embodied-ai-task-execution-on-embodiedbench-eb-alfred.md>): Embodied AI task execution; EmbodiedBench EB-ALFRED.

- [Vision-Language Multi-task Evaluation on MS COCO Unified Multi-task (test)](<https://www.sota2.com/research/sota/vision-language-multi-task-evaluation-on-ms-coco-unified-multi-task-test.md>): Vision-Language Multi-task Evaluation; MS COCO Unified Multi-task (test).

- [Question Answering with Abstention on Unified Abstention Benchmark Suite](<https://www.sota2.com/research/sota/question-answering-with-abstention-on-unified-abstention-benchmark-suite.md>): Question Answering with Abstention; Unified Abstention Benchmark Suite (MMLU, GSM-MC, UMWP, Knowledge Crosswords, HellaSwag, Quail, Misconceptions, Propaganda, BBQ).

- [Hallucination Correction on UniEdit](<https://www.sota2.com/research/sota/hallucination-correction-on-uniedit.md>): Hallucination Correction; UniEdit.

- [Computer task execution on OSWorld (verified)](<https://www.sota2.com/research/sota/computer-task-execution-on-osworld-verified.md>): Computer task execution; OSWorld (verified).

- [Prompt Injection Prevention on NQ simplified](<https://www.sota2.com/research/sota/prompt-injection-prevention-on-nq-simplified.md>): Prompt Injection Prevention; NQ simplified.

- [Code Localization on SWE-bench Verified (test)](<https://www.sota2.com/research/sota/code-localization-on-swe-bench-verified-test.md>): Code Localization; SWE-bench Verified (test).

- [Architecture performance prediction on NATS-Bench SSS ImageNet16-120 (stratified subset)](<https://www.sota2.com/research/sota/architecture-performance-prediction-on-nats-bench-sss-imagenet16-120-stratified-subset.md>): Architecture performance prediction; NATS-Bench SSS ImageNet16-120 (stratified subset).

- [Architecture performance prediction on NATS-Bench SSS CIFAR-100 (stratified subset)](<https://www.sota2.com/research/sota/architecture-performance-prediction-on-nats-bench-sss-cifar-100-stratified-subset.md>): Architecture performance prediction; NATS-Bench SSS CIFAR-100 (stratified subset).

- [Architecture performance prediction on NATS-Bench SSS CIFAR-10 (stratified subset)](<https://www.sota2.com/research/sota/architecture-performance-prediction-on-nats-bench-sss-cifar-10-stratified-subset.md>): Architecture performance prediction; NATS-Bench SSS CIFAR-10 (stratified subset).

- [Image Generation on WISE-Verified](<https://www.sota2.com/research/sota/image-generation-on-wise-verified.md>): Image Generation; WISE-Verified.

- [Software Engineering on SWE-bench Verified (Pass@1 and Time Metrics)](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-pass-at-1-and-time-metrics.md>): Software Engineering; SWE-bench Verified.

- [Vision-Language Navigation on EmbodiedBench Navigation latest (test)](<https://www.sota2.com/research/sota/vision-language-navigation-on-embodiedbench-navigation-latest-test.md>): Vision-Language Navigation; EmbodiedBench Navigation latest (test).

- [Downstream Classification on CIFAR-10 modified (test)](<https://www.sota2.com/research/sota/downstream-classification-on-cifar-10-modified-test.md>): Downstream Classification; CIFAR-10 modified (test).

- [File-level localization on SWE-Bench Verified](<https://www.sota2.com/research/sota/file-level-localization-on-swe-bench-verified.md>): File-level localization; SWE-Bench Verified.

- [Novel View Synthesis on Unspecified Dataset (train)](<https://www.sota2.com/research/sota/novel-view-synthesis-on-unspecified-dataset-train.md>): Novel View Synthesis; Unspecified Dataset (train).

- [Action Recognition on HDM05 (5-fold stratified cross-validation)](<https://www.sota2.com/research/sota/action-recognition-on-hdm05-5-fold-stratified-cross-validation.md>): Action Recognition; HDM05 (5-fold stratified cross-validation).

- [Code Agent on SWE-Bench Verified](<https://www.sota2.com/research/sota/code-agent-on-swe-bench-verified.md>): Code Agent; SWE-Bench Verified.

- [Question Answering on TriviaQA Wiki domain, Verified (dev)](<https://www.sota2.com/research/sota/question-answering-on-triviaqa-wiki-domain-verified-dev.md>): Question Answering; TriviaQA Wiki domain, Verified (dev).

- [Software Engineering on SWE-Bench Verified (Pass %, Out tok, Energy)](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-pass-percent-out-tok-energy.md>): Software Engineering; SWE-Bench Verified.

- [Misclassification Explanation Evaluation on ImageNet-1K misclassified samples 1.0 (test)](<https://www.sota2.com/research/sota/misclassification-explanation-evaluation-on-imagenet-1k-misclassified-samples-1-0-test.md>): Misclassification Explanation Evaluation; ImageNet-1K misclassified samples 1.0 (test).

- [Interactive Optimization on SFUSD Simplified 10-school problem](<https://www.sota2.com/research/sota/interactive-optimization-on-sfusd-simplified-10-school-problem.md>): Interactive Optimization; SFUSD Simplified 10-school problem.

- [Reward Modeling on RewardBench unified-feedback (test)](<https://www.sota2.com/research/sota/reward-modeling-on-rewardbench-unified-feedback-test.md>): Reward Modeling; RewardBench unified-feedback (test).

- [Software Engineering Issue Resolving on SWE-bench Verified](<https://www.sota2.com/research/sota/software-engineering-issue-resolving-on-swe-bench-verified.md>): Software Engineering / Issue Resolving; SWE-bench Verified.

- [Comparative Ranking on Unified Evaluation v1 (aggregate)](<https://www.sota2.com/research/sota/comparative-ranking-on-unified-evaluation-v1-aggregate.md>): Comparative Ranking; Unified Evaluation v1 (aggregate).

- [Taste Prediction on Unified music-taste corpus (test)](<https://www.sota2.com/research/sota/taste-prediction-on-unified-music-taste-corpus-test.md>): Taste Prediction; Unified music-taste corpus (test).

- [Spreadsheet Reasoning on SpreadsheetBench Verified](<https://www.sota2.com/research/sota/spreadsheet-reasoning-on-spreadsheetbench-verified.md>): Spreadsheet Reasoning; SpreadsheetBench Verified.

- [Prompt-level detection on matched-PP benchmark α = 1 (5-fold stratified CV)](<https://www.sota2.com/research/sota/prompt-level-detection-on-matched-pp-benchmark-1-5-fold-stratified-cv.md>): Prompt-level detection; matched-PP benchmark α = 1 (5-fold stratified CV).

- [Point Tracking on RGB-Stacking queried-first mode](<https://www.sota2.com/research/sota/point-tracking-on-rgb-stacking-queried-first-mode.md>): Point Tracking; RGB-Stacking queried-first mode.

- [Question Answering on SimpleQA-verified OOD](<https://www.sota2.com/research/sota/question-answering-on-simpleqa-verified-ood.md>): Question Answering; SimpleQA-verified OOD.

- [Software Engineering on SWE-Bench Verified (Pass@K)](<https://www.sota2.com/research/sota/software-engineering-on-swe-bench-verified-pass-at-k.md>): Software engineering; SWE-Bench Verified.

- [Anomaly Detection on MVTec-AD Unified setting (Per-Category AUROC)](<https://www.sota2.com/research/sota/anomaly-detection-on-mvtec-ad-unified-setting-per-category-auroc.md>): Anomaly Detection; MVTec-AD Unified setting.

- [Machine Reading Comprehension on CMRC Simplified Chinese 2018 (dev)](<https://www.sota2.com/research/sota/machine-reading-comprehension-on-cmrc-simplified-chinese-2018-dev.md>): Machine Reading Comprehension; CMRC Simplified Chinese 2018 (dev).

- [Watermark Detection on C4 Unmodified](<https://www.sota2.com/research/sota/watermark-detection-on-c4-unmodified.md>): Watermark Detection; C4 Unmodified.

- [Agentic Coding on SWE-Bench Verified (Pass@1)](<https://www.sota2.com/research/sota/agentic-coding-on-swe-bench-verified-pass-at-1.md>): Agentic Coding; SWE-Bench Verified.

- [Pasture Biomass Estimation on CSIRO n=357 (5-fold stratified group cross-validation)](<https://www.sota2.com/research/sota/pasture-biomass-estimation-on-csiro-n-357-5-fold-stratified-group-cross-validation.md>): Pasture Biomass Estimation; CSIRO n=357 (5-fold stratified group cross-validation).