# Scene Graph Generation on PSG v1.0 (test) Source: [Scene Graph Generation on PSG v1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-05-14T09:46:24.599Z Evaluation of Scene Graph Generation performance on the test split of the PSG v1.0 dataset, reporting object/relation precision, recall, F1 scores (strict and soft), generation time, and overall SGG scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Scene Graph Generation | | Dataset (including any stated split) | PSG v1.0 (test) | | Metric | Failure Rate | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Failure Rate](): Lower is better; unit: PERCENT. - [Generation Time (s)](): Lower is better; unit: TIME. - [Object Precision (Strict)](): Higher is better; unit: PERCENT. - [Object Precision (Soft)](): Higher is better; unit: PERCENT. - [Object Recall (Strict)](): Higher is better; unit: PERCENT. - [Object Recall (Soft)](): Higher is better; unit: PERCENT. - [Object F1 (Strict)](): Higher is better; unit: PERCENT. - [Object F1 (Soft)](): Higher is better; unit: PERCENT. - [Relation Precision (Strict)](): Higher is better; unit: PERCENT. - [Relation Precision (Soft)](): Higher is better; unit: PERCENT. - [Relation Recall (Strict)](): Higher is better; unit: PERCENT. - [Relation Recall (Soft)](): Higher is better; unit: PERCENT. - [Relation F1 (Strict)](): Higher is better; unit: PERCENT. - [Relation F1 (Soft)](): Higher is better; unit: PERCENT. - [SGG Score (Strict)](): Higher is better; unit: PERCENT. - [SGG Score (Soft)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Failure Rate | Result date | Report paper | | --- | --- | --- | --- | | SceneGraphVLM | 0 | Not recorded | 6a055729dbc836b87d828da5 | | SceneGraphVLM | 0 | Not recorded | 6a055729dbc836b87d828da5 | | Gemini-3-flash | 0.09 | Not recorded | 6a055729dbc836b87d828da5 | | R1-SGG | 0.14 | Not recorded | 6a055729dbc836b87d828da5 | | GPT-5-mini | 0.32 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL3.5-1B-Instruct (full finetune LLM head) | 0.37 | Not recorded | 6a055729dbc836b87d828da5 | | GPT-5.4-nano | 0.64 | Not recorded | 6a055729dbc836b87d828da5 | | DeepSeek-VL2-tiny-3B | 0.64 | Not recorded | 6a055729dbc836b87d828da5 | | GPT-5.4-mini | 0.74 | Not recorded | 6a055729dbc836b87d828da5 | | GPT-5-nano | 0.78 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL3-1B (full finetune LLM head) | 0.78 | Not recorded | 6a055729dbc836b87d828da5 | | SceneGraphVLM (InternVL2.5-1B) | 1.15 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL3-1B (full finetune LLM head) | 1.24 | Not recorded | 6a055729dbc836b87d828da5 | | SceneGraphVLM (InternVL2.5-1B) | 3.45 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL2.5-1B | 6.85 | Not recorded | 6a055729dbc836b87d828da5 | | Qwen3.5-4B | 7.58 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL3.5-1B-Instruct (LoRA r=16, α=32) | 10.34 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL3-1B | 11.4 | Not recorded | 6a055729dbc836b87d828da5 | | Qwen3.5-2B | 14.61 | Not recorded | 6a055729dbc836b87d828da5 | | InternVL3.5-1B | 23.81 | Not recorded | 6a055729dbc836b87d828da5 | | Qwen3-VL-4B | 45.37 | Not recorded | 6a055729dbc836b87d828da5 | | Qwen3-VL-2B | 50.09 | Not recorded | 6a055729dbc836b87d828da5 | | Qwen3.5-0.8B | 67.92 | Not recorded | 6a055729dbc836b87d828da5 | | Ovis2.5-2B-Instruct | 75.97 | Not recorded | 6a055729dbc836b87d828da5 | | SmolVLM-500M-Instruct | 86.08 | Not recorded | 6a055729dbc836b87d828da5 | ### SceneGraphVLM Failure Rate: 0. Result ID: 6a0599ee47f34ce924ed0c78. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT close"}. This record does not assert independent reproduction. ### SceneGraphVLM Failure Rate: 0. Result ID: 6a0599ee47f34ce924ed0c79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT + RL close"}. This record does not assert independent reproduction. ### Gemini-3-flash Failure Rate: 0.09. Result ID: 6a0599ee47f34ce924ed0c65. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### R1-SGG Failure Rate: 0.14. Result ID: 6a0599ee47f34ce924ed0c77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT+RL close"}. This record does not assert independent reproduction. ### GPT-5-mini Failure Rate: 0.32. Result ID: 6a0599ee47f34ce924ed0c63. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### InternVL3.5-1B-Instruct (full finetune LLM head) Failure Rate: 0.37. Result ID: 6a0599ee47f34ce924ed0c74. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT close"}. This record does not assert independent reproduction. ### GPT-5.4-nano Failure Rate: 0.64. Result ID: 6a0599ee47f34ce924ed0c61. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### DeepSeek-VL2-tiny-3B Failure Rate: 0.64. Result ID: 6a0599ee47f34ce924ed0c68. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### GPT-5.4-mini Failure Rate: 0.74. Result ID: 6a0599ee47f34ce924ed0c64. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### GPT-5-nano Failure Rate: 0.78. Result ID: 6a0599ee47f34ce924ed0c62. [Reporting paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### InternVL3-1B (full finetune LLM head) Failure Rate: 0.78. Result ID: 6a0599ee47f34ce924ed0c73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT close"}. This record does not assert independent reproduction. ### SceneGraphVLM (InternVL2.5-1B) Failure Rate: 1.15. Result ID: 6a0599ee47f34ce924ed0c76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT+RL close"}. This record does not assert independent reproduction. ### InternVL3-1B (full finetune LLM head) Failure Rate: 1.24. Result ID: 6a0599ee47f34ce924ed0c72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT open"}. This record does not assert independent reproduction. ### SceneGraphVLM (InternVL2.5-1B) Failure Rate: 3.45. Result ID: 6a0599ee47f34ce924ed0c75. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT close"}. This record does not assert independent reproduction. ### InternVL2.5-1B Failure Rate: 6.85. Result ID: 6a0599ee47f34ce924ed0c67. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### Qwen3.5-4B Failure Rate: 7.58. Result ID: 6a0599ee47f34ce924ed0c70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### InternVL3.5-1B-Instruct (LoRA r=16, α=32) Failure Rate: 10.34. Result ID: 6a0599ee47f34ce924ed0c71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"SFT close"}. This record does not assert independent reproduction. ### InternVL3-1B Failure Rate: 11.4. Result ID: 6a0599ee47f34ce924ed0c69. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### Qwen3.5-2B Failure Rate: 14.61. Result ID: 6a0599ee47f34ce924ed0c6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### InternVL3.5-1B Failure Rate: 23.81. Result ID: 6a0599ee47f34ce924ed0c6c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### Qwen3-VL-4B Failure Rate: 45.37. Result ID: 6a0599ee47f34ce924ed0c6d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### Qwen3-VL-2B Failure Rate: 50.09. Result ID: 6a0599ee47f34ce924ed0c6e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### Qwen3.5-0.8B Failure Rate: 67.92. Result ID: 6a0599ee47f34ce924ed0c6b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### Ovis2.5-2B-Instruct Failure Rate: 75.97. Result ID: 6a0599ee47f34ce924ed0c6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. ### SmolVLM-500M-Instruct Failure Rate: 86.08. Result ID: 6a0599ee47f34ce924ed0c66. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mode":"Base"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.