List your company
ResearchFormerly WizWand

Student Simulation on Java 5

84AccuracyGPT-4o
27.8442.425771.58May 26, 2025
Updated 8mo ago

Evaluation results

MethodLinks
GPT-4oBehavior Prediction=Prototype Mapping, Solution Simulation=IO2025.05843.83.24
GPT-4oBehavior Prediction=Prototype Mapping, Solution Simulation=CoT2025.05843.83.38
GPT-4oBehavior Prediction=Prototype Mapping, Solution Simulation=Refine2025.05843.83.46
LLaMA-3.3-70B-InstructBehavior Prediction=Prototype Mapping, Solution Simulation=IO2025.05683.32.78
LLaMA-3.3-70B-InstructBehavior Prediction=Prototype Mapping, Solution Simulation=CoT2025.05683.32.82
LLaMA-3.3-70B-InstructBehavior Prediction=Prototype Mapping, Solution Simulation=Refine2025.05683.32.84
GPT-4oBehavior Prediction=Level+Random, Solution Simulation=IO2025.05663.562.44
GPT-4oBehavior Prediction=Level+Random, Solution Simulation=CoT2025.05663.562.64
GPT-4oBehavior Prediction=Level+Random, Solution Simulation=Refine2025.05663.562.4
Claude-3.5-SonnetBehavior Prediction=Prototype Mapping, Solution Simulation=IO2025.05563.32.68
Claude-3.5-SonnetBehavior Prediction=Prototype Mapping, Solution Simulation=CoT2025.05563.32.44
Claude-3.5-SonnetBehavior Prediction=Prototype Mapping, Solution Simulation=Refine2025.05563.32.9
GPT-4oBehavior Prediction=Level+Similarity, Solution Simulation=IO2025.05563.282.24
GPT-4oBehavior Prediction=Level+Similarity, Solution Simulation=CoT2025.05563.282.52
GPT-4oBehavior Prediction=Level+Similarity, Solution Simulation=Refine2025.05563.282.1
Claude-3.5-SonnetBehavior Prediction=Similarity, Solution Simulation=IO2025.05543.042.66
Claude-3.5-SonnetBehavior Prediction=Similarity, Solution Simulation=CoT2025.05543.042.8
Claude-3.5-SonnetBehavior Prediction=Similarity, Solution Simulation=Refine2025.05543.042.76
GPT-3.5Behavior Prediction=Prototype Mapping, Solution Simulation=IO2025.05522.922.68
GPT-3.5Behavior Prediction=Prototype Mapping, Solution Simulation=CoT2025.05522.922.98
GPT-3.5Behavior Prediction=Prototype Mapping, Solution Simulation=Refine2025.05522.923.28
GPT-4oBehavior Prediction=Similarity, Solution Simulation=IO2025.05502.92.74
GPT-4oBehavior Prediction=Similarity, Solution Simulation=CoT2025.05502.92.78
GPT-4oBehavior Prediction=Similarity, Solution Simulation=Refine2025.05502.92.74
LLaMA-3.3-70B-InstructBehavior Prediction=Random, Solution Simulation=IO2025.05462.52.34
LLaMA-3.3-70B-InstructBehavior Prediction=Random, Solution Simulation=CoT2025.05462.52.4
LLaMA-3.3-70B-InstructBehavior Prediction=Random, Solution Simulation=Refine2025.05462.51.98
Claude-3.5-SonnetBehavior Prediction=Random, Solution Simulation=IO2025.05462.882.46
Claude-3.5-SonnetBehavior Prediction=Random, Solution Simulation=CoT2025.05462.882.46
Claude-3.5-SonnetBehavior Prediction=Random, Solution Simulation=Refine2025.05462.882.8
Claude-3.5-SonnetBehavior Prediction=Level+Random, Solution Simulation=IO2025.05462.482.24
Claude-3.5-SonnetBehavior Prediction=Level+Random, Solution Simulation=CoT2025.05462.482
Claude-3.5-SonnetBehavior Prediction=Level+Random, Solution Simulation=Refine2025.05462.482.36
Claude-3.5-SonnetBehavior Prediction=Level+Similarity, Solution Simulation=IO2025.05462.562.5
Claude-3.5-SonnetBehavior Prediction=Level+Similarity, Solution Simulation=CoT2025.05462.562.42
Claude-3.5-SonnetBehavior Prediction=Level+Similarity, Solution Simulation=Refine2025.05462.562.4
GPT-3.5Behavior Prediction=Random, Solution Simulation=IO2025.05462.362.44
GPT-3.5Behavior Prediction=Random, Solution Simulation=CoT2025.05462.362.78
GPT-3.5Behavior Prediction=Random, Solution Simulation=Refine2025.05462.363.22
GPT-3.5Behavior Prediction=Level, Solution Simulation=IO2025.05462.742.52
GPT-3.5Behavior Prediction=Level, Solution Simulation=CoT2025.05462.742.84
GPT-3.5Behavior Prediction=Level, Solution Simulation=Refine2025.05462.742.86
GPT-3.5Behavior Prediction=Level+Random, Solution Simulation=IO2025.05462.52.54
GPT-3.5Behavior Prediction=Level+Random, Solution Simulation=CoT2025.05462.52.24
GPT-3.5Behavior Prediction=Level+Random, Solution Simulation=Refine2025.05462.53.18
GPT-3.5Behavior Prediction=Level+Similarity, Solution Simulation=IO2025.05422.482.42
GPT-3.5Behavior Prediction=Level+Similarity, Solution Simulation=CoT2025.05422.482.44
GPT-3.5Behavior Prediction=Level+Similarity, Solution Simulation=Refine2025.05422.482.96
GPT-4oBehavior Prediction=Level, Solution Simulation=IO2025.05422.342.28
GPT-4oBehavior Prediction=Level, Solution Simulation=CoT2025.05422.342.8
GPT-4oBehavior Prediction=Level, Solution Simulation=Refine2025.05422.341.9
LLaMA-3.3-70B-InstructBehavior Prediction=Similarity, Solution Simulation=IO2025.05342.322.2
LLaMA-3.3-70B-InstructBehavior Prediction=Similarity, Solution Simulation=CoT2025.05342.322.34
LLaMA-3.3-70B-InstructBehavior Prediction=Similarity, Solution Simulation=Refine2025.05342.322.04
LLaMA-3.3-70B-InstructBehavior Prediction=Level, Solution Simulation=IO2025.05342.182.74
LLaMA-3.3-70B-InstructBehavior Prediction=Level, Solution Simulation=CoT2025.05342.182.14
LLaMA-3.3-70B-InstructBehavior Prediction=Level, Solution Simulation=Refine2025.05342.181.66
LLaMA-3.3-70B-InstructBehavior Prediction=Level+Similarity, Solution Simulation=IO2025.05342.342.28
LLaMA-3.3-70B-InstructBehavior Prediction=Level+Similarity, Solution Simulation=CoT2025.05342.342.48
LLaMA-3.3-70B-InstructBehavior Prediction=Level+Similarity, Solution Simulation=Refine2025.05342.342.16
GPT-3.5Behavior Prediction=Similarity, Solution Simulation=IO2025.05342.442.56
GPT-3.5Behavior Prediction=Similarity, Solution Simulation=CoT2025.05342.442.44
GPT-3.5Behavior Prediction=Similarity, Solution Simulation=Refine2025.05342.443.02
LLaMA-3.3-70B-InstructBehavior Prediction=Level+Random, Solution Simulation=IO2025.05321.881.86
LLaMA-3.3-70B-InstructBehavior Prediction=Level+Random, Solution Simulation=CoT2025.05321.882.18
LLaMA-3.3-70B-InstructBehavior Prediction=Level+Random, Solution Simulation=Refine2025.05321.882
Claude-3.5-SonnetBehavior Prediction=Level, Solution Simulation=IO2025.05301.81.86
Claude-3.5-SonnetBehavior Prediction=Level, Solution Simulation=CoT2025.05301.81.58
Claude-3.5-SonnetBehavior Prediction=Level, Solution Simulation=Refine2025.05301.81.32
GPT-4oBehavior Prediction=Random, Solution Simulation=IO2025.05302.482.28
GPT-4oBehavior Prediction=Random, Solution Simulation=CoT2025.05302.482.5
GPT-4oBehavior Prediction=Random, Solution Simulation=Refine2025.05302.482.6