Research domain
Code & Developer AI
CFG Recovery and Library Detection16 benchmarks · 1 papersLarge Language Model Inference16 benchmarks · 14 papersCode Search16 benchmarks · 10 papersSingle-deletion-correcting code construction15 benchmarks · 1 papersKernel Generation15 benchmarks · 5 papersTriton kernel generation15 benchmarks · 4 papersData Contamination Detection15 benchmarks · 6 papersAutoformalization15 benchmarks · 8 papersIssue Resolution14 benchmarks · 5 papersCode Retrieval14 benchmarks · 10 papersMulti-turn Text-to-SQL14 benchmarks · 5 papersCode Repair14 benchmarks · 9 papersBlock Coverage14 benchmarks · 1 papersDownstream Task Evaluation13 benchmarks · 14 papersMultimodal Code Generation13 benchmarks · 4 papersBinary Analysis13 benchmarks · 1 papersGenerative Code Completion13 benchmarks · 1 paperstext-to-CAD generation12 benchmarks · 9 papersSQL Generation12 benchmarks · 11 papersCode Execution12 benchmarks · 6 papersFew-shot Learning12 benchmarks · 13 papersHardware Security Assertion and Flaw Detection11 benchmarks · 1 papersVerilog Code Generation11 benchmarks · 5 papersBad Practice Detection11 benchmarks · 1 papersAutomated Program Repair11 benchmarks · 4 papersSmart contract vulnerability detection11 benchmarks · 6 papersAutonomous Penetration Testing11 benchmarks · 2 papersAuto-formalization11 benchmarks · 2 papersAutonomous LLM Fine-tuning10 benchmarks · 1 papersRegex Generation10 benchmarks · 1 papersFill-in-the-Middle Code Completion10 benchmarks · 2 papersSoftware Engineering Issue Resolution10 benchmarks · 11 papers