TY - GEN
T1 - Between Lines of Code
T2 - 47th IEEE/ACM International Conference on Software Engineering, ICSE 2025
AU - Shi, Yuling
AU - Zhang, Hongyu
AU - Wan, Chengcheng
AU - Gu, Xiaodong
N1 - Publisher Copyright:
© 2025 IEEE.
PY - 2025
Y1 - 2025
N2 - Large language models have catalyzed an unprece-dented wave in code generation. While achieving significant advances, they blur the distinctions between machine- and human-authored source code, causing integrity and authenticity issues of software artifacts. Previous methods such as DetectGPthave proven effective in discerning machine-generated texts, but they do not identify and harness the unique patterns of machine-generated code. Thus, its applicability falters when applied to code. In this paper, we carefully study the specific patterns that characterize machine- and human-authored code. Through a rigorous analysis of code attributes such as lexical diversity, conciseness, and naturalness, we expose unique patterns inherent to each source. We particularly notice that the syntactic segmentation of code is a critical factor in identifying its provenance. Based on our findings, we propose DetectCodeGPT, a novel method for detecting machine-generated code, which improves DetectGPT by capturing the distinct stylized patterns of code. Diverging from conventional techniques that depend on external LLMs for perturbations, DetectCodeGPT perturbs the code corpus by strategically inserting spaces and newlines, ensuring both efficacy and efficiency. Experiment results show that our approach significantly outperforms state-of-the-art techniques in detecting machine-generated code.
AB - Large language models have catalyzed an unprece-dented wave in code generation. While achieving significant advances, they blur the distinctions between machine- and human-authored source code, causing integrity and authenticity issues of software artifacts. Previous methods such as DetectGPthave proven effective in discerning machine-generated texts, but they do not identify and harness the unique patterns of machine-generated code. Thus, its applicability falters when applied to code. In this paper, we carefully study the specific patterns that characterize machine- and human-authored code. Through a rigorous analysis of code attributes such as lexical diversity, conciseness, and naturalness, we expose unique patterns inherent to each source. We particularly notice that the syntactic segmentation of code is a critical factor in identifying its provenance. Based on our findings, we propose DetectCodeGPT, a novel method for detecting machine-generated code, which improves DetectGPT by capturing the distinct stylized patterns of code. Diverging from conventional techniques that depend on external LLMs for perturbations, DetectCodeGPT perturbs the code corpus by strategically inserting spaces and newlines, ensuring both efficacy and efficiency. Experiment results show that our approach significantly outperforms state-of-the-art techniques in detecting machine-generated code.
KW - code generation
KW - empirical analysis
KW - large language models
KW - machine-generated code detection
UR - https://www.scopus.com/pages/publications/105010337050
U2 - 10.1109/ICSE55347.2025.00005
DO - 10.1109/ICSE55347.2025.00005
M3 - 会议稿件
AN - SCOPUS:105010337050
T3 - Proceedings - International Conference on Software Engineering
SP - 1628
EP - 1639
BT - Proceedings - 2025 IEEE/ACM 47th International Conference on Software Engineering, ICSE 2025
PB - IEEE Computer Society
Y2 - 27 April 2025 through 3 May 2025
ER -