Skip to main navigation Skip to search Skip to main content

Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers

  • Yuling Shi
  • , Hongyu Zhang
  • , Chengcheng Wan
  • , Xiaodong Gu*
  • *Corresponding author for this work
  • Shanghai Jiao Tong University
  • Chongqing University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Large language models have catalyzed an unprece-dented wave in code generation. While achieving significant advances, they blur the distinctions between machine- and human-authored source code, causing integrity and authenticity issues of software artifacts. Previous methods such as DetectGPthave proven effective in discerning machine-generated texts, but they do not identify and harness the unique patterns of machine-generated code. Thus, its applicability falters when applied to code. In this paper, we carefully study the specific patterns that characterize machine- and human-authored code. Through a rigorous analysis of code attributes such as lexical diversity, conciseness, and naturalness, we expose unique patterns inherent to each source. We particularly notice that the syntactic segmentation of code is a critical factor in identifying its provenance. Based on our findings, we propose DetectCodeGPT, a novel method for detecting machine-generated code, which improves DetectGPT by capturing the distinct stylized patterns of code. Diverging from conventional techniques that depend on external LLMs for perturbations, DetectCodeGPT perturbs the code corpus by strategically inserting spaces and newlines, ensuring both efficacy and efficiency. Experiment results show that our approach significantly outperforms state-of-the-art techniques in detecting machine-generated code.

Original languageEnglish
Title of host publicationProceedings - 2025 IEEE/ACM 47th International Conference on Software Engineering, ICSE 2025
PublisherIEEE Computer Society
Pages1628-1639
Number of pages12
ISBN (Electronic)9798331505691
DOIs
StatePublished - 2025
Event47th IEEE/ACM International Conference on Software Engineering, ICSE 2025 - Ottawa, Canada
Duration: 27 Apr 2025 → 3 May 2025

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Conference

Conference47th IEEE/ACM International Conference on Software Engineering, ICSE 2025
Country/TerritoryCanada
CityOttawa
Period27/04/25 → 3/05/25

Keywords

  • code generation
  • empirical analysis
  • large language models
  • machine-generated code detection

Fingerprint

Dive into the research topics of 'Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers'. Together they form a unique fingerprint.

Cite this