跳到主要导航 跳到搜索 跳到主要内容

VarGAN: Adversarial Learning of Variable Semantic Representations

  • Yalan Lin
  • , Chengcheng Wan
  • , Shuwen Bai
  • , Xiaodong Gu*
  • *此作品的通讯作者
  • Shanghai Jiao Tong University
  • East China University of Science and Technology

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Variable names are of critical importance in code representation learning. However, due to diverse naming conventions, variables often receive arbitrary names, leading to long-tail, out-of-vocabulary (OOV), and other well-known problems. While the Byte-Pair Encoding (BPE) tokenizer has addressed the surface-level recognition of low-frequency tokens, it has not noticed the inadequate training of low-frequency identifiers by code representation models, resulting in an imbalanced distribution of rare and common identifiers. Consequently, code representation models struggle to effectively capture the semantics of low-frequency variable names. In this paper, we propose VarGAN, a novel method for variable name representations. VarGAN strengthens the training of low-frequency variables through adversarial training. Specifically, we regard the code representation model as a generator responsible for producing vectors from source code. Additionally, we employ a discriminator that detects whether the code input to the generator contains low-frequency variables. This adversarial setup regularizes the distribution of rare variables, making them overlap with their corresponding high-frequency counterparts in the vector space. Experimental results demonstrate that VarGAN empowers CodeBERT to generate code vectors that exhibit more uniform distribution for both low-and high-frequency identifiers. There is an improvement of 8% in similarity and relatedness scores compared to VarCLR in the IdBench benchmark. VarGAN is also validated in downstream tasks, where it exhibits enhanced capabilities in capturing token-and code-level semantics.

源语言英语
页(从-至)1505-1517
页数13
期刊IEEE Transactions on Software Engineering
卷50
期6
DOI
出版状态已出版 - 1 6月 2024

学术指纹

探究 'VarGAN: Adversarial Learning of Variable Semantic Representations' 的科研主题。它们共同构成独一无二的学术指纹。

引用此