qyliang/DivoT5-60M

Directional Diffusion-Style Code Editing Pre-training

We propose a directional diffusion technique at the data level of code evolution and apply it within an Encoder-Decoder framework.

The directional diffusion can be mainly divided into the following two steps:

Noising process at the data level.

• Seting the starting point to old code.
• Adding artificial noise to perturb the old data.
• Exploring the intermediate step in the real evolution process.

Denoising process within the Encoder-Decoder framework.

• Using the auto-regressive nature of the Encoder-Decoder framework
• Perceiving the evolutionary direction in generation.

@article{liang2025directional,
  title={Directional Diffusion-Style Code Editing Pre-training},
  author={Liang, Qingyuan and Sun, Zeyu and Zhu, Qihao and Hu, Junhao and Zhao, Yifan and Chen, Yizhou and Zhu, Mingxuan and Wang, Guoqing and Zhang, Lu},
  journal={arXiv preprint arXiv:2501.12079},
  year={2025}
}