MAF-CPR: LLM-Based Multi-agent Framework for Complex Pull Request Review in GitHub

Pull request (PR) is essential for collaboration in open-source development, as it facilitates the review and integration of code changes efficiently, ensuring quality and coordination among contributors through the processes of PR review. Large language models (LLMs) have proven effective in supporting code reviewers, they still encounter significant challenges when processing complex PR. A single model has difficulty capturing key information when addressing complex PRs that encompass extensive descriptions, substantial code changes, and associated issues. To address these challenges, we propose MAF-CPR, a novel LLM-based multi-agent framework for automated review of complex pull requests on GitHub. Inspired by the real-world PR handling process, the framework consists of four specialized agents: the Repository Manager, PR Analyzer, Issue Tracker, and Code Reviewer. To enhance coordination and context-awareness, we further introduce a dynamic prompt refinement mechanism that adapts each agent’s prompt based on the evolving context within the multi-agent workflow. Experiments have demonstrated that the proposed multi-agent framework outperforms LLMs like GPT-3.5, GPT-4, and Claude-3-Sonnet in four tasks. Further analysis shows that our proposed agents and collaboration process benefit the model's understanding of PR and code change.