Preprints‎ > ‎

A Study on the Interplay between Pull Request Review and Continuous Integration Builds by Fiorella Zampetti, Gabriele Bavota, Gerardo Canfora and Massimiliano Di Penta

pubblicato 11 feb 2019, 12:38 da Gerardo Canfora
Modern code review (MCR) is nowadays well- adopted in industrial and open source projects. Recent studies have investigated how developers perceive its ability to foster code quality, developers' code ownership, and team building. MCR is often being used with automated quality checks through static analysis tools, testing or, ultimately, through automated builds on a Continuous Integration (CI) infrastructure. With the aim of understanding how developers use the outcome of CI builds during code review and, more specifically, during the discussion of pull requests, this paper empirically investigates the interplay between pull request discussion and the use of CI by means of 64,865 pull request discussions belonging to 69 open source projects. After having analyzed to what extent a build outcome influences the pull request merger, we qualitatively analyze the content of 857 pull request discussions. Also, we complement such an analysis with a survey involving 13 developers. While pull requests with passed build have a higher chance of being merged than failed ones, and while survey participants confirmed this quantitative finding, other process-related factors play a more important role in the pull request merge decision. Also, the survey participants point out cases where a pull request can be merged in presence of a CI failure, e.g., when a new pull request is opened to cope with the failure, when the failure is due to minor static analysis warnings. The study also indicates that CI introduces extra complexity, as in many pull requests developers have to solve non-trivial CI configuration issues.
Proc. of 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER'19) Zhejiang University, Hangzhou, China - February 24-27, 2019
Gerardo Canfora,
11 feb 2019, 12:38