C-3PR: A Bot for Fixing Static Analysis Violations via Pull Requests

Abstract

Static analysis tools are frequently used to detect common programming mistakes or bad practices. Yet, the existing literature reports that these tools are still underused in the industry, which is partly due to (1) the frequent high number of false positives generated, (2) the lack of automated repairing solutions, and (3) the possible mismatches between tools and workflows of development teams. In this study we explored the question: “How could a bot-based approach allow seamless integration of static analysis tools into developers' workflows?” To this end we introduce C-3PR, an event-based bot infrastructure that automatically proposes fixes to static analysis violations through pull requests (PRs). We have been using C-3PR in an industrial setting for a period of eight months. To evaluate C-3PR usefulness, we monitored its operation in response to 2179 commits to the code base of the tracked projects. The bot autonomously executed 201346 analyses, yielding 610 pull requests. Among them, 346 (57%) were merged into the projects' code bases. We observed that, on average, these PRs are evaluated faster than general-purpose PRs (2.58 and 5.78 business days, respectively). Accepted transformations take even shorter time (1.56 days). Among the reasons for rejection, bugs in C-3PR and in the tools it uses are the most common ones. PRs that require the resolution of a merge conflict are almost always rejected as well. We also conducted a focus group to assess how C-3PR affected the development workflow. We observed that developers perceived C-3PR as efficient, reliable, and useful. For instance, the participants mentioned that, given the chance, they would keep using C-3PR. Our findings bring new evidence that a bot-based infrastructure could mitigate some challenges that hinder the wide adoption of static analysis tools.

Topics & Concepts