Dependent-test-aware regression testing techniques
Wing Lam, August Shi, Reed Oei, Sai Zhang, Michael D. Ernst, Tao Xie
Abstract
Developers typically rely on regression testing techniques to ensure that their changes do not break existing functionality. Unfortunately, these techniques suffer from flaky tests, which can both pass and fail when run multiple times on the same version of code and tests. One prominent type of flaky tests is order-dependent (OD) tests, which are tests that pass when run in one order but fail when run in another order. Although OD tests may cause flaky-test failures, OD tests can help developers run their tests faster by allowing them to share resources. We propose to make regression testing techniques dependent-test-aware to reduce flaky-test failures.
Topics & Concepts
Regression testingComputer scienceReliability engineeringTest (biology)White-box testingRegressionTest strategyCode (set theory)Test Management ApproachSoftwareStatisticsProgramming languageEngineeringSoftware developmentMathematicsSet (abstract data type)Software constructionBiologyPaleontologySoftware Testing and Debugging TechniquesSoftware Reliability and Analysis ResearchSoftware Engineering Research