Preempting flaky tests via non-idempotent-outcome tests
Anjiang Wei, Pu Yi, Zhengxi Li, Tao Xie, Darko Marinov, Wing Lam
Abstract
Regression testing can greatly help in software development, but it can be seriously undermined by flaky tests, which can both pass and fail, seemingly nondeterministically, on the same code commit. Flaky tests are an emerging topic in both research and industry. Prior work has identified multiple categories of flaky tests, developed techniques for detecting these flaky tests, and analyzed some detected flaky tests.
Topics & Concepts
CommitRegression testingComputer scienceOutcome (game theory)Code (set theory)Work (physics)SoftwareReliability engineeringProgramming languageSoftware developmentEngineeringDatabaseMathematicsSoftware constructionSet (abstract data type)Mathematical economicsMechanical engineeringSoftware Testing and Debugging TechniquesSoftware Engineering ResearchSoftware Reliability and Analysis Research