What is the Vocabulary of Flaky Tests?
Gustavo Pinto, Breno Miranda, Supun Dissanayake, Marcelo d’Amorim, Christoph Treude, Antonia Bertolino
Abstract
Flaky tests are tests whose outcomes are non-deterministic. Despite the recent research activity on this topic, no effort has been made on understanding the vocabulary of flaky tests. This work proposes to automatically classify tests as flaky or not based on their vocabulary. Static classification of flaky tests is important, for example, to detect the introduction of flaky tests and to search for flaky tests after they are introduced in regression test suites.
Topics & Concepts
VocabularyComputer scienceTest (biology)Machine learningWork (physics)Artificial intelligenceNatural language processingEngineeringLinguisticsPhilosophyBiologyPaleontologyMechanical engineeringSoftware Engineering ResearchSoftware Testing and Debugging TechniquesImbalanced Data Classification Techniques