Dynamically reconfiguring software microbenchmarks: reducing execution time without sacrificing result quality
Christoph Laaber, Stefan Würsten, Harald C. Gall, Philipp Leitner
Abstract
Executing software microbenchmarks, a form of small-scale performance tests predominantly used for libraries and frameworks, is a costly endeavor. Full benchmark suites take up to multiple hours or days to execute, rendering frequent checks, e.g., as part of continuous integration (CI), infeasible. However, altering benchmark configurations to reduce execution time without considering the impact on result quality can lead to benchmark results that are not representative of the software’s true performance.
Topics & Concepts
Computer scienceBenchmark (surveying)Rendering (computer graphics)SoftwareExecution timeDistributed computingEmbedded systemOperating systemArtificial intelligenceGeographyGeodesySoftware System Performance and ReliabilitySoftware Testing and Debugging TechniquesAdvanced Software Engineering Methodologies