TorchMetrics - Measuring Reproducibility in PyTorch
Nicki Skafte Detlefsen, Jiří Borovec, Justus Schock, Ananya Jha, Teddy Koker, Luca Di Liello, Daniel Stancl, Changsheng Quan, Maxim Grechkin, William Falcon
Abstract
A main problem with reproducing machine learning publications is the variance of metric implementations across papers. A lack of standardization leads to different behavior in mechanisms such as checkpointing, learning rate schedulers or early stopping, that will influence the reported results. For example, a complex metric such as Frchet inception distance (FID) for synthetic image quality evaluation
Topics & Concepts
ReproducibilityComputer scienceArtificial intelligenceMathematicsStatisticsImage Processing and 3D ReconstructionMetallurgy and Material Forming