Automated testing of software that uses machine learning APIs
Chengcheng Wan, Shicheng Liu, Sophie Xie, Yifan Liu, Henry Hoffmann, Michael Maire, Shan Lu
Abstract
An increasing number of software applications incorporate machine learning (ML) solutions for cognitive tasks that statistically mimic human behaviors. To test such software, tremendous human effort is needed to design image/text/audio inputs that are relevant to the software, and to judge whether the software is processing these inputs as most human beings do. Even when misbehavior is exposed, it is often unclear whether the culprit is inside the cognitive ML API or the code using the API.
Topics & Concepts
Computer scienceSoftwareSoftware constructionArtificial intelligenceSoftware engineeringMachine learningSoftware systemHuman–computer interactionProgramming languageAdversarial Robustness in Machine LearningAnomaly Detection Techniques and ApplicationsMachine Learning and Data Classification