Closing the RISC-V Compliance Gap: Looking from the Negative Testing Side
Vladimir Herdt, Daniel Grose, Rolf Drechsler
Abstract
Compliance testing for RISC-V is very important. Therefore, an official hand-written compliance test-suite is being actively developed. However, besides requiring significant manual effort, it focuses on positive testing (the implemented instructions work as expected) only and neglects negative testing (consider illegal instructions to also ensure that no additional/unexpected behavior is accidentally added). This leaves a large gap in compliance testing. In this paper we propose a fuzzing-based test-suite generation approach to close this gap. We found new bugs in several RISC-V simulators including riscvOVPsim from Imperas which is the official reference simulator for compliance testing.
Topics & Concepts
Fuzz testingConformance testingComputer scienceSuiteTest suiteCompliance (psychology)Closing (real estate)Test (biology)White-box testingSoftware testingComputer securitySoftwareOperating systemTest caseLawPsychologySoftware developmentPolitical scienceSoftware constructionStandardizationBiologyPaleontologySocial psychologySoftware Testing and Debugging TechniquesAdvanced Malware Detection TechniquesSecurity and Verification in Computing