A Crowdsourcing Approach to Video Quality Assessment
Babak Naderi, Ross Cutler
Abstract
We propose an open-source extension of the ITU-T Rec. P.910 subjective video quality test based on crowdsourcing principles. This extension addresses the speed, usage cost, and barrier to usage issues of P.910. We implement Absolute Category Rating (ACR), ACR with hidden reference (ACRHR), Degradation Category Rating (DCR), and Comparison Category Rating (CCR), and include rater, environment, hardware, and network qualifications, as well as gold and trapping questions to ensure quality. We have validated that the implementation is both accurate and highly reproducible.
Topics & Concepts
CrowdsourcingComputer scienceExtension (predicate logic)Quality (philosophy)Video qualityDegradation (telecommunications)Quality assessmentSubjective video qualityTest (biology)Data miningReliability engineeringMachine learningArtificial intelligenceEvaluation methodsWorld Wide WebImage qualityTelecommunicationsImage (mathematics)EngineeringOperations managementBiologyPhilosophyProgramming languageMetric (unit)EpistemologyPaleontologyImage and Video Quality AssessmentIndustrial Vision Systems and Defect DetectionAdvanced Computing and Algorithms