ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts
Mikolaj Sacha, Dawid Rymarczyk, Łukasz Struski, Jacek Tabor, Bartosz Zieliński
20232023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)36 citationsDOIOpen Access PDF
Abstract
We introduce ProtoSeg, a novel model for interpretable semantic image segmentation, which constructs its predictions using similar patches from the training set. To achieve accuracy comparable to baseline methods, we adapt the mechanism of prototypical parts and introduce a diversity loss function that increases the variety of prototypes within each class. We show that ProtoSeg discovers semantic concepts, in contrast to standard segmentation models. Experiments conducted on Pascal VOC and Cityscapes datasets confirm the precision and transparency of the presented method.
Topics & Concepts
Pascal (unit)SegmentationComputer scienceArtificial intelligenceTransparency (behavior)Image segmentationSet (abstract data type)Contrast (vision)Pattern recognition (psychology)Variety (cybernetics)Training setMachine learningComputer visionProgramming languageComputer securityExplainable Artificial Intelligence (XAI)Advanced Neural Network ApplicationsAdversarial Robustness in Machine Learning