SSBC 2020: Sclera Segmentation Benchmarking Competition in the Mobile Environment
Matej Vitek, Abhijit Das, Y. Pourcenoux, A. Missler, C. Paumier, Sumanta Das, Ishita De Ghosh, Diego Rafael Lucio, Luiz A. Zanlorensi, David Menotti, Fadi Boutros, Naser Damer, Jonas Henry Grebe, Arjan Kuijper, J. Hu, Yongchuan He, Changshuo Wang, Hongjia Liu, Yangang Wang, Zhenan Sun, Dailé Osorio-Roig, Christian Rathgeb, Christoph Busch, Juan Tapia, Andrés Valenzuela, Georgios Zampoukis, Lazaros Tsochatzidis, Ioannis Pratikakis, Sabari Nathan, R. Suganya, Vineet Mehta, Abhinav Dhall, Kiran Raja, Gourav Gupta, Jalil Nourmohammadi Khiarak, M. Akbari-Shahper, Farhang Jaryani, Meysam Asgari-Chenaghlu, Ritesh Vyas, Sagnik Dakshit, Peter Peer, Umapada Pal, Vitomir Štruc
Abstract
The paper presents a summary of the 2020 Sclera Segmentation Benchmarking Competition (SSBC), the 7th in the series of group benchmarking efforts centred around the problem of sclera segmentation. Different from previous editions, the goal of SSBC 2020 was to evaluate the performance of sclera-segmentation models on images captured with mobile devices. The competition was used as a platform to assess the sensitivity of existing models to i) differences in mobile devices used for image capture and ii) changes in the ambient acquisition conditions. 26 research groups registered for SSBC 2020, out of which 13 took part in the final round and submitted a total of 16 segmentation models for scoring. These included a wide variety of deep-learning solutions as well as one approach based on standard image processing techniques. Experiments were conducted with three recent datasets. Most of the segmentation models achieved relatively consistent performance across images captured with different mobile devices (with slight differences across devices), but struggled most with low-quality images captured in challenging ambient conditions, i.e., in an indoor environment and with poor lighting.