A Review of the Impact of Rain on Camera-Based Perception in Automated Driving Systems
Tim Brophy, Darragh Mullins, Ashkan Parsi, Jonathan Horgan, Enda Ward, Patrick Denny, Ciarán Eising, Brian Deegan, Martin Glavin, Edward Jones
Abstract
Automated vehicles rely heavily on image data from visible spectrum cameras to perform a wide range of tasks from object detection, classification, and avoidance to path planning. The availability and reliability of these sensors in adverse weather is therefore of critical importance to the safe and continuous operation of an automated vehicle. This review paper presents a data communication-inspired Image Formation Framework that characterizes the data flow from object through channel to sensor, and subsequent processing of the data. This framework is used to explore the degree to which adverse weather conditions impact the cameras used on automated vehicles for sensing and perception. The effect of rain on each element of the model is reviewed. Furthermore, the prevalence of the appearance of these rain-induced changes in publicly available, open-source datasets is reviewed. The degree to which synthetic rain generation techniques can accurately capture these changes is also examined. Finally, the paper offers some suggestions on how future adverse weather automotive datasets should be collected.