CARLA Simulated Data for Rare Road Object Detection
Tom Bu, Xinhe Zhang, Christoph Mertz, John M. Dolan
Abstract
Labeled data are paramount for modern, deep learning object detection models. However, such data are not always available, either due to time and financial constraints or due to the rarity of certain objects. In this paper, we show that the CARLA simulator can be used effectively to provide automatic annotations for custom street-view objects, boosting datasets for objects with few labels. We evaluate our models on real world images and show that low-shot training data expanded by synthetic images rendered in CARLA can provide better performance than training models with low-shot examples alone. To overcome the sim-to-real domain gap, we perform domain randomization by taking advantage of CARLA's diverse simulations of weather conditions, actors, and maps. We train detectors on CARLA-generated images of two different object classes and evaluate them on publicly available datasets. We provide access to our synthetic fire hydrant <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> https://www.kaggle.com/xinhez/synthetic-fire-hydrants and crosswalk <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> https://www.kaggle.com/buvision/synthetic-crosswalks datasets as well as provide step-by-step instructions <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> https://www.github.com/xinhez/simulation-for-detection to generate custom datasets in CARLA.