Dynamic V2X Perception From Road-to-Vehicle Vision

Abstract

Vehicle-to-everything (V2X) perception is an innovative technology that enhances vehicle perception accuracy, thereby elevating the security and reliability of autonomous systems. However, existing V2X perception methods exhibit limitations at traffic intersections due to a failure to account for effective collaboration between dynamic vehicles and stable Roadside Unit (RSU) in adapting to traffic environments. To harness the use of RSU and adapt V2X perception models to dynamic scenes, we propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Adaptive Road-to-Vehicle Perception</i> (AR2VP) approach, building V2X perception based on road-to-vehicle visual information. In AR2VP, we leverage RSU to offer stable, wide-range sensing capabilities and serve as communication hubs. AR2VP is devised to tackle both intra-scene and inter-scene changes. For the intra-scene changes, we construct a Dynamic Perception Representation (DPR) module, which effectively integrates vehicle perceptions, enabling vehicles to capture a more comprehensive range of dynamic factors within the scene. Moreover, we introduce a Road-to-Vehicle Perception Compensating (R2VPC) module, aiming at preserving the maximized RSU perception information in the presence of intra-scene changes. For the inter-scene changes, we implement a Prompt-Replay mechanism leveraging the RSU's limited storage capacity to retain a subset of typical historical scene samples, maintaining model robustness in response to inter-scene changes. We conduct comprehensive experiments on 3D object detection and segmentation, and the results show that AR2VP excels in both performance-bandwidth trade-offs and adaptability within dynamic environments. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/tjy1423317192/AR2VP</uri>.

Topics & Concepts