RF Genesis: Zero-Shot Generalization of mmWave Sensing through Simulation-Based Data Synthesis and Generative Diffusion Models

Abstract

This paper presents RF Genesis (RFGen), a novel and cost-effective method for synthesizing RF sensing data using cross-modal diffusion models, in order to improve the generalization capability of millimeter-wave (mmWave) sensing systems. Traditional machine learning models used in mmWave sensing struggle with limited training datasets. Their performance degrades drastically when confronted with unseen users, environments, sensor configurations, test classes, etc. RFGen mitigates these challenges by using a cross-modal generative framework to synthesize and expand mmWave sensing data. We specifically propose a custom ray tracing simulator to simulate RF propagation and interaction with objects/environments. We then leverage a set of diffusion models to generate massive 3D scenes, and transform the visual scene representation into the corresponding mmWave sensing data, under the direction of application-specific "prompts". Our proposed approach reconciles the physics-based ray tracing with the blackbox diffusion model, leading to accurate, scalable, and explainable vision-to-RF data synthesis. Our extensive real-world experiments highlight RFGen's effectiveness in diverse mmWave sensing applications, enhancing their generalization to unseen test cases without laborious data collection.

Topics & Concepts