Omni-NeRF: Neural Radiance Field from 360° Image Captures
Kai Gu, Thomas Maugey, Sebastian Knorr, Christine Guillemot
Abstract
This paper tackles the problem of novel view synthesis (NVS) from 360° images with imperfect camera poses or intrinsic parameters. We propose a novel end-to-end framework for training Neural Radiance Field (NeRF) models given only 360° RGB images and their rough poses, which we refer to as Omni-NeRF. We extend the pinhole camera model of NeRF to a more general camera model that better fits omni-directional fish-eye lenses. The approach jointly learns the scene geometry and optimizes the camera parameters without knowing the fisheye projection.
Topics & Concepts
RadiancePinhole cameraArtificial intelligenceComputer scienceComputer visionPinhole (optics)Projection (relational algebra)Pinhole camera modelField of viewField (mathematics)Artificial neural networkImage (mathematics)Computer graphics (images)Camera auto-calibrationCamera resectioningMathematicsPhysicsOpticsAlgorithmPure mathematicsAdvanced Vision and ImagingRobotics and Sensor-Based LocalizationComputer Graphics and Visualization Techniques