Overcoming the TradeOff between Accuracy and Plausibility in 3D Hand Shape Reconstruction
Ziwei Yu, Chen Li, Linlin Yang, Xiaoxu Zheng, Michael Bi Mi, Gim Hee Lee, Angela Yao
Abstract
Direct mesh fitting for 3D hand shape reconstruction is highly accurate. However, the reconstructed meshes are prone to artifacts and do not appear as plausible hand shapes. Conversely, parametric models like MANO ensure plausible hand shapes but are not as accurate as the non-parametric methods. In this work, we introduce a novel weakly-supervised hand shape estimation framework that integrates non-parametric mesh fitting with MANO model in an end-to-end fashion. Our joint model overcomes the tradeoff in accuracy and plausibility to yield well-aligned and high-quality 3D meshes, especially in challenging two-hand and hand-object interaction scenarios.
Topics & Concepts
Polygon meshParametric statisticsComputer scienceArtificial intelligenceParametric modelSolid modelingObject (grammar)Iterative reconstruction3D reconstructionComputer visionAlgorithm3d modelPattern recognition (psychology)MathematicsComputer graphics (images)StatisticsHuman Pose and Action Recognition3D Shape Modeling and AnalysisHand Gesture Recognition Systems