Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE.
Juntang Zhuang, Nicha C. Dvornek, Xiaoxiao Li, Sekhar Tatikonda, Xenophon Papademetris, James S. Duncan
Abstract
solvers. On image classification tasks, compared with the adjoint and naive method, ACA achieves half the error rate in half the training time; NODE trained with ACA outperforms ResNet in both accuracy and test-retest reliability. On time-series modeling, ACA outperforms competing methods. Finally, in an example of the three-body problem, we show NODE with ACA can incorporate physical knowledge to achieve better accuracy. We provide the PyTorch implementation of ACA: https://github.com/juntang-zhuang/torch-ACA.
Topics & Concepts
Computer scienceOdeAutomatic differentiationBenchmark (surveying)ComputationArtificial neural networkTrajectoryNode (physics)Ordinary differential equationAlgorithmMathematical optimizationArtificial intelligenceDifferential equationMathematicsApplied mathematicsEngineeringMathematical analysisAstronomyGeographyGeodesyPhysicsStructural engineeringModel Reduction and Neural NetworksGenerative Adversarial Networks and Image SynthesisNumerical methods for differential equations