Bridging known and unknown dynamics by transformer-based machine-learning inference from sparse observations

Abstract

In applications, an anticipated issue is where the system of interest has never been encountered before and sparse observations can be made only once. Can the dynamics be faithfully reconstructed? We address this challenge by developing a hybrid transformer and reservoir-computing scheme. The transformer is trained without using data from the target system, but with essentially unlimited synthetic data from known chaotic systems. The trained transformer is then tested with the sparse data from the target system, and its output is further fed into a reservoir computer for predicting its long-term dynamics or the attractor. The proposed hybrid machine-learning framework is tested using various prototypical nonlinear systems, demonstrating that the dynamics can be faithfully reconstructed from reasonably sparse data. The framework provides a paradigm of reconstructing complex and nonlinear dynamics in the situation where training data do not exist and the observations are random and sparse. In experimental situations, random and sparse observations hinder understanding of the underlying complex dynamical system. The authors introduce a hybrid, transformer-based machine-learning framework to reconstruct the dynamics of new, unseen systems from sparse observations by training on a diverse set of synthetic systems.

Topics & Concepts