Event-LSTM: An Unsupervised and Asynchronous Learning-Based Representation for Event-Based Data

Abstract

Event cameras are activity-driven bio-inspired vision sensors that respond asynchronously to intensity changes resulting in sparse data known as events. It has potential advantages over conventional cameras, such as high temporal resolution, low latency, and low power consumption. Given the sparse and asynchronous spatio-temporal nature of the data, event processing is predominantly solved by transforming events into a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2D$</tex-math></inline-formula> spatial grid representation and applying standard vision pipelines. In this work, we propose an auto-encoder architecture named as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-LSTM</i> to generate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2D$</tex-math></inline-formula> spatial grid representation. Ours has the following main advantages 1) Unsupervised, task-agnostic learning of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2D$</tex-math></inline-formula> spatial grid. Ours is ideally suited for the event domain, where task-specific labeled data is scarce, 2) Asynchronous sampling of event <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$2D$</tex-math></inline-formula> spatial grid. This leads to speed invariant and energy-efficient representation. Evaluations on appearance-based and motion-based tasks demonstrate that our approach yields improvement over state-of-the-art techniques while providing the flexibility to learn spatial grid representation from unlabelled data.

Topics & Concepts