Combining video telemetry and wearable MEG for naturalistic imaging
George C. O’Neill, Robert A. Seymour, Stephanie Mellor, Nicholas Alexander, Tim M. Tierney, Léa Bernachot, Mansoureh Fahimi Hnazaee, Meaghan Elizabeth Spedden, Ryan C. Timms, Daniel Bush, Sven Bestmann, Matthew J. Brookes, Gareth R. Barnes
Abstract
Neuroimaging studies have typically relied on rigorously controlled experimental paradigms to probe cognition, in which movement is restricted, primitive, an afterthought or merely used to indicate a subject's choice. Whilst powerful, these paradigms do not often resemble how we behave in everyday life, so a new generation of ecologically valid experiments are being developed. Magnetoencephalography (MEG) measures neural activity by sensing extracranial magnetic fields. It has recently been transformed from a large, static imaging modality to a wearable method where participants can move freely. This makes wearable MEG systems a prime candidate for naturalistic experiments going forward. However, these experiments will also require novel methods to capture and integrate information about behaviour executed during neuroimaging, and it is not yet clear how this could be achieved. Here, we use video recordings of multi-limb dance moves, processed with open-source machine learning methods, to automatically identify time windows of interest in concurrent, wearable MEG data. In a first step, we compare a traditional, block-designed analysis of limb movements, where the times of interest are based on stimulus presentation, to an analysis pipeline based on hidden Markov model states derived from the video telemetry. Next, we show that it is possible to identify discrete modes of neuronal activity related to specific limbs and body posture by processing the participants' choreographed movement in a dancing paradigm. This demonstrates the potential of combining video telemetry with mobile magnetoencephalography and other legacy imaging methods for future studies of complex and naturalistic behaviours.