Eye-Tracking Software Developed for fMRI

Article: Artificial intelligence predicts eye movements 
Source: Max Planck Institute (Germany)
Published: November 8, 2021 

Viewing behavior can provide meaningful information about neurological health. As such, eye-tracking technology can be clinically relevant in the diagnosis and management of neurological injury. Typically, this eye-tracking comes in the form of sensor technology, in which infrared light is projected onto the retina, reflected, and then measured by the sensor. Although functional magnetic resonance imaging (fMRI) is the gold standard of functional brain imaging, MRIs use strong magnets and integrating MRI-compatible camera systems often comes at a high cost. This has thus far prevented the widespread use of eye-tracking in MRI exams. Researchers at the Max Planck Institute in Germany sought to improve upon eye-tracking availability by directly applying software to fMRI. These researchers developed a software called DeepMReye, a convolutional neural network (CNN) that decodes gaze position from the magnetic resonance signal of the eyeballs. Notable features of this technology include the fact that it performs cameraless eye-tracking during an fMRI scan, and works even in existing datasets and when the eyes are closed. First author of the study explains, "The neural network we use detects specific patterns in the MRI signal from the eyes. This allows us to predict where the person is looking." The software is trained on both publicly available data and study participants to now be able to perform eye-tracking on data that it was not trained on, such as existing MRI imaging that was previously acquired without eye-tracking. Because the software can predict eye movements even when the eyes are closed, it can facilitate studies of individuals in a sleeping state or of individuals who are blind. In the latter case, the researchers remark that whereas traditional eye-tracking has suffered from calibration difficulties in blind patients, "Here too, studies can be carried out more easily with DeepMReye, as the artificial intelligence can be calibrated with the help of healthy subjects and then be applied in examinations of blind patients." They have made the DeepMReye software an open source application for other researchers to use in the hopes of making eye-tracking more widespread in MRI examinations.

My rating of this study: ⭐⭐⭐

Frey M, Nau M and Doeller CF. "Magnetic resonance-based eye tracking using deep neural networks." Nature Neuroscience.  24:1772–1779. 8 November 2021. https://doi.org/10.1038/s41593-021-00947-w