Movie clip reconstructed by an AI reading mice’s brains as they watch

Researchers have put together a 30-second movie clip based on a group of mice’s brain activity data that was recorded while they watched the footage

By Carissa Wong

3 May 2023

A mouse’s brain activity may give some indication into what it is seeing

EPFL/Hillary Sancutary/Alain Herzog/Allen Institute/Roddy Grieves

A black-and-white movie has been extracted almost perfectly from the brain signals of mice using an artificial intelligence tool.

Mackenzie Mathis at the Swiss Federal Institute of Technology Lausanne and her colleagues examined brain activity data from around 50 mice while they watched a 30-second movie clip nine times. The researchers then trained an AI to link this data to the 600-frame clip, in which a man runs to a car and opens its trunk.

Advertisement

The data was previously collected by other researchers who inserted metal probes, which record electrical pulses from neurons, into the mice’s primary visual cortexes, the area of the brain involved in processing visual information. Some brain activity data was also collected by imaging the mice’s brains using a microscope.

Next, Mathis and her team tested the ability of their trained AI to predict the order of frames within the clip using brain activity data that was collected from the mice as they watched the movie for the tenth time.

This revealed that the AI could predict the correct frame within one second 95 per cent of the time.