ADA: Decoding Brain Signals Misaligned in Time

Decoding what is happening in a person’s mind from their brain activity is one of the ultimate goals of modern neuroscience. Over recent decades, advances in brain decoding techniques have made it possible to identify neural patterns associated with specific actions and external stimuli, such as moving a hand, looking at an image, or listening to a sound. However, decoding internal mental processes—such as remembering an experience, imagining a scene, or retrieving information from memory—remains a fundamental challenge.

A study published by Pablo Oyarzo (Freie Universität Berlin and Aarhus University), Radoslaw M. Cichy (Aarhus University), and Diego Vidaurre (Centre de Recerca Matemàtica, Aarhus University, and University of Oxford) presents a new strategy to address this issue. The researchers have developed the Adaptive Decoding Algorithm (ADA), a method capable of identifying and decoding relevant neural signals even when they appear at different moments from one trial to another.

The Challenge of Internal Mental Processes

Current brain decoding methods work particularly well when there is a clearly defined external event in time. For example, if a person observes an image at a specific moment, researchers can analyze the subsequent brain response and compare data across trials, since the relevant signals typically appear at roughly the same time in each repetition.

However, many cognitive processes do not follow such a precise timetable. Mental activities such as imagination, memory retrieval, or certain forms of spontaneous thought can unfold at different paces each time they occur. A person may take more or less time to retrieve a memory or generate a mental image, causing the neural signals of interest to appear earlier or later relative to other trials.

This temporal variability poses a major limitation for conventional techniques. When these differences are not taken into account, methods can end up comparing signals that correspond to different moments of brain processing, thereby reducing decoding accuracy.

The authors emphasize that this problem has two main dimensions. The first is spatial: where the relevant information is generated in the brain and how accurately it can be measured. The second is temporal: when that information appears and how its timing varies across trials. While much of the previous research has focused on the spatial dimension, this work directly addresses the temporal uncertainty that characterizes many internal cognitive processes.

In fact, from a machine learning perspective, this situation can be understood as an input feature misalignment problem. Even if the labels associated with each trial are correct, the relevant information does not necessarily appear at the same point in time, making it difficult for algorithms to identify common patterns across different cases.

ADA: Identifying When Relevant Information Appears

Although tools exist to study how brain activity varies across trials, many are designed to interpret data once collected rather than to make predictions on new trials. ADA aims to combine both capabilities within a single framework.

ADA’s starting point is a simple idea: to properly interpret a brain signal, it is not enough to identify what information it contains—it is also necessary to know when it appears.

To achieve this, the researchers developed ADA using a two-stage process. First, it analyzes each trial independently to locate the temporal window with the highest probability of containing task-relevant information. Next, it uses those selected windows to train and apply a decoder capable of interpreting new data.

Unlike traditional approaches, the method does not assume that all trials are perfectly synchronized. Instead, it searches for informative neural patterns that can recur across trials even if they appear at different times. To do so, the framework leverages a non-parametric approach based on k-nearest neighbors and cosine similarity metrics, which allows it to detect pattern recurrences regardless of their absolute position in time and prevents the algorithm from mistaking global signal fluctuations for the actual onset of a cognitive process. In this way, the algorithm can detect signals associated with the same experimental condition without requiring exact temporal alignment.

Beyond neuroscience, the problem addressed by ADA connects to a broader question in machine learning. In many datasets, relevant information can be displaced or misaligned across observations, making it hard to identify using conventional techniques. The new method proposes a way to automatically locate those informative segments prior to classification or prediction, offering a solution that is potentially useful in domains beyond neuroscience.

Decoding Better and Without a Shared Clock

The researchers evaluated ADA’s performance through controlled computational simulations and by applying latency shifts to real visual perception MEG data to emulate the temporal variability typical of memory recall.

The results show that traditional methods lose accuracy as temporal variability between trials increases. ADA, in contrast, maintains significantly superior decoding performance by explicitly adapting to differences in the onset of relevant neural activity. At the same time, it maintains remarkably stable performance as temporal variability grows, with its non-parametric implementation proving particularly robust across noisy scenarios.

In addition to improving prediction accuracy, the algorithm was able to estimate when relevant information appeared in each trial. This feature provides a new tool for studying the temporal dynamics of cognitive processes rather than just their content.

According to the authors, ADA constitutes the first unified supervised framework capable of automatically identifying informative temporal segments and subsequently generalizing that knowledge to new trials. The method can also be applied to different types of continuous signals without depending on specific assumptions about the recording technique used, thus broadening its potential applications.

New Opportunities to Study Mental Processes

Beyond its immediate application, the work opens new avenues for investigating mental processes whose temporal evolution is difficult to observe directly.

The approach could contribute to a better understanding of the neural mechanisms involved in functions such as memory, imagination, or the recall of experiences, as well as facilitate the future development of brain-computer interfaces capable of operating in situations where the exact timing of relevant brain information cannot be predicted in advance.

The authors also note that future versions of the method could incorporate more powerful decoders, such as convolutional neural networks or attention-based architectures, capable of learning spatiotemporal representations directly from brain signals and capturing more complex temporal patterns.

By providing a way to identify not only what information a brain signal contains, but also when that information emerges, ADA offers a new window into studying the temporal dynamics of brain activity. The results suggest that understanding these temporal variations is not merely a technical issue, but a key piece in advancing toward one of neuroscience’s greatest challenges: deciphering how the brain generates thoughts, memories, and mental images.