Two subjects perform a token-based decision-making task. On each trial two bars appear, one after the other, and each one is a gamble. A bar is usually split in two, one possible outcome stacked on top of the other: the colour of each section says how many virtual tokens it would bring or take away, anywhere from losing two to winning three, and its height says how likely it is. The subject reports its choice by looking at one of the bars. A small reward is delivered either way to keep the subject engaged. The goal of the task sits along the bottom of the screen, six circles that fill as tokens accumulate, and when the sixth one fills, there is a jackpot reward, and the count goes back to zero.

The experiments were carried out during previous years at the universities of Rochester and Minnesota by Benjamin Hayden’s group and collaborators. Parts of the dataset have been analysed before, for other research questions. Demetrio Ferro (CRM, UPF) curated the data and set about modelling the choices: roughly 106,000 trials across 227 sessions. In the behaviour he found two things that do not usually travel together. As the circles filled, the decisions became faster, and they also became more accurate.

“There is typically a trade-off between speed and accuracy,” says Ferro, a postdoctoral researcher in the Computational Neuroscience group at the CRM. “When we decide very quickly we tend to make more mistakes, while having more time allows more accurate decisions.” Accuracy here has a precise meaning: picking the option with the higher expected value, what it would pay out on average once its outcomes are weighted by their probabilities. On the whole, subjects also picked the riskier of the two bars less often once the jackpot came within reach.

One explanation is that they had stopped playing the same game.

“From there I thought that perhaps, on reaching a certain number of tokens, a different strategy began to be used,” says Ferro. “Instead of simply comparing the two options against each other, what might start to matter is getting the tokens still missing to reach the jackpot.” He built a mathematical model in which the utility of an option depends both on its outcome and on how many tokens are already showing at the bottom of the screen.

The changeover between the two strategies is carried by a transition parameter, estimated from the data rather than fixed by hand. The estimate landed at around three tokens in both subjects. That is also where the arithmetic of the task puts it, since the largest available win is three: from three tokens onwards, the jackpot can be reached in a single trial. Compared against other reasonable models, and there were more of them than the three reported in the paper, the version with the moving reference predicted the subject’s real choices more accurately.

The framework behind this is prospect theory, introduced by Kahneman and Tversky in 1979, which holds that we judge outcomes as gains or losses relative to a reference point, and that a loss weighs more heavily than an equivalent gain. Ferro wanted both ingredients in the model, a reference point that moves with the token count, changing how offers are evaluated, and an asymmetry between gains and losses in the utility function.

“We were not only interested in finding the model that best reproduced the choices, but in checking whether the mechanism it proposed made sense in terms of the task and of decision-making,” Ferro says.

The cells lean towards gains

The second half of the study takes those models to the neural data; 129 single cells recorded in the dorsal anterior cingulate cortex while the subjects solved the task. Gains and losses here are the model’s own categories, measured against the moving reference, so with two tokens still missing even winning one can count as a loss. By that measure most offer values, around 57 per cent, fell on the loss side. Even so, more cells were tuned to potential gains, and their activity predicted the upcoming choice considerably better. The authors checked that trial numbers were not behind it: an effect driven by trial counts would have favoured losses, the more common category.

Ferro’s reading distinguishes between what an outcome is worth and what the brain needs in order to select an action. One possible explanation, he says, is that potential gains provide particularly useful information for deciding how to move towards the goal: depending on how many tokens are already in hand, a gain can be compared with another gain or with the tokens still missing.

“Potential gains may represent something more than the immediate value of an option: they can act as a prospective signal, indicating what can be obtained and which action can bring the goal closer.”

Demetrio Ferro, Computational Neuroscience group, CRM

That fits a region already associated with monitoring actions, their context and their consequences. The token count has a neural signal of its own in these recordings, and for much of the trial, away from the moments when the offers are on screen, more cells track it than track the value of the offers. What the reference-dependent model suggests is that the region does not simply track the token count and the risk of each option independently, and may also combine them, non-linearly, into a value that depends on where the subject stands relative to the jackpot. “For me that is one of the most interesting aspects of the work,” says Ferro. “The same signal that reports the potential value of a decision can also serve to orient future action towards a goal.”

The paper belongs to a longer line. Ferro has worked on attention in visual cortex, on how gaze gates and reactivates value signals in orbitofrontal cortex, and now on reward accumulation in the cingulate. What holds them together, he says, is cognitive selection: attention, gaze, the valuation of options and the pull of a distant goal are different processes, and every one of them can modulate neural activity and determine which information counts at a given moment.

He intends to follow the same principles at other levels of description, from single neurons and interactions between cortical layers up to the coarser signals that can be recorded in people, where he also sees translational potential, in understanding which of these mechanisms are altered in cognitive and motor dysfunction. For the moment the account rests on two subjects, 129 cells and a screen with six circles on it.