Read on Twitter
New paper on how global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys now published https://www.nature.com/articles/s41467-020-17343-w ! With @EFouragnan , @davidefolloni , @MKFlugge , BKH Chau, M Khamassi and MFS Rushworth at @OxfordWIN - Summary below 
We tested the idea that animals hold a representation of the global reward state – how good an environment is overall and not just how good specific choices are 
. We examined how the global reward state affects choices and where it is encoded in the brain 
. 1/11
. We examined how the global reward state affects choices and where it is encoded in the brain . 1/11
To do this, we analyzed decision making in a probabilistic learning task in monkeys – behaviorally and with a reinforcement learning model. Afterwards, we investigated the neural correlates of the global reward state with fMRI. 
2/11
2/11
Monkeys employed a win-stay/lose-shift strategy that followed the reward contingencies 
. However, they stayed more after a win if the general reward state (R-history here) was high – and they shifted more after no reward if it was low . 3/11
. However, they stayed more after a win if the general reward state (R-history here) was high – and they shifted more after no reward if it was low . 3/11
We next build a simple RL model that keeps track of the global reward state. The global reward state (called R-trace here) biases the prediction error calculation and hence value updates. Turns out, this maps very well onto the previous win-stay/lose-shift results 
. 4/11
. 4/11
These results remain valid on the individual monkey level and after accounting for a bunch of alternative factors such as choice repetition, side biases, contingent reward effects, asymmetric alphas, changes in inverse temperature, bla bla bla 
5/11
5/11
Such a representation of the global reward state can also appear as misassignment of reward to later choices as some people have observed in the past. 
It also influences task engagement. It’s the best predictor of quick decisions. 
6/11
It also influences task engagement. It’s the best predictor of quick decisions. 6/11
So: Where is the global reward state in the brain? Answer: most prominently in the agranular insula, very close to OFC. But also in the dorsal raphe and dACC. All these regions encode global reward state and also whether reward is delivered in the present moment. + 7/11
+ 7/11
The effects in the agranular insula are particularly striking. The BOLD time course early on carries information about the global reward state (GRS), but later on, BOLD signals current reward (R and no R). --> 8/11
--> 8/11
Take-away 
: Choices are reinforced by contingent outcomes but also the global reward state. Exploration is increased when global rewards are sparse. This is reflected in activity patterns in the anterior insula, dorsal raphe and dACC. +. 9/11
: Choices are reinforced by contingent outcomes but also the global reward state. Exploration is increased when global rewards are sparse. This is reflected in activity patterns in the anterior insula, dorsal raphe and dACC. +. 9/11
The project is very much inspired by the great work from @daeyeol_lee, @ali_r_soltani, Hyojung Seo, @StePalminteri , @rei_akaishi @RudebeckLab, M Walton, @OFC_oracle, Hannah Clarke,
@behrenstimb, @nathanieldaw and many others
. 10/11
@behrenstimb, @nathanieldaw and many others
. 10/11
Behavioural data and analyses scripts are online. - Finally, the project is also an extension of my own previous work in humans. Like in macaques, a dACC-insula network encoded past and current rewards, but with a different frame of reference. 
https://www.nature.com/articles/ncomms12327 11/11
https://www.nature.com/articles/ncomms12327 11/11
