Mind matters: placebo enhances reward learning in parkinson's disease

ABSTRACT Expectations have a powerful influence on how we experience the world. Neurobiological and computational models of learning suggest that dopamine is crucial for shaping expectations

of reward and that expectations alone may influence dopamine levels. However, because expectations and reinforcers are typically manipulated together, the role of expectations _per se_ has

remained unclear. We separated these two factors using a placebo dopaminergic manipulation in individuals with Parkinson's disease. We combined a reward learning task with functional

magnetic resonance imaging to test how expectations of dopamine release modulate learning-related activity in the brain. We found that the mere expectation of dopamine release enhanced

reward learning and modulated learning-related signals in the striatum and the ventromedial prefrontal cortex. These effects were selective to learning from reward: neither medication nor

placebo had an effect on learning to avoid monetary loss. These findings suggest a neurobiological mechanism by which expectations shape learning and affect. Access through your institution

Buy or subscribe This is a preview of subscription content, access via your institution ACCESS OPTIONS Access through your institution Subscribe to this journal Receive 12 print issues and

online access $209.00 per year only $17.42 per issue Learn more Buy this article * Purchase on SpringerLink * Instant access to full article PDF Buy now Prices may be subject to local taxes

which are calculated during checkout ADDITIONAL ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our FAQs * Contact customer support SIMILAR CONTENT BEING VIEWED BY

OTHERS LITHIUM MODULATES STRIATAL REWARD ANTICIPATION AND PREDICTION ERROR CODING IN HEALTHY VOLUNTEERS Article 30 October 2020 DOPAMINE RELEASE IN HUMAN ASSOCIATIVE STRIATUM DURING REVERSAL

LEARNING Article Open access 02 January 2024 THE PREDICTION-ERROR HYPOTHESIS OF SCHIZOPHRENIA: NEW DATA POINT TO CIRCUIT-SPECIFIC CHANGES IN DOPAMINE ACTIVITY Article Open access 29

September 2021 REFERENCES * Goetz, C.G. et al. Placebo influences on dyskinesia in Parkinson's disease. _Mov. Disord._ 23, 700–707 (2008). Article  PubMed  PubMed Central  Google

Scholar  * Goetz, C.G. et al. Placebo response in Parkinson's disease: comparisons among 11 trials covering medical and surgical interventions. _Mov. Disord._ 23, 690–699 (2008).

Article  PubMed  Google Scholar  * McRae, C. et al. Effects of perceived treatment on quality of life and medical outcomes in a double-blind placebo surgery trial. _Arch. Gen. Psychiatry_

61, 412–420 (2004). Article  PubMed  Google Scholar  * Benedetti, F. et al. Placebo-responsive Parkinson patients show decreased activity in single neurons of subthalamic nucleus. _Nat.

Neurosci._ 7, 587–588 (2004). Article  CAS  PubMed  Google Scholar  * de la Fuente-Fernández, R. et al. Expectation and dopamine release: mechanism of the placebo effect in Parkinson's

disease. _Science_ 293, 1164–1166 (2001). Article  PubMed  Google Scholar  * Lidstone, S.C. et al. Effects of expectation on placebo-induced dopamine release in Parkinson disease. _Arch.

Gen. Psychiatry_ 67, 857–865 (2010). Article  CAS  PubMed  Google Scholar  * Voon, V. et al. Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. _Neuron_ 65, 135–142

(2010). Article  CAS  PubMed  PubMed Central  Google Scholar  * de la Fuente-Fernández, R. et al. Dopamine release in human ventral striatum and expectation of reward. _Behav. Brain Res._

136, 359–363 (2002). Article  PubMed  Google Scholar  * Schweinhardt, P., Seminowicz, D.A., Jaeger, E., Duncan, G.H. & Bushnell, M.C. The anatomy of the mesolimbic reward system: a link

between personality and the placebo analgesic response. _J. Neurosci._ 29, 4882–4887 (2009). Article  CAS  PubMed  PubMed Central  Google Scholar  * Zubieta, J.K. & Stohler, C.S.

Neurobiological mechanisms of placebo responses. _Ann. NY Acad. Sci._ 1156, 198–210 (2009). Article  CAS  PubMed  Google Scholar  * Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J.

& Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behavior in humans. _Nature_ 442, 1042–1045 (2006). Article  CAS  PubMed  PubMed Central  Google Scholar  *

Sutton, R.S. & Barto, A.G. _Reinforcement Learning: an Introduction_ (Bradford Book, 1998). * Barto, A.G. Reinforcement learning and dynamic programming. in _Analysis, Design and

Evaluation of Man-Machine Systems, Vol. 1,2_ (ed. Johannsen, G.) 407–412 (1995). * Daw, N.D. in Trial-by-trial data analysis using computational models. _Decision Making, Affect, Learning:

Attention and Performance XXIII, Vol. 1_ (eds. Delgado, M.R., Phelps, E.A. & Robbins, T.W.) 3–38 (Oxford University Press, 2011). * Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour,

B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. _Nature_ 441, 876–879 (2006). Article  CAS  PubMed  PubMed Central  Google Scholar  * Hare, T.A.,

O'Doherty, J., Camerer, C.F., Schultz, W. & Rangel, A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors.

_J. Neurosci._ 28, 5623–5630 (2008). Article  CAS  PubMed  PubMed Central  Google Scholar  * Levy, D.J. & Glimcher, P.W. The root of all value: a neural common currency for choice.

_Curr. Opin. Neurobiol._ 22, 1027–1038 (2012). Article  CAS  PubMed  PubMed Central  Google Scholar  * Plassmann, H., O'Doherty, J.P. & Rangel, A. Appetitive and aversive goal

values are encoded in the medial orbitofrontal cortex at the time of decision making. _J. Neurosci._ 30, 10799–10808 (2010). Article  CAS  PubMed  PubMed Central  Google Scholar  * McClure,

S.M., Berns, G.S. & Montague, P.R. Temporal prediction errors in a passive learning task activate human striatum. _Neuron_ 38, 339–346 (2003). Article  CAS  PubMed  Google Scholar  *

O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. _Neuron_ 38, 329–337 (2003). Article

  CAS  PubMed  Google Scholar  * Daw, N.D. & Doya, K. The computational neurobiology of learning and reward. _Curr. Opin. Neurobiol._ 16, 199–204 (2006). Article  CAS  PubMed  Google

Scholar  * Rushworth, M.F., Mars, R.B. & Summerfield, C. General mechanisms for decision making. _Curr. Opin. Neurobiol._ 19, 75–83 (2009). Article  CAS  PubMed  Google Scholar  *

Chowdhury, R. et al. Dopamine restores reward prediction errors in old age. _Nat. Neurosci._ 16, 648–653 (2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Schönberg, T., Daw,

N.D., Joel, D. & O'Doherty, J.P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. _J. Neurosci._ 27,

12860–12867 (2007). Article  CAS  PubMed  PubMed Central  Google Scholar  * Schönberg, T. et al. Selective impairment of prediction error signaling in human dorsolateral but not ventral

striatum in Parkinson's disease patients: evidence from a model-based fMRI study. _Neuroimage_ 49, 772–781 (2010). Article  PubMed  Google Scholar  * Behrens, T.E., Hunt, L.T.,

Woolrich, M.W. & Rushworth, M.F. Associative learning of social value. _Nature_ 456, 245–249 (2008). Article  CAS  PubMed  PubMed Central  Google Scholar  * Li, J., Delgado, M.R. &

Phelps, E.A. How instructed knowledge modulates the neural systems of reward learning. _Proc. Natl. Acad. Sci. USA_ 108, 55–60 (2011). Article  PubMed  Google Scholar  * Niv, Y., Edlund,

J.A., Dayan, P. & O'Doherty, J.P. Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. _J. Neurosci._ 32, 551–562 (2012). Article  CAS

  PubMed  PubMed Central  Google Scholar  * Kirsch, I. Response expectancy as a determinant of experience and behavior. _Am. Psychol._ 40, 1189–1202 (1985). Article  Google Scholar  * Wager,

T.D. et al. Placebo-induced changes in FMRI in the anticipation and experience of pain. _Science_ 303, 1162–1167 (2004). Article  CAS  PubMed  Google Scholar  * Pessiglione, M. et al.

Subliminal instrumental conditioning demonstrated in the human brain. _Neuron_ 59, 561–567 (2008). Article  CAS  PubMed  PubMed Central  Google Scholar  * Schmidt, L., Palminteri, S.,

Lafargue, G. & Pessiglione, M. Splitting motivation: unilateral effects of subliminal incentives. _Psychol. Sci._ 21, 977–983 (2010). Article  PubMed  Google Scholar  * Benedetti, F. How

the doctor's words affect the patient's brain. _Eval. Health Prof._ 25, 369–386 (2002). Article  PubMed  Google Scholar  * Fournier, J.C. et al. Antidepressant drug effects and

depression severity: a patient-level meta-analysis. _J. Am. Med. Assoc._ 303, 47–53 (2010). Article  CAS  Google Scholar  * Kirsch, I. & Sapirstein, G. Listening to Prozac but hearing

placebo: a meta-analysis of antidepressant medication. _Prev. Treat._ 1, 2a (1998). Google Scholar  * Bódi, N. et al. Reward-learning and the novelty-seeking personality: a between- and

within-subjects study of the effects of dopamine agonists on young Parkinson's patients. _Brain_ 132, 2385–2395 (2009). Article  PubMed  PubMed Central  Google Scholar  * Frank, M.J.,

Seeberger, L.C. & O'Reilly, R.C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. _Science_ 306, 1940–1943 (2004). Article  CAS  PubMed  Google Scholar  *

Palminteri, S. et al. Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes. _Proc. Natl. Acad. Sci. USA_ 106, 19179–19184 (2009). Article 

PubMed  PubMed Central  Google Scholar  * Maia, T.V. Reinforcement learning, conditioning, and the brain: successes and challenges. _Cogn. Affect. Behav. Neurosci._ 9, 343–364 (2009).

Article  PubMed  Google Scholar  * Gold, J.M. et al. Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence.

_Arch. Gen. Psychiatry_ 69, 129–138 (2012). Article  PubMed  PubMed Central  Google Scholar  * Bamford, N.S. et al. Heterosynaptic dopamine neurotransmission selects sets of corticostriatal

terminals. _Neuron_ 42, 653–663 (2004). Article  CAS  PubMed  Google Scholar  * Wang, W. et al. Regulation of prefrontal excitatory neurotransmission by dopamine in the nucleus accumbens

core. _J. Physiol. (Lond.)_ 590, 3743–3769 (2012). Article  CAS  Google Scholar  * Bamford, N.S. et al. Dopamine modulates release from corticostriatal terminals. _J. Neurosci._ 24,

9541–9552 (2004). Article  CAS  PubMed  PubMed Central  Google Scholar  * Colloca, L. et al. Learning potentiates neurophysiological and behavioral placebo analgesic responses. _Pain_ 139,

306–314 (2008). Article  PubMed  Google Scholar  * Voudouris, N.J., Peck, C.L. & Coleman, G. The role of conditioning and verbal expectancy in the placebo response. _Pain_ 43, 121–128

(1990). Article  CAS  PubMed  Google Scholar  * Kordower, J.H. et al. Disease duration and the integrity of the nigrostriatal system in disease. _Brain_ 136, 2419–2431 (2013). Article 

PubMed  PubMed Central  Google Scholar  * Cools, R., Barker, R.A., Sahakian, B.J. & Robbins, T.W. Enhanced or impaired cognitive function in Parkinson's disease as a function of

dopaminergic medication and task demands. _Cereb. Cortex_ 11, 1136–1143 (2001). Article  CAS  PubMed  Google Scholar  * Frank, M.J., Seeberger, L.C. & O'Reilly, R.C. By carrot or by

stick: cognitive reinforcement learning in parkinsonism. _Science_ 306, 1940–1943 (2004). Article  CAS  PubMed  Google Scholar  * Shohamy, D., Myers, C.E., Geghman, K.D., Sage, J. &

Gluck, M.A. L-dopa impairs learning, but spares generalization, in Parkinson's disease. _Neuropsychologia_ 44, 774–784 (2006). Article  PubMed  Google Scholar  * Shohamy, D., Myers,

C.E., Grossman, S., Sage, J. & Gluck, M.A. The role of dopamine in cognitive sequence learning: evidence from Parkinson's disease. _Behav. Brain Res._ 156, 191–199 (2005). Article 

CAS  PubMed  Google Scholar  * Benedetti, F. et al. Placebo-responsive Parkinson patients show decreased activity in single neurons of subthalamic nucleus. _Nat. Neurosci._ 7, 587–588

(2004). Article  CAS  PubMed  Google Scholar  * Benedetti, F. et al. Electrophysiological properties of thalamic, subthalamic and nigral neurons during the anti-parkinsonian placebo

response. _J. Physiol. (Lond.)_ 587, 3869–3883 (2009). Article  CAS  Google Scholar  * Starkstein, S.E. et al. Reliability, validity, and clinical correlates of apathy in Parkinson's

disease. _J. Neuropsychiatry Clin. Neurosci._ 4, 134–139 (1992). Article  CAS  PubMed  Google Scholar  * Spielberger, C.D. _State-Trait Anxiety Inventory: Bibliography, 2nd edn._ (Consulting

Psychologists Press, 1989). * Movement Disorder Society Task Force on Rating Scales for Parkinson's Disease. The Unified Parkinson's Disease Rating Scale (UPDRS): status and

recommendations. _Mov. Disord._ 18, 738–750 (2003). * Sutton, R.S. & Barto, A.G. _Reinforcement Learning: an Introduction_ (Bradford Book, 1998). * Lindquist, M.A., Spicer, J., Asllani,

I. & Wager, T.D. Estimating and testing variance components in a multi-level GLM. _Neuroimage_ 59, 490–501 (2012). Article  PubMed  Google Scholar  * Mazaika, P., Hoeft, F., Glover, G.H.

& Reiss, A.L. Methods and software for fMRI analysis of clinical subjects. _Organ. Hum. Brain Mapp_ 475 (2009). * Hare, T.A., O'Doherty, J., Camerer, C.F., Schultz, W. &

Rangel, A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. _J. Neurosci._ 28, 5623–5630 (2008). Article  CAS 

PubMed  PubMed Central  Google Scholar  * Pessiglione, M., Seymour, B., Flandin, G., Dolan, R.J. & Frith, C.D. Dopamine-dependent prediction errors underpin reward-seeking behavior in

humans. _Nature_ 442, 1042–1045 (2006). Article  CAS  PubMed  PubMed Central  Google Scholar  * Behrens, T.E., Hunt, L.T., Woolrich, M.W. & Rushworth, M.F. Associative learning of social

value. _Nature_ 456, 245–249 (2008). Article  CAS  PubMed  PubMed Central  Google Scholar  * Niv, Y., Edlund, J.A., Dayan, P. & O'Doherty, J.P. Neural prediction errors reveal a

risk-sensitive reinforcement-learning process in the human brain. _J. Neurosci._ 32, 551–562 (2012). Article  CAS  PubMed  PubMed Central  Google Scholar  * Li, J., Delgado, M.R. &

Phelps, E.A. How instructed knowledge modulates the neural systems of reward learning. _Proc. Natl. Acad. Sci. USA_ 108, 55–60 (2011). Article  PubMed  Google Scholar  * Maldjian, J.A.,

Laurienti, P.J., Kraft, R.A. & Burdette, J.H. An automated method for neuroanatomic and cytoarchitectonic atlas–based interrogation of fMRI data sets. _Neuroimage_ 19, 1233–1239 (2003).

Article  PubMed  Google Scholar  Download references ACKNOWLEDGEMENTS We thank neurologists P. Greene, R. Alcalay, L. Coté and the nursing staff of the Center for Parkinson's Disease

and Other Movement Disorders at Columbia University Presbyterian Hospital for help with patient recruitment and discussion of the findings, N. Johnston and B. Vail for help with data

collection, M. Sharp, K. Duncan, D. Sulzer, J. Weber and B. Doll for insightful discussion, and M. Pessiglione and G.E. Wimmer for helpful comments on an earlier version of the manuscript.

This study was supported by the Michael J. Fox Foundation and the US National Institutes of Health (R01MH076136). AUTHOR INFORMATION Author notes * Tor D Wager and Daphna Shohamy: These

authors contributed equally to this work. AUTHORS AND AFFILIATIONS * Psychology Department, Columbia University, New York, New York, USA Liane Schmidt, Erin Kendall Braun & Daphna

Shohamy * Psychology Department, University of Colorado at Boulder, Boulder, Colorado, USA Tor D Wager * Kavli Center for Brain Science, Columbia University, New York, New York, USA Daphna

Shohamy Authors * Liane Schmidt View author publications You can also search for this author inPubMed Google Scholar * Erin Kendall Braun View author publications You can also search for

this author inPubMed Google Scholar * Tor D Wager View author publications You can also search for this author inPubMed Google Scholar * Daphna Shohamy View author publications You can also

search for this author inPubMed Google Scholar CONTRIBUTIONS D.S. and T.D.W. planned the experiment. L.S., T.D.W. and D.S. developed the experimental design. L.S. and E.K.B. collected data.

L.S. analyzed data. D.S. and T.D.W. supervised and assisted in data analysis. L.S., E.K.B., T.D.W. and D.S. wrote the manuscript. CORRESPONDING AUTHORS Correspondence to Tor D Wager or

Daphna Shohamy. ETHICS DECLARATIONS COMPETING INTERESTS The authors declare no competing financial interests. INTEGRATED SUPPLEMENTARY INFORMATION SUPPLEMENTARY FIGURE 1 EXPERIMENTAL AND

TASK DESIGN. (A) Experimental design. Each patient was scanned 3 times: off drug, on placebo, and on drug. Off drug and placebo scan sessions were counterbalanced across patients (order 1: 8

patients; order 2: 10 patients). Scan sessions lasted for 1 h and were separated by a 1 h break. Placebo and dopaminergic medication were crushed into orange juice and administered 30 min

before the respective scan session. (B) Task structure. The outcome of a trial could be a gain of $10, nothing ($0), or a loss of $10. Two cue pairs, a gain and a loss cue pair, were

randomly intermixed. Within the gain cue pair, optimal choices led to a gain of $10 with a probability of.75 and to nothing ($0) with a probability of.25. Within the loss cue pair, optimal

choices led to nothing ($0) with a probability of.75 and to a loss of $10 with a probability of.25. SUPPLEMENTARY FIGURE 2 PARTIAL CORRELATIONS BETWEEN ON DRUG AND PLACEBO CONTROLLING FOR

THE EFFECTS OF OFF DRUG (_N_ = 18). Partial correlations between on drug and placebo controlling for effects of off drug for reward learning (left) (r = 0.34, p = 0.08) and motor symptoms

(right) (r = 0.59, p = 0.01). On drug and placebo expressed as residuals after confounds due to off drug effects were regressed out. SUPPLEMENTARY FIGURE 3 BOLD RESPONSES TO CHOICES AND

FEEDBACK IN THE VMPFC AND THE VENTRAL STRIATUM (_N_ = 15). (A) Parameter estimates for choices (correct vs. incorrect) from the vmPFC region of interest in the gain and loss condition for

off drug (gray), placebo (blue), and on drug (black) treatments. (B) Parameter estimates for feedback (correct vs. incorrect) from the ventral striatum region of interest in the gain and

loss condition for the off drug (gray), placebo (blue), and on drug (black) treatments. Error bars represent within subject standard errors. SUPPLEMENTARY FIGURE 4 VENTRAL STRIATUM RESPONSES

TO COMPONENTS OF THE PREDICTION ERROR (_N_ = 15). Parameter estimates (betas) from the ventral striatum at time of outcome for the two components of prediction error – expected value and

reward – in the off drug (gray), placebo (blue) and on drug (black) treatments. Error bars represent within subject standard errors. SUPPLEMENTARY FIGURE 5 REACTION TIMES DURING LEARNING

ACROSS CONDITIONS AND TREATMENT. Average reaction time in seconds for the gain (left) and loss condition (right) in the off drug (gray), placebo (blue), and on drug (black) treatments. The

reaction time curves depict how fast patients chose the optimal choice cue. Error bars represent within-subject standard errors. SUPPLEMENTARY FIGURE 6 BEHAVIORAL RESULTS FOR THE SUBGROUP OF

PATIENTS SCANNED WITH FMRI (_N_ = 15). Percentage of observed optimal choices binned across bocks of 8 trials (left), smoothed (middle), and modeled optimal choices (right) in the gain

(top) and loss (bottom) conditions for the off drug (gray), placebo (blue), and on drug (black) treatments. The learning curves depict how often patients chose the 75% rewarding cue (t11 =

5.2, p < 0.001) during the gain condition and the 75% nothing cue (t11 = 4.1, p < 0.01) during the loss condition. Error bars represent within-subject standard errors. SUPPLEMENTARY

FIGURE 7 OBSERVED AND MODELED BEHAVIORAL RESULTS FOR THE SCANNED PATIENTS (N = 15). Percentage of observed behavioral choices (dots) and modeled optimal choices (solid lines) across trials

for off drug (gray), placebo (blue), and on drug (black) treatments. The learning curves depict how often patients chose the 75% rewarding cue (t11 = 5.2, p < 0.001) during the gain

condition and the 75% loss cue (t11 = −4.1, p < 0.01) during the loss condition. The modeled learning curves represent the probabilities of choice on each trial, as predicted by the RL

model. SUPPLEMENTARY FIGURE 8 DIRECT COMPARISONS OF VALUE AND PREDICTION ERROR RESPONSES ACROSS TREATMENTS. Statistical parametric maps (SPMs) are superimposed on the average structural

scan. (A) SPMs are masked for the vmPFC ROI, defined a priori by MNI = [−1, 27, −8]) from Hare et al. 2008, at p < 0.05 uncorrected. (B) Whole brain activations for value at p < 0.05

uncorrected. (C) SPMs are masked for the ventral striatum ROI, defined a priori by MNI = [−10, 12, −8]) from Pessiglione et al. 2006, at p < 0.05 uncorrected. (D) Whole brain activations

for prediction error at p < 0.05 uncorrected. SUPPLEMENTARY INFORMATION SUPPLEMENTARY TEXT AND FIGURES Supplementary Figures 1–8 and Supplementary Tables 1–2 (PDF 795 kb) SUPPLEMENTARY

METHODS CHECKLIST (PDF 530 kb) RIGHTS AND PERMISSIONS Reprints and permissions ABOUT THIS ARTICLE CITE THIS ARTICLE Schmidt, L., Braun, E., Wager, T. _et al._ Mind matters: placebo enhances

reward learning in Parkinson's disease. _Nat Neurosci_ 17, 1793–1797 (2014). https://doi.org/10.1038/nn.3842 Download citation * Received: 15 April 2014 * Accepted: 16 September 2014 *

Published: 19 October 2014 * Issue Date: December 2014 * DOI: https://doi.org/10.1038/nn.3842 SHARE THIS ARTICLE Anyone you share the following link with will be able to read this content:

Get shareable link Sorry, a shareable link is not currently available for this article. Copy to clipboard Provided by the Springer Nature SharedIt content-sharing initiative