Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis.

PubWeight™: 2.33‹?› | Rank: Top 2%

🔗 View Article (PMC 3176615)

Published in Proc Natl Acad Sci U S A on March 09, 2011

Authors

Paul W Glimcher1

Author Affiliations

1: Center for Neuroeconomics, New York University, New York, NY 10003, USA. glimcher@cns.nyu.edu

Articles citing this

(truncated to the top 100)

Rethinking the emotional brain. Neuron (2012) 3.19

A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci (2013) 2.68

Updating dopamine reward signals. Curr Opin Neurobiol (2012) 2.14

Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci (2014) 2.14

From prediction error to incentive salience: mesolimbic computation of reward motivation. Eur J Neurosci (2012) 2.02

From perception to pleasure: music and its neural substrates. Proc Natl Acad Sci U S A (2013) 1.55

Attention, learning, and the value of information. Neuron (2012) 1.45

Neuronal Reward and Decision Signals: From Theories to Data. Physiol Rev (2015) 1.24

Evaluating the role of the alpha-7 nicotinic acetylcholine receptor in the pathophysiology and treatment of schizophrenia. Biochem Pharmacol (2013) 1.13

Coordinated activity of ventral tegmental neurons adapts to appetitive and aversive learning. PLoS One (2012) 1.00

Quantification of behavior. Proc Natl Acad Sci U S A (2011) 0.98

Multiple memory systems as substrates for multiple decision systems. Neurobiol Learn Mem (2014) 0.98

Isolating the delay component of impulsive choice in adolescent rats. Front Integr Neurosci (2014) 0.98

Establishing causality for dopamine in neural function and behavior with optogenetics. Brain Res (2012) 0.96

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia. Neurosci Biobehav Rev (2013) 0.95

Assessment of reward responsiveness in the response bias probabilistic reward task in rats: implications for cross-species translational research. Transl Psychiatry (2013) 0.93

Novelty or surprise? Front Psychol (2013) 0.93

Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors. Nat Neurosci (2015) 0.93

Microstimulation of the human substantia nigra alters reinforcement learning. J Neurosci (2014) 0.93

Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework. Elife (2016) 0.93

Bidirectional modulation of substantia nigra activity by motivational state. PLoS One (2013) 0.92

Variability in Dopamine Genes Dissociates Model-Based and Model-Free Reinforcement Learning. J Neurosci (2016) 0.91

Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia. J Neurosci (2013) 0.91

Role of dopamine D2 receptors in human reinforcement learning. Neuropsychopharmacology (2014) 0.91

Basal ganglia circuit loops, dopamine and motivation: A review and enquiry. Behav Brain Res (2015) 0.90

Mefloquine and psychotomimetics share neurotransmitter receptor and transporter interactions in vitro. Psychopharmacology (Berl) (2014) 0.89

Dopamine-associated cached values are not sufficient as the basis for action selection. Proc Natl Acad Sci U S A (2014) 0.88

Understanding interpersonal function in psychiatric illness through multiplayer economic games. Biol Psychiatry (2012) 0.87

How acute and chronic alcohol consumption affects brain networks: insights from multimodal neuroimaging. Alcohol Clin Exp Res (2012) 0.87

Explaining human recreational use of 'pesticides': The neurotoxin regulation model of substance use vs. the hijack model and implications for age and sex differences in drug consumption. Front Psychiatry (2013) 0.87

Striatal dopamine ramping may indicate flexible reinforcement learning with forgetting in the cortico-basal ganglia circuits. Front Neural Circuits (2014) 0.85

Real-time dopamine measurement in awake monkeys. PLoS One (2014) 0.84

Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry. Neuron (2015) 0.84

Task preparation processes related to reward prediction precede those related to task-difficulty expectation. Neuroimage (2013) 0.84

The effects of the previous outcome on probabilistic choice in rats. J Exp Psychol Anim Behav Process (2012) 0.84

Reward learning as a potential target for pharmacological augmentation of cognitive remediation for schizophrenia: a roadmap for preclinical development. Front Neurosci (2013) 0.83

Frank Beach Award Winner: Steroids as neuromodulators of brain circuits and behavior. Horm Behav (2014) 0.83

Reconsolidation of a well-learned instrumental memory. Learn Mem (2014) 0.82

Modulation of feedback-related negativity during trial-and-error exploration and encoding of behavioral shifts. Front Neurosci (2013) 0.81

A simple solution for model comparison in bold imaging: the special case of reward prediction error and reward outcomes. Front Neurosci (2013) 0.81

Mouse models of neurodevelopmental disease of the basal ganglia and associated circuits. Curr Top Dev Biol (2014) 0.81

In Your Face: Risk of Punishment Enhances Cognitive Control and Error-Related Activity in the Corrugator Supercilii Muscle. PLoS One (2013) 0.81

Release of endogenous cannabinoids from ventral tegmental area dopamine neurons and the modulation of synaptic processes. Prog Neuropsychopharmacol Biol Psychiatry (2014) 0.81

Many hats: intratrial and reward level-dependent BOLD activity in the striatum and premotor cortex. J Neurophysiol (2013) 0.80

Learning to represent reward structure: a key to adapting to complex environments. Neurosci Res (2012) 0.80

A Framework for Understanding the Emerging Role of Corticolimbic-Ventral Striatal Networks in OCD-Associated Repetitive Behaviors. Front Syst Neurosci (2015) 0.80

The rostromedial tegmental nucleus modulates behavioral inhibition following cocaine self-administration in rats. Neuropsychopharmacology (2014) 0.80

Relative gains, losses, and reference points in probabilistic choice in rats. PLoS One (2015) 0.79

Dopamine function and the efficiency of human movement. J Cogn Neurosci (2013) 0.79

Cannabinoids and value-based decision making: implications for neurodegenerative disorders. Basal Ganglia (2012) 0.79

Electrophysiological evidence for functionally distinct neuronal populations in the human substantia nigra. Front Hum Neurosci (2014) 0.79

Changes in corticostriatal connectivity during reinforcement learning in humans. Hum Brain Mapp (2014) 0.79

Corticostriatothalamic reward prediction error signals and executive control in late-life depression. Psychol Med (2014) 0.78

Rule learning enhances structural plasticity of long-range axons in frontal cortex. Nat Commun (2016) 0.78

Impaired neural response to negative prediction errors in cocaine addiction. J Neurosci (2015) 0.78

Two-phase model of the basal ganglia: implications for discontinuous control of the motor system. Philos Trans R Soc Lond B Biol Sci (2014) 0.78

Motivational Deficits in Schizophrenia and the Representation of Expected Value. Curr Top Behav Neurosci (2016) 0.78

Goal or gold: overlapping reward processes in soccer players upon scoring and winning money. PLoS One (2015) 0.78

Further evidence for ClockΔ19 mice as a model for bipolar disorder mania using cross-species tests of exploration and sensorimotor gating. Behav Brain Res (2013) 0.78

Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks. PLoS Comput Biol (2015) 0.77

Using fMRI to study reward processing in humans: past, present, and future. J Neurophysiol (2016) 0.77

Glutamate neurons are intermixed with midbrain dopamine neurons in nonhuman primates and humans. Sci Rep (2016) 0.77

Inferring reward prediction errors in patients with schizophrenia: a dynamic reward task for reinforcement learning. Front Psychol (2014) 0.77

Mild Reinforcement Learning Deficits in Patients With First-Episode Psychosis. Schizophr Bull (2016) 0.77

BRAIN REWARD ACTIVITY TO MASKED IN-GROUP SMILING FACES PREDICTS FRIENDSHIP DEVELOPMENT. Soc Psychol Personal Sci (2015) 0.77

Surprise beyond prediction error. Hum Brain Mapp (2014) 0.77

Morphological elucidation of basal ganglia circuits contributing reward prediction. Front Neurosci (2015) 0.77

Reward Anticipation Is Encoded Differently by Adolescent Ventral Tegmental Area Neurons. Biol Psychiatry (2015) 0.77

Dopamine-System Genes and Cultural Acquisition: The Norm Sensitivity Hypothesis. Curr Opin Psychol (2015) 0.77

Decoding the dopamine signal in macaque prefrontal cortex: a simulation study using the Cx3Dp simulator. PLoS One (2013) 0.76

Co-Evolution of Social Learning and Evolutionary Preparedness in Dangerous Environments. PLoS One (2016) 0.76

Expectation modulates neural representations of valence throughout the human brain. Neuroimage (2015) 0.76

Multiple brain networks contribute to the acquisition of bias in perceptual decision-making. Front Neurosci (2015) 0.76

Neural correlates of rules and conflict in medial prefrontal cortex during decision and feedback epochs. Front Behav Neurosci (2015) 0.76

Modeling dopaminergic and other processes involved in learning from reward prediction error: contributions from an individual differences perspective. Front Hum Neurosci (2014) 0.76

Dopaminergic Genetic Polymorphisms Predict Rule-based Category Learning. J Cogn Neurosci (2016) 0.76

The human subthalamic nucleus encodes the subjective value of reward and the cost of effort during decision-making. Brain (2016) 0.76

Levodopa administration modulates striatal processing of punishment-associated items in healthy participants. Psychopharmacology (Berl) (2014) 0.76

Expanding the role of striatal cholinergic interneurons and the midbrain dopamine system in appetitive instrumental conditioning. J Neurophysiol (2015) 0.76

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity. Front Neural Circuits (2016) 0.76

Effect of Frustration on Brain Activation Pattern in Subjects with Different Temperament. Front Psychol (2016) 0.75

Striatal Dopamine, Externalizing Proneness, and Substance Abuse: Effects on Wanting and Learning during Reward-Based Decision Making. Clin Psychol Sci (2016) 0.75

Ketamine decreases sensitivity of male rats to misleading negative feedback in a probabilistic reversal-learning task. Psychopharmacology (Berl) (2016) 0.75

Dopamine: Context and counterfactuals. Proc Natl Acad Sci U S A (2015) 0.75

'Proactive' use of cue-context congruence for building reinforcement learning's reward function. BMC Neurosci (2016) 0.75

Effects of affective arousal on choice behavior, reward prediction errors, and feedback-related negativities in human reward-based decision making. Front Psychol (2015) 0.75

Implicit learning on a probabilistic classification task in adults and adolescents with Bulimia Nervosa. J Psychiatr Res (2016) 0.75

From habits to self-regulation: how do we change? Yale J Biol Med (2012) 0.75

Motivational modes and learning in Parkinson's disease. Soc Cogn Affect Neurosci (2014) 0.75

Dissecting Neural Responses to Temporal Prediction, Attention, and Memory: Effects of Reward Learning and Interoception on Time Perception. Cereb Cortex (2014) 0.75

Learning from silver linings. Front Neurosci (2013) 0.75

A Flexible Mechanism of Rule Selection Enables Rapid Feature-Based Reinforcement Learning. Front Neurosci (2016) 0.75

Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning. Nat Commun (2016) 0.75

Striatal Associative Learning Signals Are Tuned to In-groups. J Cogn Neurosci (2016) 0.75

Neural computations underlying social risk sensitivity. Front Hum Neurosci (2012) 0.75

Transfer of conflict and cooperation from experienced games to new games: a connectionist model of learning. Front Neurosci (2015) 0.75

Neural signatures of experience-based improvements in deterministic decision-making. Behav Brain Res (2016) 0.75

Relative response cost determines the sensitivity of instrumental reward seeking to dopamine receptor blockade. Neuropsychopharmacology (2012) 0.75

Monetary reward speeds up voluntary saccades. Front Integr Neurosci (2014) 0.75

Criterion learning in rule-based categorization: simulation of neural mechanism and new data. Brain Cogn (2015) 0.75

Articles cited by this

A neural substrate of prediction and reward. Science (1997) 31.30

The framing of decisions and the psychology of choice. Science (1981) 23.80

Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J Physiol (1973) 14.99

A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci (1996) 9.54

Discrete coding of reward probability and uncertainty by dopamine neurons. Science (2003) 9.54

The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl) (2006) 9.18

Neurons in the orbitofrontal cortex encode economic value. Nature (2006) 8.59

Temporal difference models and reward-related learning in the human brain. Neuron (2003) 8.50

Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron (2005) 7.56

Temporal prediction errors in a passive learning task activate human striatum. Neuron (2003) 6.44

Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature (2009) 6.35

Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci (2000) 6.02

Representation of action-specific reward values in the striatum. Science (2005) 5.76

Cerebellar loops with motor cortex and prefrontal cortex of a nonhuman primate. J Neurosci (2003) 4.81

Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) (2006) 4.43

Opponent interactions between serotonin and dopamine. Neural Netw (2002) 4.12

The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci (2006) 4.06

Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience (1991) 4.05

The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J Physiol (2006) 3.97

Circuits and circuit disorders of the basal ganglia. Arch Neurol (2007) 3.97

Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci (2007) 3.96

Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev (1981) 3.88

Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci (1993) 3.83

Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav (2005) 3.83

EVIDENCE FOR THE EXISTENCE OF MONOAMINE-CONTAINING NEURONS IN THE CENTRAL NERVOUS SYSTEM. I. DEMONSTRATION OF MONOAMINES IN THE CELL BODIES OF BRAIN STEM NEURONS. Acta Physiol Scand Suppl (1964) 3.57

Intracellular and extracellular electrophysiology of nigral dopaminergic neurons--1. Identification and characterization. Neuroscience (1983) 3.09

A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci (2009) 3.08

Value representations in the primate striatum during matching behavior. Neuron (2008) 3.02

Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex (1998) 2.43

Bee foraging in uncertain environments using predictive hebbian learning. Nature (1995) 2.32

Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol (2008) 2.24

Action and outcome encoding in the primate caudate nucleus. J Neurosci (2007) 2.14

Basal-ganglia 'projections' to the prefrontal cortex of the primate. Cereb Cortex (2002) 1.98

A mathematical model for simple learning. Psychol Rev (1951) 1.94

Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol (2007) 1.85

Dopamine and synaptic plasticity in dorsal striatal circuits controlling action selection. Curr Opin Neurobiol (2009) 1.58

Macro-architecture of basal ganglia loops with the cerebral cortex: use of rabies virus to reveal multisynaptic circuits. Prog Brain Res (2004) 1.58

Controls of tonic and phasic dopamine transmission in the dorsal and ventral striatum. Mol Pharmacol (2009) 1.56

Mesencephalic dopamine neurons projecting to neocortex. Brain Res (1974) 1.54

Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron (2010) 1.44

The discovery of central monoamine neurons gave volume transmission to the wired brain. Prog Neurobiol (2009) 1.40

Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J Neurophysiol (2000) 1.33

Neurochemical architecture of the human striatum. J Comp Neurol (1997) 1.23

Effects of stress and aversion on dopamine neurons: implications for addiction. Neurosci Biobehav Rev (2010) 1.21

A model for stimulus generalization and discrimination. Psychol Rev (1951) 1.12

Topographic organization of ascending dopaminergic projections. Ann N Y Acad Sci (1988) 1.00

Electrical synapses between dopaminergic neurons of the substantia nigra pars compacta. J Neurosci (2005) 0.96

Electrical coupling between model midbrain dopamine neurons: effects on firing pattern and synchrony. J Neurophysiol (2002) 0.93