Cortical substrates for exploratory decisions in humans.

PubWeight™: 10.73‹?› | Rank: Top 0.1%

🔗 View Article (PMC 2635947)

Published in Nature on June 15, 2006

Authors

Nathaniel D Daw1, John P O'Doherty, Peter Dayan, Ben Seymour, Raymond J Dolan

Author Affiliations

1: Gatsby Computational Neuroscience Unit, University College London (UCL), Alexandra House, 17 Queen Square, London WC1N 3AR, UK. daw@gatsby.ucl.ac.uk

Associated clinical trials:

Evaluating the Neurocomputational Mechanisms of Explore-Exploit Decision Making in Older Adults | NCT05178381

Articles citing this

(truncated to the top 100)

The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology (2010) 10.17

The neural correlates of subjective value during intertemporal choice. Nat Neurosci (2007) 9.46

Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology (2010) 6.00

States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron (2010) 5.30

Associative learning of social value. Nature (2008) 4.76

Model-based influences on humans' choices and striatal prediction errors. Neuron (2011) 4.58

Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) (2006) 4.43

The social brain: neural basis of social knowledge. Annu Rev Psychol (2009) 4.17

Relating introspective accuracy to individual differences in brain structure. Science (2010) 3.68

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci (2007) 3.55

Risky business: the neuroeconomics of decision making under uncertainty. Nat Neurosci (2008) 3.50

Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A (2007) 3.33

A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci (2009) 3.25

A time of change: behavioral and neural correlates of adolescent sensitivity to appetitive and aversive environmental cues. Brain Cogn (2009) 3.22

Decision making in recurrent neuronal circuits. Neuron (2008) 3.19

The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron (2013) 3.03

Cognitive control in media multitaskers. Proc Natl Acad Sci U S A (2009) 2.97

Common and distinct networks underlying reward valence and processing stages: a meta-analysis of functional neuroimaging studies. Neurosci Biobehav Rev (2010) 2.94

Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J Neurosci (2007) 2.92

Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron (2011) 2.84

Changes of mind in decision-making. Nature (2009) 2.70

Neural signature of fictive learning signals in a sequential investment task. Proc Natl Acad Sci U S A (2007) 2.70

Decision making, movement planning and statistical decision theory. Trends Cogn Sci (2008) 2.67

Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci (2009) 2.64

Neural mechanisms of foraging. Science (2012) 2.62

Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain. J Neurosci (2011) 2.55

Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat Neurosci (2011) 2.55

The role of human orbitofrontal cortex in value comparison for incommensurable objects. J Neurosci (2009) 2.51

Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn Affect Behav Neurosci (2010) 2.47

The proactive brain: memory for predictions. Philos Trans R Soc Lond B Biol Sci (2009) 2.45

Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex (2008) 2.40

Value-based modulations in human visual cortex. Neuron (2008) 2.35

Diffusion-weighted imaging tractography-based parcellation of the human parietal cortex and comparison with human and macaque resting-state functional connectivity. J Neurosci (2011) 2.29

The neurobiology and genetics of impulse control disorders: relationships to drug addictions. Biochem Pharmacol (2007) 2.22

How choice reveals and shapes expected hedonic outcome. J Neurosci (2009) 2.12

Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci (2012) 2.09

Goals and habits in the brain. Neuron (2013) 2.08

Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J Neurosci (2011) 2.06

Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. Brain (2009) 2.04

Informatic parcellation of the network involved in the computation of subjective value. Soc Cogn Affect Neurosci (2013) 2.04

Dopamine restores reward prediction errors in old age. Nat Neurosci (2013) 2.00

Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron (2010) 1.99

Neural computations underlying action-based decision making in the human brain. Proc Natl Acad Sci U S A (2009) 1.98

An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J Neurosci (2010) 1.95

The neural representation of unexpected uncertainty during value-based decision making. Neuron (2013) 1.95

What we know and do not know about the functions of the orbitofrontal cortex after 20 years of cross-species studies. J Neurosci (2007) 1.93

Striatal activity underlies novelty-based choice in humans. Neuron (2008) 1.92

Gambling near-misses enhance motivation to gamble and recruit win-related brain circuitry. Neuron (2009) 1.90

Forebrain pain mechanisms. Brain Res Rev (2008) 1.89

How do the basal ganglia contribute to categorization? Their roles in generalization, response selection, and learning via feedback. Neurosci Biobehav Rev (2007) 1.84

Endocannabinoid signaling is critical for habit formation. Front Integr Neurosci (2007) 1.81

Separate neural mechanisms underlie choices and strategic preferences in risky decision making. Neuron (2009) 1.77

Emotion, cognition, and mental state representation in amygdala and prefrontal cortex. Annu Rev Neurosci (2010) 1.76

Conceptual challenges and directions for social neuroscience. Neuron (2010) 1.74

Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J Neurosci (2009) 1.72

Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J Neurosci (2009) 1.72

Neurocomputational models of basal ganglia function in learning, memory and choice. Behav Brain Res (2008) 1.71

Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J Neurosci (2011) 1.68

Functionally dissociable influences on learning rate in a dynamic environment. Neuron (2014) 1.67

Decision making as a window on cognition. Neuron (2013) 1.67

A reservoir of time constants for memory traces in cortical neurons. Nat Neurosci (2011) 1.66

Human reinforcement learning subdivides structured action spaces by learning effector-specific values. J Neurosci (2009) 1.66

Instructional control of reinforcement learning: a behavioral and neurocomputational investigation. Brain Res (2009) 1.65

A bayesian foundation for individual learning under uncertainty. Front Hum Neurosci (2011) 1.64

Posterior cingulate cortex: adapting behavior to a changing world. Trends Cogn Sci (2011) 1.64

Clan mentality: evidence that the medial prefrontal cortex responds to close others. J Neurosci (2010) 1.61

Cortical signals for rewarded actions and strategic exploration. Neuron (2013) 1.59

Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol Psychiatry (2010) 1.58

The exploration-exploitation dilemma: a multidisciplinary framework. PLoS One (2014) 1.58

A computational framework for the study of confidence in humans and animals. Philos Trans R Soc Lond B Biol Sci (2012) 1.58

A review of reward processing and motivational impairment in schizophrenia. Schizophr Bull (2013) 1.52

The human amygdala and orbital prefrontal cortex in behavioural regulation. Philos Trans R Soc Lond B Biol Sci (2007) 1.52

Cognitive impairment in pain through amygdala-driven prefrontal cortical deactivation. J Neurosci (2010) 1.52

Evaluating self-generated decisions in frontal pole cortex of monkeys. Nat Neurosci (2009) 1.52

Mind matters: placebo enhances reward learning in Parkinson's disease. Nat Neurosci (2014) 1.51

Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb Cortex (2011) 1.49

Serotonin selectively modulates reward value in human decision-making. J Neurosci (2012) 1.49

Lateral habenula neurons signal errors in the prediction of reward information. Nat Neurosci (2011) 1.48

Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry (2014) 1.48

The neural code of reward anticipation in human orbitofrontal cortex. Proc Natl Acad Sci U S A (2010) 1.46

Decision-making during gambling: an integration of cognitive and psychobiological approaches. Philos Trans R Soc Lond B Biol Sci (2010) 1.46

Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron (2012) 1.46

Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Curr Biol (2009) 1.46

Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology (2010) 1.45

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput Biol (2011) 1.45

An "as soon as possible" effect in human intertemporal decision making: behavioral evidence and neural mechanisms. J Neurophysiol (2010) 1.44

Neural correlates of forward planning in a spatial decision task in humans. J Neurosci (2011) 1.43

Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex. PLoS Biol (2011) 1.41

Behavioral and neural properties of social reinforcement learning. J Neurosci (2011) 1.39

Review. Parallel studies of cocaine-related neural and cognitive impairment in humans and monkeys. Philos Trans R Soc Lond B Biol Sci (2008) 1.37

Separate prefrontal-subcortical circuits mediate different components of risk-based decision making. J Neurosci (2012) 1.36

Observing the observer (I): meta-bayesian models of learning and decision-making. PLoS One (2010) 1.36

Adaptive decision making and value in the anterior cingulate cortex. Neuroimage (2007) 1.36

A mechanistic account of value computation in the human brain. Proc Natl Acad Sci U S A (2010) 1.34

Reinforcement learning in professional basketball players. Nat Commun (2011) 1.34

With age comes wisdom: decision making in younger and older adults. Psychol Sci (2011) 1.33

Functional connectivity of reward processing in the brain. Front Hum Neurosci (2009) 1.32

Age-based differences in strategy use in choice tasks. Front Neurosci (2012) 1.30

Signals in human striatum are appropriate for policy update rather than value prediction. J Neurosci (2011) 1.29

Alterations in choice behavior by manipulations of world model. Proc Natl Acad Sci U S A (2010) 1.29

Articles cited by this

An integrative theory of prefrontal cortex function. Annu Rev Neurosci (2001) 36.82

Optimal foraging, the marginal value theorem. Theor Popul Biol (1976) 23.27

Separate neural systems value immediate and delayed monetary rewards. Science (2004) 13.85

Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science (2004) 11.67

Neural correlates of decision variables in parietal cortex. Nature (1999) 10.33

Matching behavior and the representation of value in the parietal cortex. Science (2004) 9.83

A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci (1996) 9.54

Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci (2001) 9.49

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci (2005) 9.16

Temporal difference models and reward-related learning in the human brain. Neuron (2003) 8.50

Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol (2004) 8.12

The architecture of cognitive control in the human prefrontal cortex. Science (2003) 7.99

Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron (2005) 7.56

Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science (2003) 6.76

Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol (2000) 6.61

Temporal prediction errors in a passive learning task activate human striatum. Neuron (2003) 6.44

Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci (2004) 5.54

Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat Rev Neurosci (2004) 5.39

FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage (2000) 5.15

Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron (2004) 4.60

The functional organization of the intraparietal sulcus in humans and monkeys. J Anat (2005) 3.24

The role of locus coeruleus in the regulation of cognitive performance. Science (1999) 2.89

Metalearning and neuromodulation. Neural Netw (2002) 2.77

Dopamine: generalization and bonuses. Neural Netw (2002) 2.75

The role of frontopolar cortex in subgoal processing during working memory. Neuroimage (2002) 2.54

The cognitive and neuroanatomical correlates of multitasking. Neuropsychologia (2000) 2.48

Cognitive planning in humans: neuropsychological, neuroanatomical and neuropharmacological perspectives. Prog Neurobiol (1997) 1.42

Articles by these authors

Neural systems supporting interoceptive awareness. Nat Neurosci (2004) 11.88

Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science (2004) 11.67

Empathy for pain involves the affective but not sensory components of pain. Science (2004) 11.33

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci (2005) 9.16

Temporal difference models and reward-related learning in the human brain. Neuron (2003) 8.50

Uncertainty, neuromodulation, and attention. Neuron (2005) 7.48

Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature (2006) 6.90

Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science (2003) 6.76

Frames, biases, and rational decision-making in the human brain. Science (2006) 6.67

Neural responses during anticipation of a primary taste reward. Neuron (2002) 6.40

Empathic neural responses are modulated by the perceived fairness of others. Nature (2006) 6.04

Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology (2010) 6.00

Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. Brain (2003) 5.48

States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron (2010) 5.30

Temporal difference models describe higher-order learning in humans. Nature (2004) 4.94

Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nat Neurosci (2004) 4.72

Model-based influences on humans' choices and striatal prediction errors. Neuron (2011) 4.58

The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci (2006) 4.55

Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) (2006) 4.43

Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci (2003) 4.21

Opponent interactions between serotonin and dopamine. Neural Netw (2002) 4.12

Regret and its avoidance: a neuroimaging study of choice behavior. Nat Neurosci (2005) 3.99

Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nat Neurosci (2003) 3.90

Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol (2006) 3.80

When fear is near: threat imminence elicits prefrontal-periaqueductal gray shifts in humans. Science (2007) 3.78

Relating introspective accuracy to individual differences in brain structure. Science (2010) 3.68

Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J Neurophysiol (2006) 3.53

Determining the neural substrates of goal-directed learning in the human brain. J Neurosci (2007) 3.31

Neural correlates of mentalizing-related computations during strategic interactions in humans. Proc Natl Acad Sci U S A (2008) 3.29

A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci (2009) 3.25

Context-dependent human extinction memory is mediated by a ventromedial prefrontal and hippocampal network. J Neurosci (2006) 3.17

Differential encoding of losses and gains in the human striatum. J Neurosci (2007) 3.16

Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain. Nat Neurosci (2004) 3.12

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci (2005) 3.05

Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex. J Neurosci (2009) 3.02

Clinical prediction rule for identifying children with cerebrospinal fluid pleocytosis at very low risk of bacterial meningitis. JAMA (2007) 2.91

Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci (2007) 2.83

Dopamine: generalization and bonuses. Neural Netw (2002) 2.75

Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron (2006) 2.72

Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J Neurosci (2002) 2.65

Neural evidence for inequality-averse social preferences. Nature (2010) 2.64

Inference and computation with population codes. Annu Rev Neurosci (2003) 2.62

Brain responses to the acquired moral status of faces. Neuron (2004) 2.61

Subliminal instrumental conditioning demonstrated in the human brain. Neuron (2008) 2.55

The role of human orbitofrontal cortex in value comparison for incommensurable objects. J Neurosci (2009) 2.51

Brain activity underlying encoding and retrieval of source memory. Cereb Cortex (2002) 2.48

Anterior cingulate activity during error and autonomic response. Neuroimage (2005) 2.47

Oxytocin attenuates affective evaluations of conditioned faces and amygdala activity. J Neurosci (2008) 2.41

Computational psychiatry. Trends Cogn Sci (2011) 2.40

Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex (2008) 2.40

The neural basis of mood-congruent processing biases in depression. Arch Gen Psychiatry (2002) 2.35

Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology (2010) 2.34

Anxiety reduction through detachment: subjective, physiological, and neural effects. J Cogn Neurosci (2005) 2.32

Neural correlates of value, risk, and risk aversion contributing to decision making under risk. J Neurosci (2009) 2.29

Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value. Nat Neurosci (2004) 2.25

Fear conditioning in humans: the influence of awareness and autonomic arousal on functional neuroanatomy. Neuron (2002) 2.22

An agent independent axis for executed and modeled choice in medial prefrontal cortex. Neuron (2012) 2.21

Attentional load and sensory competition in human vision: modulation of fMRI responses by load at fixation during task-irrelevant stimulation in the peripheral visual field. Cereb Cortex (2004) 2.19

Neural mechanisms of belief inference during cooperative games. J Neurosci (2010) 2.18

The misbehavior of value and the discipline of the will. Neural Netw (2006) 2.17

How humans integrate the prospects of pain and reward during choice. J Neurosci (2009) 2.14

How choice reveals and shapes expected hedonic outcome. J Neurosci (2009) 2.12

Brain systems for assessing facial attractiveness. Neuropsychologia (2006) 2.09

Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci (2012) 2.09

Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. J Neurosci (2010) 2.08

Threatening a rubber hand that you feel is yours elicits a cortical anxiety response. Proc Natl Acad Sci U S A (2007) 2.08

Encoding of emotional memories depends on amygdala and hippocampus and their interactions. Nat Neurosci (2004) 2.08

Goals and habits in the brain. Neuron (2013) 2.08

Human pavlovian-instrumental transfer. J Neurosci (2008) 2.06

Calculating consequences: brain systems that encode the causal effects of actions. J Neurosci (2008) 2.05

Task and content modulate amygdala-hippocampal connectivity in emotional retrieval. Neuron (2006) 2.05

Confidence in value-based choice. Nat Neurosci (2012) 2.00

Dopamine restores reward prediction errors in old age. Nat Neurosci (2013) 2.00