Dopamine, prediction error and associative learning: A model-based account

Andrew Smith, Ming Li, Sue Becker, Shitij Kapur

Research output: Contribution to journalArticle

53 Citations (Scopus)

Abstract

The notion of prediction error has established itself at the heart of formal models of animal learning and current hypotheses of dopamine function. Several interpretations of prediction error have been offered, including the model-free reinforcement learning method known as temporal difference learning (TD), and the important Rescorla-Wagner (RW) learning rule. Here, we present a model-based adaptation of these ideas that provides a good account of empirical data pertaining to dopamine neuron firing patterns and associative learning paradigms such as latent inhibition, Kamin blocking and overshadowing. Our departure from model-free reinforcement learning also offers: 1) a parsimonious distinction between tonic and phasic dopamine functions; 2) a potential generalization of the role of phasic dopamine from valence-dependent "reward" processing to valence-independent "salience" processing; 3) an explanation for the selectivity of certain dopamine manipulations on motivation for distal rewards; and 4) a plausible link between formal notions of prediction error and accounts of disturbances of thought in schizophrenia (in which dopamine dysfunction is strongly implicated). The model distinguishes itself from existing accounts by offering novel predictions pertaining to the firing of dopamine neurons in various untested behavioral scenarios.

Original languageEnglish (US)
Pages (from-to)61-84
Number of pages24
JournalNetwork: Computation in Neural Systems
Volume17
Issue number1
DOIs
StatePublished - Mar 1 2006

Fingerprint

Dopamine
Learning
Dopaminergic Neurons
Reward
Motivation
Schizophrenia
Animal Models
Reinforcement (Psychology)

Keywords

  • Associative learning
  • Blocking
  • Dopamine
  • Incentive salience
  • Latent inhibition
  • Motivated behavior
  • Overshadowing
  • Prediction error
  • Psychosis
  • Reinforcement learning
  • Rescorla-Wagner learning rule
  • Schizophrenia
  • Temporal difference algorithm

ASJC Scopus subject areas

  • Neuroscience (miscellaneous)

Cite this

Dopamine, prediction error and associative learning : A model-based account. / Smith, Andrew; Li, Ming; Becker, Sue; Kapur, Shitij.

In: Network: Computation in Neural Systems, Vol. 17, No. 1, 01.03.2006, p. 61-84.

Research output: Contribution to journalArticle

Smith, Andrew ; Li, Ming ; Becker, Sue ; Kapur, Shitij. / Dopamine, prediction error and associative learning : A model-based account. In: Network: Computation in Neural Systems. 2006 ; Vol. 17, No. 1. pp. 61-84.
@article{4e86011ecb574c95a4cb958d18e9939f,
title = "Dopamine, prediction error and associative learning: A model-based account",
abstract = "The notion of prediction error has established itself at the heart of formal models of animal learning and current hypotheses of dopamine function. Several interpretations of prediction error have been offered, including the model-free reinforcement learning method known as temporal difference learning (TD), and the important Rescorla-Wagner (RW) learning rule. Here, we present a model-based adaptation of these ideas that provides a good account of empirical data pertaining to dopamine neuron firing patterns and associative learning paradigms such as latent inhibition, Kamin blocking and overshadowing. Our departure from model-free reinforcement learning also offers: 1) a parsimonious distinction between tonic and phasic dopamine functions; 2) a potential generalization of the role of phasic dopamine from valence-dependent {"}reward{"} processing to valence-independent {"}salience{"} processing; 3) an explanation for the selectivity of certain dopamine manipulations on motivation for distal rewards; and 4) a plausible link between formal notions of prediction error and accounts of disturbances of thought in schizophrenia (in which dopamine dysfunction is strongly implicated). The model distinguishes itself from existing accounts by offering novel predictions pertaining to the firing of dopamine neurons in various untested behavioral scenarios.",
keywords = "Associative learning, Blocking, Dopamine, Incentive salience, Latent inhibition, Motivated behavior, Overshadowing, Prediction error, Psychosis, Reinforcement learning, Rescorla-Wagner learning rule, Schizophrenia, Temporal difference algorithm",
author = "Andrew Smith and Ming Li and Sue Becker and Shitij Kapur",
year = "2006",
month = "3",
day = "1",
doi = "10.1080/09548980500361624",
language = "English (US)",
volume = "17",
pages = "61--84",
journal = "Network: Computation in Neural Systems",
issn = "0954-898X",
publisher = "Informa Healthcare",
number = "1",

}

TY - JOUR

T1 - Dopamine, prediction error and associative learning

T2 - A model-based account

AU - Smith, Andrew

AU - Li, Ming

AU - Becker, Sue

AU - Kapur, Shitij

PY - 2006/3/1

Y1 - 2006/3/1

N2 - The notion of prediction error has established itself at the heart of formal models of animal learning and current hypotheses of dopamine function. Several interpretations of prediction error have been offered, including the model-free reinforcement learning method known as temporal difference learning (TD), and the important Rescorla-Wagner (RW) learning rule. Here, we present a model-based adaptation of these ideas that provides a good account of empirical data pertaining to dopamine neuron firing patterns and associative learning paradigms such as latent inhibition, Kamin blocking and overshadowing. Our departure from model-free reinforcement learning also offers: 1) a parsimonious distinction between tonic and phasic dopamine functions; 2) a potential generalization of the role of phasic dopamine from valence-dependent "reward" processing to valence-independent "salience" processing; 3) an explanation for the selectivity of certain dopamine manipulations on motivation for distal rewards; and 4) a plausible link between formal notions of prediction error and accounts of disturbances of thought in schizophrenia (in which dopamine dysfunction is strongly implicated). The model distinguishes itself from existing accounts by offering novel predictions pertaining to the firing of dopamine neurons in various untested behavioral scenarios.

AB - The notion of prediction error has established itself at the heart of formal models of animal learning and current hypotheses of dopamine function. Several interpretations of prediction error have been offered, including the model-free reinforcement learning method known as temporal difference learning (TD), and the important Rescorla-Wagner (RW) learning rule. Here, we present a model-based adaptation of these ideas that provides a good account of empirical data pertaining to dopamine neuron firing patterns and associative learning paradigms such as latent inhibition, Kamin blocking and overshadowing. Our departure from model-free reinforcement learning also offers: 1) a parsimonious distinction between tonic and phasic dopamine functions; 2) a potential generalization of the role of phasic dopamine from valence-dependent "reward" processing to valence-independent "salience" processing; 3) an explanation for the selectivity of certain dopamine manipulations on motivation for distal rewards; and 4) a plausible link between formal notions of prediction error and accounts of disturbances of thought in schizophrenia (in which dopamine dysfunction is strongly implicated). The model distinguishes itself from existing accounts by offering novel predictions pertaining to the firing of dopamine neurons in various untested behavioral scenarios.

KW - Associative learning

KW - Blocking

KW - Dopamine

KW - Incentive salience

KW - Latent inhibition

KW - Motivated behavior

KW - Overshadowing

KW - Prediction error

KW - Psychosis

KW - Reinforcement learning

KW - Rescorla-Wagner learning rule

KW - Schizophrenia

KW - Temporal difference algorithm

UR - http://www.scopus.com/inward/record.url?scp=33646230819&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33646230819&partnerID=8YFLogxK

U2 - 10.1080/09548980500361624

DO - 10.1080/09548980500361624

M3 - Article

C2 - 16613795

AN - SCOPUS:33646230819

VL - 17

SP - 61

EP - 84

JO - Network: Computation in Neural Systems

JF - Network: Computation in Neural Systems

SN - 0954-898X

IS - 1

ER -