Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling.

@article{citeulike:1047089, abstract = {In Pavlovian and instrumental conditioning, reward typically comes seconds after reward-triggering actions, creating an explanatory conundrum known as "distal reward problem": How does the brain know what firing patterns of what neurons are responsible for the reward if 1) the patterns are no longer there when the reward arrives and 2) all neurons and synapses are active during the waiting period to the reward? Here, we show how the conundrum is resolved by a model network of cortical spiking neurons with spike-timing-dependent plasticity (STDP) modulated by dopamine (DA). Although STDP is triggered by nearly coincident firing patterns on a millisecond timescale, slow kinetics of subsequent synaptic plasticity is sensitive to changes in the extracellular DA concentration during the critical period of a few seconds. Random firings during the waiting period to the reward do not affect STDP and hence make the network insensitive to the ongoing activity-the key feature that distinguishes our approach from previous theoretical studies, which implicitly assume that the network be quiet during the waiting period or that the patterns be preserved until the reward arrives. This study emphasizes the importance of precise firing patterns in brain dynamics and suggests how a global diffusive reinforcement signal in the form of extracellular DA can selectively influence the right synapses at the right time.}, address = {The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121, USA.}, author = {Izhikevich, Eugene M M. }, citeulike-article-id = {1047089}, doi = {http://dx.doi.org/10.1093/cercor/bhl152}, issn = {1047-3211}, journal = {Cereb Cortex}, keywords = {abstract-model, dopamine, prefrontal-cortex, reinforcement-learning, stdp}, month = {January}, posted-at = {2007-04-27 13:25:13}, priority = {0}, title = {Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling.}, url = {http://dx.doi.org/10.1093/cercor/bhl152}, year = {2007} }

TY - JOUR ID - citeulike:1047089 L3 - citeulike-article-id:1047089 TI - Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. JF - Cereb Cortex AD - The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121, USA. SN - 1047-3211 N2 - In Pavlovian and instrumental conditioning, reward typically comes seconds after reward-triggering actions, creating an explanatory conundrum known as "distal reward problem": How does the brain know what firing patterns of what neurons are responsible for the reward if 1) the patterns are no longer there when the reward arrives and 2) all neurons and synapses are active during the waiting period to the reward? Here, we show how the conundrum is resolved by a model network of cortical spiking neurons with spike-timing-dependent plasticity (STDP) modulated by dopamine (DA). Although STDP is triggered by nearly coincident firing patterns on a millisecond timescale, slow kinetics of subsequent synaptic plasticity is sensitive to changes in the extracellular DA concentration during the critical period of a few seconds. Random firings during the waiting period to the reward do not affect STDP and hence make the network insensitive to the ongoing activity-the key feature that distinguishes our approach from previous theoretical studies, which implicitly assume that the network be quiet during the waiting period or that the patterns be preserved until the reward arrives. This study emphasizes the importance of precise firing patterns in brain dynamics and suggests how a global diffusive reinforcement signal in the form of extracellular DA can selectively influence the right synapses at the right time. KW - abstract-model KW - dopamine KW - prefrontal-cortex KW - reinforcement-learning KW - stdp AU - Izhikevich, Eugene M PY - 2007/01/13/ UR - http://dx.doi.org/10.1093/cercor/bhl152 ER -