Models of basal ganglia

• Dr. Jim Houk, Northwestern University Medical School, Chicago IL

Figure 1: Distributed modular architecture linking cerebral cortex, basal ganglia and cerebellum (see (Houk 2005)). Green bi-directional arrows are used to signify the predominantly excitatory linkages between different areas of cerebral cortex; M1 designates the primary motor cortex which generates voluntary motor commands. Red bi-directional arrows are used to signify the predominantly inhibitory linkages between any given area of cerebral cortex and the basal ganglia. Blue bi-directional arrows show the linkages between most areas of cerebral cortex and the cerebellum. Blue is used to signify the dual loop of cerebellar interaction, one loop though cerebellar nucleus and another through cerebellar cortex. An individual cortical area together with its loops through basal ganglia and cerebellum form a powerful computational structure that has been dubbed a distributed processing module (DPM; (Houk 2005)). DPMs communicate with each other via the cortical-cortical connections. There are on the order of a hundred DPMs in the human brain, forming a large-scale neural network.

The basal ganglia are located interior to the cerebral cortex, and they receive prominent input from essentially all of the pallium, both isocortex and allocortex (Swanson 2000). Generally speaking, these inputs form relatively discrete channels that loop back to the same area of cerebral cortex from which they originated, a feature that is similar to the loops that form between the cerebellum and the cerebral cortex (Middleton and Strick 2000). This loop architecture, summarized in Figure 1, provides important perspective for this entry on models of basal ganglia. In the entry basal ganglia, anatomy and physiology are reviewed and the conclusion is reached that the two essential functions of the basal ganglia are action selection and reinforcement learning. In the present entry, key features of the computational architecture of the loops through the basal ganglia are first described. Then specific models of action selection are reviewed. Some of these models are anatomically and physiologically constrained, whereas others are abstract but are nevertheless motivated by behavioral functions of the basal ganglia. The level of modeling also varies from abstract, to population-based, to connectionist, to biophysical, to molecular. Hybrid models combine salient features at different neuroscientific levels.

Computational Architecture of the Basal Ganglia (BG)

This entry focuses on the loops through the basal ganglia shown on the left side of Figure 1. Each of these loops functions as a macroscopic signal processing module, being comprised of thousands of microscopic loops. Figure 2 provides an overview of the computational architecture of one macroscopic BG module. The architecture of just two of its microscopic loops is diagrammed and shown interacting with a reinforcement learning (RL) circuit. There is a highly divergent projection from large numbers of cerebral cortical neurons (eight CCs are shown) to the two input nuclei of the BG network, namely the striatum (shaded box containing six spiny neurons (SpNs)) and the subthalamic nucleus (STN). For simplicity, only the divergent projections from CC5 are shown explicitly. The outputs of the two microscopic modules loop back to focal sites in cortex (not explicitly shown) via thalamic neurons T1 and T2. The dashed box labeled RL controls dopamine (DA) neuron firing to regulate reinforcement learning.

Figure 2: One macroscopic signal processing module linking BG and cerebral cortex. The cerebral cortex sends divergent excitatory projections to a network (shaded box) of medium spiny neurons and to STN. The output neurons (ONs) of the BG are cells in globus pallidus pars interna (GPi) and in substantia nigra pars reticulata (SNr). The reciprocal excitatory thalamocortical loop that they project to is shown in green. Red open arrows designate inhibitory connections and purple shows the neuromodulatory connections of dopamine (DA) neurons located in the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA). The dashed box labeled RL designates the reinforcement learning circuit.

SpNs in the striatum are endowed with several features that make them ideal for classifying the complex patterns of activity sent from the cerebral cortex. These features include: (1) a high convergence ratio (Kincaid, Zheng and Wilson 1998) that presents nearly 20,000 different cortical inputs to any given spiny neuron, (2) a 3-factor learning rule that uses reward-predicting training signals from dopamine neurons to consolidate LTP learning (Houk, Adams and Barto 1995), (3) an attentional neuromodulatory factor (Nicola, Surmeier and Malenka 2000) that induces bistability and nonlinear amplification in spiny neurons (Gruber, Solla, Surmeier and Houk 2003), (4) competition among spiny neurons mediated by presynaptic and postsynaptic collateral inhibition (Plenz 2003; Houk, Bastianen, Fansler, Fishbach, Fraser, Reber, Roy and Simo 2007).

An unusual feature of loops through the basal ganglia is the presence of multiple inhibitory stages mediated by the neurotransmitter GABA. These are shown by the open arrow heads in the projections colored red. The most direct projection has two inhibitory stages. Since the output neurons (ONs) have high spontaneous rates, promoted by the excitatory projections from STN, when SpN1 fires a burst, it causes ON1 to pause, and this disinhibits discharge of T1, the targeted neuron in thalamus. This appears to initiate repetitive discharge in the reciprocal thalamocortical loop which activates a small cluster of CC neurons, amounting to the selection of action 1.

Indirect pathways through the basal ganglia encounter three inhibitory stages which results in increased firing in targeted ONs. For example, if SpN4 bursts, GPe2 pauses, which disinhibits ON2. This inhibits T2, thus helping to prevent action 2 from being selected. The projection via STN has a balancing action, since it excites both the GPe neurons and the ONs.

A newly reported architectural feature (Lei, Jiao, Mar and Reiner 2004) relevant here is that the cortical cells that participate in the thalamocortical loops that are disinhibited by BG output are not the output cells of cortex. These non-output cells probably project to pons and initiate activity in the loop through the cerebellum. This fits with the theory that the loop through the basal ganglia selects a ballpark estimate of an action which then needs to be amplified and refined by the loop through the cerebellum (Houk, Bastianen, Fansler, Fishbach, Fraser, Reber, Roy and Simo 2007). Cortical output cells instead project via striatum to the GPe cells that inhibit undesired actions.

Action Selection and Thought Selection

One of the first modern models of the basal ganglia, the Albin-DeLong model (Albin, Young and Penney 1989; DeLong 1990), was motivated by hypokinetic clinical manifestations of Parkinson’s disease. According to this model, over-activity of BG output neurons, due to under-activity of spiny neurons in the direct pathway, inhibits thalamocortical activity to cause akinesia. The corollary model for normal subjects is that bursts of spiny neuron activity in the direct pathway disinhibit their thalamocortical targets to initiate desired actions and bursts of spiny neuron discharge in the indirect pathways inhibit their thalamocortical targets to suppress unwanted actions ( Figure 2). This model has been applied to limb movements (Mink 1996) and to eye movements (Hikosaka, Matsumura, Kojima and Gardiner 1993), although the circuit for eye movements is slightly different since they are controlled by a subcortical structure, the superior colliculus (McHaffie, Stanford, Stein, Coizet and Redgrave 2005). The clinical model tends to emphasize the M1 loop in Figure 1 for the control of action, but this has been extended to cognitive loops for the control of thinking (Graybiel 1997).

Wickens’(1993) theory of the striatum invoked a combination of local biophysical and global connectionist mechanisms – it is a hybrid model. The biophysical level dealt with the neurodynamics of competition between spiny neurons in the striatum (Wickens, Alexander and Miller 1991; Wickens and Oorschot 2000). Spiny neurons were given a potassium ion channel and synaptic input, and the large aspiny interneurons that project to them were given cholinergic synaptic transmission that had modulatory effects on the spiny neuron potassium channel. The aspiny interneurons (not explicitly shown in Figure 2) were given neuromodulatory effects of DA input to them. DA and cholinergic activity reciprocally modulated the degree of competition. Another key feature was the organization of spiny neurons into domains of competition that were based on cellular anatomy. The model predicted many peaks of activity surrounded by troughs of inhibition, which the authors related to the selection of muscle groups controlling actions. This is a more intricate version of the Hikosaka, Takikawa & Kawagoe (2000) neurophysiological model of gaze control, which explains how a salient target for eye movement can be selected through the direct pathway while many alternative targets that are less salient can be de-selected through indirect pathways.

At the global level, Wickens’ (1993) theory expanded on the Albin-DeLong model, postulating that disinhibition of thalamocortical loops leads to amplification in the cortical cell assemblies envisioned by Hebb (1949). Closely associated is Hebb’s postulate that a local learning rule (LTP) would allow the formation of cell assemblies as a function of repeated reactivations, as might occur when one practices an action. Transmission through BG in direct pathways that target a cell assembly would “ignite” positive feedback as in the active state of an attractor network, whereas transmission through indirect BG pathways would serve to dampen positive feedback and flatten the attractor landscape. An intricate balance of basal ganglionic control over the cerebral cortex would be required for stable performance of the cortical network on combined operational and learning time scales. Miller & Wickens (1991) discuss these problems and relate the model to selective attention and to the representation of predictable and controllable events.

The model of motor planning and control by Houk & Wise (1994) includes loops through the cerebellum (CB) in addition to loops through BG. This model’s architecture is founded on anatomical modularity in the spatially distributed network summarized in Figure 1. The signal processing operations in the loops were deduced from extracellular recordings of neuronal activity in behaving animals. The most recent version of this model (Houk 2005) incorporates key cellular and molecular neuroscientific data that highlight unique computational features and special learning rules for spiny neurons in the BG loop and for Purkinje cells in the loop through cerebellar cortex. Each of the CB loops shown in Figure 1 includes a reciprocal excitatory loop through the cerebellar nucleus that transmits positive feedback and behaves like an attractor network, analogous to the Hebbian cortical assemblages in Wickens’ global model. The prominent inhibitory projections from Purkinje cells to nuclear cells are believed to exert powerful control over the fixed points of the CB attractor networks (Houk and Mugnaini 2003). This mechanism might resolve the stability problems faced by Wickens’ global model.

Redgrave, Prescott & Gurney (1999a) combine bottom-up and top-down approaches toward modeling the loops through the basal ganglia. The bottom-up models stem from neuroscientific data and resemble the models already discussed. In contrast, the top down approach starts with a major behavioral problem that the organism faces. They draw on ideas from ethology and cybernetics to identify and characterize the action selection problem faced by an autonomous being. Their theory is that the basal ganglia are the vertebrate solution to the action selection problem. This theory fits well with the Actor-Critic model discussed in the section below on reinforcement learning. Abstract models of the Actor implement action selection in an AI framework (see Chapter 2 of Sutton & Barto (1998)).

Gurney, Prescott & Redgrave’s (2001) population-based model gave a somewhat different interpretation of the computational architecture of the BG, particularly the GPe stage in Figure 2. They left action selection in the direct pathway through BG and de-selection in the indirect pathway from cortex to STN to ON, but they conceived the pathways to and from GPe as a control loop that stabilizes the former selection pathways and enhances selectivity. In a later article, Humphries & Gurney (2002) introduced a population-based model of the thalamocortical circuit and showed how it was compatible with the intrinsic model of BG processing and actually improved the former’s performance of action selection.

Since many of the loops through the basal ganglia subserve cognitive regions of cerebral cortex, BG architecture clearly participates in the selection of thoughts and plans as well as actions (reviewed by Prescott, Bryson & Seth (2007)). The PBWM model of prefrontal cortex and its loops through the basal ganglia (Hazy, Frank and O'Reilly 2006) has served to explain decision-making performance in several cognitive tasks. PBWM is an abstract implementation of the signal processing operations schematized in Figure 2. Rather than modeling a choice between several potential actions, these authors simulate GO versus NoGO decisions in a variety of cognitive tasks. Since there are actually thousands of microscopic modules that comprise any given macroscopic module, large numbers of potential actions or thoughts might be interesting to investigate. Frank has begun to enhance the PBWM model with neuroscientific features (Frank 2005; O'Reilly and Frank 2006).

The models by Rubchinsky, Kopell & Sigvardt (2003), and by Humphries, Stewart & Gurney (2006) are quite detailed in their biophysics. For example, the latter authors included effects of dopamine in STN and GPe, transmission delays between neurons, and specific distributions of synaptic input over dendrites, parameters that were derived from experimental studies. While the focus was on explaining how various oscillations resulted from feedback between STN and GPe, the model was also able to do action selection in a manner compatible with the Gurney et al. (2001) population-level model. The STN-GPe feedback loop was found to be functionally decoupled by tonic dopamine under normal conditions and recoupled by dopamine depletion which simulates many in vivo experimental conditions and the situation in Parkinson’s disease. There are starting to be many similarly detailed models of BG cells or circuits and more focus on temporal detail, namely the presence of high frequency oscillations in the networks and their models. It is not yet clear whether high frequency oscillations are simply epiphenomena or are actually used by the brain for signal processing.

The striatal beat frequency (SBF) model by Matell and Meck (2000; 2004) relies on lower frequency oscillations (∼5-15 Hz) to learn time intervals of seconds to minutes by rewarding correct actions. To summarize the operations of this model (see Figure 2 in Lustig et al 2005), a reset stimulus starts oscillatory activity in cortex which reverberates at different frequencies in different cortical units. Spiny neurons learn to detect patterns of activity in the cortical input vector, presumably through the operation of competitive pattern classification as summarized in Figure 2. Different spatial patterns of coincidence occur at different time intervals, so spiny neurons learn to detect the pattern that corresponds to the desired time interval on the basis of reinforcement learning. This model was tested by simultaneous recordings from ensembles of cortical and striatal neurons (Matell, Meck & Nicolelis 2003). The data supported an important role for this circuit in interval timing but did not support the oscillatory assumptions.

Serial Order Processing and Sequencing

Serial order processing, a crucial feature of higher order intelligence (Lashley 1951), was the focus of the Beiser & Houk (1998) hybrid model -- it was mainly connectionist but also included critical biophysical features. The rebound responses of thalamic neurons to disinhibitory output from BG loops are captured by T-type calcium ion channels and competitive pattern classification utilized GABAa collateral inhibition between SpNs. The model’s capacity for encoding the serial order of sensory events results from three computational features that combine in a cooperative manner: (1) a classification of patterns in the cortical input vector by computations within the striatum, (2) working memory of the outcome of pattern classification in cortical-thalamic loops, (3) a recursion-like operation brought about because the loop deposits the working memory of prior classifications into an updated input vector to striatum from cortex. The updated vector represents not only current events but also prior events that function as temporal context.

Spiny neurons have extensive GABAergic inhibitory collaterals within the striatum (shaded box in Figure 2) capable of making the SpN pattern classification operation competitive. Collateral inhibition is deemed an effective mechanism for mediating competition in some models (Beiser and Houk 1998; Plenz 2003) and an ineffective mechanism in others (Tepper, Koos & Wilson (2004)). In Houk et al. (2007), two kinds of collateral inhibition were modeled – postsynaptic and presynaptic. While postsynaptic inhibition was found to be effective, presynaptic inhibition was appreciably more so. Since presynaptic inhibition in the striatum appears to be phylogenetically more recent, this might help to explain the central paradox of schizophrenia that is being modeled by Crow (1997).

In the Botvinick & Plaut (2006) connectionist model, sequence information is encoded through sustained patterns of activation within a simple recurrent network (SRN) architecture. This computationally simple model provides a parsimonious account for numerous benchmark characteristics of immediate serial recall, including data that have been considered to preclude the application of recurrent neural networks in this domain. Unlike most competing accounts, the model deals naturally with findings concerning the role of background knowledge in serial order recall from working memory.

Many of our sequential actions rely on long-term memories that might be stored in BG or in cortex. Wickens & Arbuthnott (1993) model sequential actions by interactions in a network of SpNs. In contrast, Berns & Sejnowski (1998) propose a systems-level model based on delays in the loop between GPe and STN (not explicitly shown in Figure 2). When monkeys perform sequential saccades, interesting covariances develop between the striatum and frontal cortex that may serve to mark onsets and offsets of learned sequences (Fujii and Graybiel 2005).

Reinforcement Learning

Numerous models have addressed the basal ganglia's role in learning, based on the uncanny resemblance of dopaminergic cell firing to the requirements of an error signal in temporal difference (TD) learning. Such models map the Actor-Critic implementation of TD learning on to the basal ganglia (Barto, 1995; Houk, Adams & Barto, 1995; Montague, Dayan & Sejnowski, 1996; Schultz, Dayan & Montague, 1997), where, roughly, the Actor is mapped on to the selection function of the basal ganglia, and the Critic is mapped on to the RL circuit in Figure 2. As such the dopamine signal is envisaged as the teaching signal that alters the Actor's responses to maximise future reward.

Joel, Niv and Ruppin (2002) evaluated both anatomical and computational perspectives of Actor-Critic models. They review several models and conclude that they are not compatible with current anatomical data.

Though computationally elegant in approach, critics of these models have variously argued for its biological implausibility, citing:

• dopamine cells increase firing to aversive, neutral, and rewarding stimuli, and thus cannot uniquely signal reward prediction error (Pennartz, 1996, however, see Ungless et al. 2004 for claims that the neurons responding to aversive stimuli are, in fact, not dopaminergic)
• dopamine cells fire before the stimulus can be fully perceived, and thus its value cannot be known at the time (Redgrave, Gurney & Prescott, 1999b)
• stimulus-reward learning can take place in the absence of dopamine, and thus it is not necessary for learning (Berridge, 2007)

Principal Components Analysis

The Bar-Gad, Morris & Bergman (2003) model promotes an alternative to action selection. It postulates that the main function of loops through BG is to reduce the dimensionality, perhaps using principal components analysis, of the representation of an action that is present in the area of cortex that provides the input. Anatomical evidence supports this theory. There are about 10 times as many cortical inputs to the striatum as there are spiny neurons and about 100 times as many spiny neurons as there are output neurons. The input information needs to be prioritized and compressed. This seems quite compatible with the action selection hypothesis rather than being an alternative. The dimensionality of potential actions (or thoughts) that are salient is likely to be much less than the dimensionality of the activations that are sent to BG from any given area of cerebral cortex.

Network Dynamics

Models of basal ganglia networks have also been studied for their dynamic properties rather than to pursue overt functional hypotheses. A trio of STN-GP network models explored separate but complementary dynamical properties of this sub-system. The model of Humphries & Gurney (2001) explored the slow-bursting properties of STN and GP neurons in-vitro (Plenz & Kitai, 1999); the population-level model of Gillies, Willshaw & Li (2002) showed the range of dynamic responses that could be achieved, including oscillatory responses; and the more detailed biophysical model of Terman et al (2002) generalised these results to show how dynamics of the network depended heavily on the pattern of connections within and between the STN and GP.

Conclusions

The uniform architecture of the many loops through the basal ganglia (Figures 1 & 2) has been an important constraint for most models of BG. Spiny neurons in the striatum have computational properties that are well suited for a pattern classification of the input vectors from different areas of cerebral cortex (CC) or from thalamus. The output of any given loop then selects approximate actions or thoughts, or even time intervals. Activity in a corresponding loop through the cerebellum (CB) then proceeds to amplify and refine the approximate action or thought that was selected by the BG, so as to achieve more accuracy in guiding behavior and in thinking. According to this overall model, cooperative computations by BG—CC—CB distributed processing modules leads to more success in the many trying experiences that the world presents.

Bibliography

• Albin, R. L., A. B. Young and J. B. Penney (1989). "The functional anatomy of basal ganglia disorders." TINS 12(10): 366-375.
• Barto, A. G. (1995). Adaptive critics and the basal ganglia. Models of Information Processing in the Basal Ganglia. J.C. Houk J.L. Davis D.G. Beiser, MIT Press Cambridge.
• Barto, A.G., Singh, S., and Chentanez, N. (2004) Intrinsically Motivated Learning of Hierarchical Collections of Skills International Conference on Developmental Learning (ICDL), La Jolla, CA, USA
• Bar-Gad, I., G. Morris and H. Bergman (2003). "Information processing, dimensionality reduction and reinforcement learning in the basal ganglia." Prog in Neurobiology 71: 439-473.
• Beiser, D. G. and J. C. Houk (1998). "Model of cortical-basal ganglionic processing: Encoding the serial order of sensory events." Journal of Neurophysiology 79: 3168-3188.
• Berridge, K. C. (2007) The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl), 191, 391-431
• Berns, G. S. and T. J. Sejnowski (1998). "A computational model of how the basal ganglia produce sequences." Journal of Cognitive Neuroscience 10(1): 108-121.
• Botvinick, M. M. and D. C. Plaut (2006). "Short-term memory for serial order: A recurrent neural network model." Psychological Review 113: 201-233.
• Crow, T. J. (1997). "Is schizophrenia the price that Homo sapiens pays for language?" Schizophr Res 28(2-3): 127-41.
• DeLong, J. R. (1990). "Primate models of movement disorders of basal ganglia origin." Trends in Neuroscience 13: 281-285.
• Frank, M. J. (2005). "Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism." Journal of Cognitive Neuroscience 17: 51-72.
• Fujii, N. and A. M. Graybiel (2005). "Time-varying covariance of neural activities recorded in striatum and frontal cortex as monkeys perform sequential-saccade tasks." Proc Natl Acad Sci U S A 102(25): 9032-7.
• Gillies, A., Willshaw, D. and Li, Z. P. (2002) Subthalamic-pallidal interactions are critical in determining normal and abnormal functioning of the basal ganglia Proceedings of the Royal Society of London B, 269, 545-551
• Graybiel, A. M. (1997). "The basal ganglia and cognitive pattern generators." Schizophrenia Bulletin 23(3): 459-469.
• Gruber, A. J., S. A. Solla, D. J. Surmeier and J. C. Houk (2003). "Modulation of striatal single units by expected reward: A spiny neuron model displaying dopamine-induced bistability." J. Neurophysiology 90: 1095-1114.
• Gurney, K., T. J. Prescott and P. Redgrave (2001). "A computational model of action selection in the basal ganglia. I. A new functional anatomy." Biol. Cybern. 84: 401-410.
• Gurney, K., T. J. Prescott and P. Redgrave (2001). "A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour." Biol. Cybern. 84: 411-423.
• Hazy, T. E., M. J. Frank and R. C. O'Reilly (2006). "Banishing the homunculus: making working memory work." Neuroscience 139(1): 105-18.
• Hebb, D. O. (1949). The Organization of Behavior. New York, Wiley.
• Hikosaka, O., M. Matsumura, J. Kojima and T. W. Gardiner (1993). Role of basal ganglia in initiation and suppression of saccadic eye movements. Role of the Cerebellum and Basal Ganglia in Voluntary Movement. N. Mano, I. Hamada and M. Delong, Elsevier Science Publishers: 213-219.
• Hikosaka, O., Y. Takikawa and R. Kawagoe (2000). "Role of the basal ganglia in the control of purposive saccadic eye movements." Physiol Rev 80(3): 953-78.
• Houk, J. and E. Mugnaini (2003). Ch. 32: Cerebellum. Fundamental Neuroscience. L. R. * Squire, F. E. Bloom, J. L. Roberts and S. K. McConnell, Academic Press: 841-872.
• Houk, J. C. (2005). "Agents of the mind." Biological Cybernetics 92: 427-437.
• Houk, J. C., J. L. Adams and A. G. Barto (1995). A model of how the basal ganglia generates and uses neural signals that predict reinforcement. Models of Information Processing in the Basal Ganglia. J. C. Houk, J. L. Davis and D. G. Beiser. Cambridge, MA, MIT Press: 249-274.
• Houk, J. C., C. Bastianen, D. Fansler, A. Fishbach, D. Fraser, P. J. Reber, S. A. Roy and L. S. Simo (2007). "Action selection in subcortical loops through the basal ganglia and cerebellum." Phil. Trans. R. Soc. B.
• Houk, J. C. and S. P. Wise (1995). "Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: Their role in planning and controlling action." Cerebral Cortex 5: 95-110.
• Humphries, M. D. & Gurney, K. N. (2001) A pulsed neural network model of bursting in the basal ganglia Neural Networks, 14, 845-863
• Humphries, M. D. and K. N. Gurney (2002). "The role of intra-thalamic and thalamocortical circuits in action selection." Network 13(1): 131-56.
• Humphries, M. D., R. D. Stewart and K. Gurney (2006). "A physiologically plausible model of action selection and oscillatory activity in the Basal Ganglia." Journal of Neuroscience 26: 12921-12942.
• Joel D, Niv Y, Ruppin E (2002) Actor—critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks 15: 535-547
• Kincaid, A. E., T. Zheng and C. J. Wilson (1998). "Connectivity and convergence of single corticostriatal axons." J Neurosci 18(12): 4722-31.
• Lashley, K. S. (1951). The problem of serial order in behavior. Cerebral Mechanisms in Behavior. L. A. Jeffres, John Wiley & Sons, Inc.; New York: 112-136.
• Lei, W., Y. Jiao, N. D. Mar and A. Reiner (2004). "Evidence for Differential Cortical Input to Direct Pathway versus Indirect Pathway Striatal Projection Neurons in Rats." J Neuroscience 24(38): 8289 – 8299.
• Lustig, C., M. S. Matell and W. H. Meck (2005). "Not "just" a coincidence: Frontal-striatal interactions in working memory and interval timing." Memory 13(3/4): 441-448.
• Matell, M. S. and W. H. Meck (2000). "Neuropsychological mechanisms of interval timing behavior." BioEssays 22: 94-103.
• Matell, M. S. and W. H. Meck (2004). "Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes." Cognitive Brain Research 21: 139-170.
• Matell, M. S., W. H. Meck and M. A. L. Nicolelis (2003). "Interval Timing and the Encoding of Signal Duration by Ensembles of Cortical and Striatal Neurons." Behavioral Neuroscience 117(4): 760-773.
• McHaffie, J. G., T. R. Stanford, B. E. Stein, V. Coizet and P. Redgrave (2005). "Subcortical loops through the basal ganglia." Trends Neurosci 28(8): 401-407.
• Middleton, F. A. and P. L. Strick (2000). "Basal ganglia and cerebellar loops: motor and cognitive circuits." Brain Res. Rev. 31(2-3): 236-250.
• Miller, R. and J. R. Wickens (1991). "Corticostriatal cell assemblies in selective attention and in representation of predictable and controllable events." Concepts in Neuroscience 2(1): 65-95.
• Mink, J. W. (1996). "The basal ganglia: focused selection and inhibition of competing motor programs." Prog Neurobiol 50(4): 381-425.
• Montague, P. R., P. Dayan and T. J. Sejnowski (1996). "A framework for mesencephalic dopamine systems based on predictive Hebbian learning." J. Neuroscience 16(5): 1936-1947
• Nicola, S. M., J. Surmeier and R. C. Malenka (2000). "Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens." Annu Rev Neurosci 23: 185-215.
• O'Reilly, R. C. and M. J. Frank (2006). "Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia." Neural Computation 18: 283-328.
• Pennartz, C. M. (1995) The ascending neuromodulatory systems in learning by reinforcement: comparing computational conjectures with experimental findings Brain Res Brain Res Rev, 21, 219-245
• Plenz, D. (2003). "When inhibition goes incognito: feedback interaction between spiny projection neurons in striatal function." TINS 26: 14427–14432.
• Prescott, T. J., J. J. Bryson and A. K. Seth (2007). "Modelling natural action selection: An introduction to the theme issue." Philosophical Transactions of the Royal Society-B 362: (September)
• Redgrave, P., T. J. Prescott and K. Gurney (1999a). "The basal ganglia: A vertebrate solution to the selection problem?" Neuroscience 89(4): 1009-1023.
• Redgrave, P., Prescott, T. J. & Gurney, K. (1999b) Is the short-latency dopamine response too short to signal reward error? Trends in Neuroscience, 22, 146-151
• Rubchinsky, L. L., N. Kopel and K. A. Sigvardt (2003). "Modeling facilitation and inhibition of competing motor programs in basal ganglia subthalamic nucleus–pallidal circuits." PNAS 100: 14427–14432.
• Schultz, W., P. Dayan and P. R. Montague (1997). "A neural substrate of prediction and reward." Science 275: 1593-1599
• Sutton, R. S. and A. G. Barto (1998). Reinforcement Learning. An Introduction. Cambridge, MA, MIT Press.
• Swanson, L. W. (2000). "Cerebral hemisphere regulation of motivated behavior." Brain Research 886: 113-164.
• Tepper, J. M., T. Koos and C. J. Wilson (2004). "GABAergic microcircuits in the neostriatum." Trends Neurosci 27(11): 662-9.
• Terman, D., J. E. Rubin, A. C. Yew and C. J. Wilson (2002). "Activity patterns in a model for the subthalamopallidal network of the basal ganglia." J Neurosci 22(7): 2963-76.
• Wickens, J. (1993). A Theory of the Striatum. Oxford, Pergamon Press.
• Wickens, J. R., M. E. Alexander and R. Miller (1991). "Two dynamic modes of striatal function under dopaminergic-cholinergic control: simulation and analysis of a model." Synapse 8: 1-12.
• Wickens, J. R. and G. W. Arbuthnott (1993). "The corticostriatal system on computer simulation: an intermediate mechanism for sequencing of actions." Prog Brain Res 99: 325-39.
• Wickens, J. R. and D. E. Oorschot (2000). Neural dynamics and surround inhibition in the neostriatum: a possible connection. Brain Dynamics and the Striatal Complex. R. Miller. London, UK, CRC Press: p. 141.

Internal references

• John W. Milnor (2006) Attractor. Scholarpedia, 1(11):1815.
• Chris Eliasmith (2007) Attractor network. Scholarpedia, 2(10):1380.
• Peter Redgrave (2007) Basal ganglia. Scholarpedia, 2(6):1825.
• Valentino Braitenberg (2007) Brain. Scholarpedia, 2(11):2918.
• Eugene M. Izhikevich (2006) Bursting. Scholarpedia, 1(3):1300.
• James Meiss (2007) Dynamical systems. Scholarpedia, 2(2):1629.
• Keith Rayner and Monica Castelhano (2007) Eye movements. Scholarpedia, 2(10):3649.
• Eugene Roberts (2007) Gamma-aminobutyric acid. Scholarpedia, 2(10):3356.
• Peter Jonas and Gyorgy Buzsaki (2007) Neural inhibition. Scholarpedia, 2(9):3286.
• Jeff Moehlis, Kresimir Josic, Eric T. Shea-Brown (2006) Periodic orbit. Scholarpedia, 1(7):1358.
• Wolfram Schultz (2007) Reward. Scholarpedia, 2(3):1652.
• Wolfram Schultz (2007) Reward signals. Scholarpedia, 2(6):2184.
• Philip Holmes and Eric T. Shea-Brown (2006) Stability. Scholarpedia, 1(10):1838.
• Andrew G. Barto (2007) Temporal difference learning. Scholarpedia, 2(11):1604.
• S. Murray Sherman (2006) Thalamus. Scholarpedia, 1(9):1583.

See Also

Action Selection, Basal Ganglia, Cerebellum, Models of Cerebellum, PCA, Reinforcement Learning, Reward, Reward Signals

Sponsored by: Eugene M. Izhikevich, Editor-in-Chief of Scholarpedia, the peer-reviewed open-access encyclopedia

Sponsored by: Dr. Marc-Oliver Gewaltig, Blue Brain Project, École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland

Reviewed by: Anonymous

Accepted on: 2007-10-05 01:34:42 GMT