Pearce-Hall error learning theory
| Geoffrey Hall (2008), Scholarpedia, 3(2):5274. | doi:10.4249/scholarpedia.5274 | revision #91640 [link to/cite this article] |
The Pearce-Hall model (1980) describes the circumstances in which an event comes to be established as a signal for its consequences. The model was developed in the context of classical (Pavlovian) conditioning in which the signal is referred to as a conditioned stimulus (CS) and the consequence as an unconditioned stimulus (US); it thus dealt, specifically, with the conditions necessary for the formation of an associative link between the central representations of a CS and a US. The model proposed that effectiveness of concurrent activation of the two representations in establishing or strengthening a link between them depends on the associability of (informally, the attention paid to) the CS. The associability of a CS is held to change as a result of experience, declining when a CS accurately predicts its consequences, and increasing or remaining high when a CS is followed by unpredictable consequences. These proposals capture the intuition that an animal needs to attend to and fully process an event the consequences of which are uncertain, but may deal differently with an event the consequences of which are known. The model dealt with many of the same phenomena as were addressed by the earlier Rescorla-Wagner model of conditioning, but whereas the latter explained them in terms of learned changes in US effectiveness, the Pearce-Hall model ascribed them to changes in CS effectiveness. The Pearce-Hall model was thus able to extend the analysis to a range of attentional phenomena not dealt with by its predecessor.
Contents |
Background
The 1970s saw a revolution in the approach to conditioning as a result of experimental studies showing that contiguity (the concurrent presentation of CS and US) was not enough to ensure the formation of an association. A critical phenomenon was that known as blocking - the observation that preliminary training in which one CS (X) is paired with the US will block conditioning to CS A when the AX compound is paired with the US. Stimulus A and the US co-occur, but A fails to gain associative strength. Latent inhibition provides another example. In this procedure the event to be used as the CS is repeatedly presented alone, prior to CS-US pairings. These pairings are, at least initially, ineffective in endowing the CS with associative strength. The model explains these, and related, phenomena in terms of learned changes in CS associability.
The model
According to the model, the change (\(\Delta\)) in the associative strength (symbolized V) of a CS as result of a CS-US pairing is governed by the following equation\[\Delta V = S.\alpha.\lambda\] (Eq. 1)
where S is determined by the intensity of the CS and \(\lambda\) by the intensity of the US. The parameter \(\alpha\) represents the associability of the CS and is assumed to be high for a novel CS.
The associability parameter is modified by experience according to the following equation\[\alpha_n = |\lambda - \Sigma V|_{n-1}\] (Eq. 2)
where \(\Sigma V\) represents the sum of the associative strengths of all stimuli present on trial n-1. That is, the value of \(\alpha\) on trial n is set by the absolute value of the discrepancy between \(\lambda\) and summed associative strength experienced on the preceding trial.
In simple conditioning, in which a CS is reliably paired with a US, V increases trial by trial and \(\alpha\) declines, approaching zero as asymptote is reached.
Applications
1) Blocking. In the blocking procedure the value of V for CS X rises to asymptote (\(\lambda\)) during the first phase of training. The value of \(\alpha\) for the added CS (A) is high on the first compound (AX) trial and A acquires strength on this trial (Eq. 1). But the presence of the trained CS X means that \(\alpha\) for A falls to zero (Eq. 2) as a result of this trial, and no further acquisition occurs despite continued pairing of AX with the US.
2) Latent inhibition. For a stimulus presented alone, Eq. 2 implies that its \(\alpha\)-value will fall to zero, thus precluding the acquisition of associative strength when the CS is first paired with a US. (On subsequent trials the discrepancy between \(\lambda\) and V will restore the value of \(\alpha\ ,\) and conditioning will start to occur.)
3) Latent inhibition during conditioning. Eq. 2 implies that the \(\alpha\)-value of a CS will fall to zero whenever it is accompanied by a consistent consequence (or US). It follows that latent inhibition (taken to be the decline in \(\alpha\)) will occur during the standard conditioning procedure. This unique prediction of the model has been confirmed by studies showing that a CS that has been a reliable predictor of a consequence will be learned about only slowly when it is subsequently employed as a CS signalling some other consequence (Hall & Pearce, 1979).
4) Effect of inconsistency. When the consequence of an event changes from one presentation to the next the discrepancy between \(\lambda\) and V of Eq. 2 is maintained and with it the value of \(\alpha\ .\) Experimental study (e.g., Swan & Pearce, 1988) confirms that a CS treated in this way is learned about readily when it is subsequently subjected to an orthodox conditioning procedure.
Limitations and developments
1) In its original form the model made the \(\alpha\)-value of a CS on trial n entirely dependent on the state of affairs on trial n-1. In fact, changes in associability occur more gradually, and Eq. 2 can been modified to allow the value of \(\alpha\) on trial n to be determined by a weighted average of its values on a run of preceding trials. The basic principle remains as described.
2) Latent inhibition has been found to be a more complex phenomenon than envisaged by the model, with the accumulation of evidence to show the retardation of learning produced by stimulus pre-exposure depends not solely on the decline in \(\alpha\) but on associative learning that goes on during pre-exposure. The model has been extended to allow the possibility that (for example) pre-exposure allows the gradual formation of a CS-no event association that contributes to the effect. The decline in \(\alpha\) continues to play a role (being a consequence of the formation of the association).
3) The model was devised to accommodate evidence indicating that learning about the added CS occurred normally on the first trial of the blocking procedure. Subsequent research has shown that this is not the case - that some blocking occurs even on that trial. To deal with this finding it is necessary to assume that the reduced learning is a consequence of a loss of effectiveness by the US (the central proposition of the Rescorla-Wagner model). Adding this feature detracts from the purity of the original model, but it can be done without damage to the (parallel) mechanisms proposed for changing CS effectiveness (and indeed the notion that the effectiveness of a US might be subject to change by experience was an intrinsic part of the original model's account of inhibitory learning).
4) It has long been thought that attention might increase to an event that is a good predictor of its consequences (an effect known as acquired distinctiveness). The model explicitly denies this, in that associability is supposed to decline in such circumstances, being restored only when unexpected consequences occur. Evidence taken to demonstrate the acquired distinctiveness effect must, therefore be explained in other terms. One possibility is to distinguish two forms of attention, one concerned with learning, the other with performance. The former (the associability parameter of the model) will decline when the consequences of a CS are certain and no further learning is required; but performance needs to be controlled by just such CSs. This second form of attention should increase for predictive CSs and its effects on performance could be responsible for the acquired distinctiveness phenomenon.
References
Hall G. and Pearce J.M. (1979) Latent inhibition of a CS during CS-US pairings. Journal of Experimental Psychology: Animal Behavior Processes, 5:31-42.
Pearce J.M. and Hall G. (1980) A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87:532-552.
Swan J.A. and Pearce J.M. (1988) The orienting response as an index of stimulus associability in rats. Journal of Experimental Psychology: Animal Behavior Processes, 14:292-301.
Internal references
- Nestor A. Schmajuk (2008) Classical conditioning. Scholarpedia, 3(3):2316.
- Peter Jonas and Gyorgy Buzsaki (2007) Neural inhibition. Scholarpedia, 2(9):3286.
- Florentin Woergoetter and Bernd Porr (2008) Reinforcement learning. Scholarpedia, 3(3):1448.
- Robert Rescorla (2008) Rescorla-Wagner model. Scholarpedia, 3(3):2237.
Recommended reading
Hall G. (1991) Perceptual and associative learning. Clarendon Press.
Pearce J.M and Bouton M.E. (2001) Theories of associative learning in animals. Annual Review of Psychology, 52:111-139.
Le Pelley M.E. (2004) The role of associative history in models of associative learning: A selective review and a hybrid model. Quarterly Journal of Experimental Psychology, 57B:193-243.
External links
Author's home page:
http://www.york.ac.uk/depts/psych/www/people/biogs/gh1.html
See also
Rescorla-Wagner model, classical conditioning, reinforcement learning, attention
