Suprathreshold stochastic resonance
| Mark D. McDonnell and Nigel Stocks (2009), Scholarpedia, 4(6):6508. | doi:10.4249/scholarpedia.6508 | revision #142760 [link to/cite this article] |
Suprathreshold Stochastic Resonance (SSR) is a variant of stochastic resonance (SR) that occurs for a specific set of conditions that are somewhat different from those of stochastic resonance. Like stochastic resonance, suprathreshold stochastic resonance describes the observation of noise enhanced behaviour in signal processing systems. Unlike conventional stochastic resonance, suprathreshold stochastic resonance does not disappear when the signal is no longer "subthreshold."
Introduction
Suprathreshold stochastic resonance was first demonstrated in arrays of identical threshold devices in 2000. This initial work (Stocks 2000) assumed an aperiodic random input signal (meaning that suprathreshold stochastic resonance is a form of aperiodic stochastic resonance), and stochastic resonance was shown to occur in the Shannon average mutual information between the input and output of the array. Like all forms of stochastic resonance, this means that the output performance is maximised by nonzero input noise. In the threshold array, a sufficient condition for suprathreshold stochastic resonance is that the array has more than one threshold device, and all threshold levels are set to the same value.
Most importantly, stochastic resonance occurs regardless of whether the input signal is entirely subthreshold or not, and indeed the suprathreshold stochastic resonance peak in performance is maximized when the threshold levels are set equal to the signal mean, and diminishes otherwise. This was the first known form of stochastic resonance occur under these conditions. The qualifier suprathreshold in suprathreshold stochastic resonance is therefore used to distinguish the effect from the occurrence of stochastic resonance in single-threshold systems, in which suprathreshold signals general do not give rise to stochastic resonance.
Other research has since demonstrated suprathreshold stochastic resonance need not be restricted to arrays of devices with "hard thresholds." The effect can occur in networks that have the following properties:
- the presence of multiple parallel signal processing devices or nodes;
- each device operates on a common input signal;
- the common input to each device is corrupted by a different noise source;
- each noisy version of the input signal is processed by a nonlinearity, such as a comparator.
Figure 1 shows a simple example that satisfies these properties. These properties alone are not sufficient for suprathreshold stochastic resonance to occur when the noise level varies. An additional set of assumptions that are known to lead to suprathreshold stochastic resonance include:
- each noise source has the same distribution;
- each noise source is independent of each other and of the signal;
- each nonlinearity is identical.
However, it is likely that not all of these conditions are necessary.
Suprathreshold stochastic resonance occurs when there is a tradeoff between degradation of the input signal due to random noise, and degradation due to deformation by each nonlinearity. This can be understood as follows.
- In the absence of noise, all degradation is due to the nonlinearities. If each nonlinearity is identical, the overall system's performance is the same as a single element's.
- For intermediate noise levels, the random noise alters the properties of the nonlinearities, so that each is effectively no longer identical. While any nonzero level of noise introduces randomness into the system's output, this is offset to some degree by the linearizing effects of the noise. The fact that multiple devices operate on a common signal allows what is essentially a nonlinear averaging effect to occur. This also means that performance increases for every additional device. The end result is that performance increases with increasing noise, up to some optimal noise level.
- For large noise, while performance still increases with the number of devices, degradation of the signal due to the randomness of the noise outweighs the effects of the noise of the nonlinearity, and performance only decreases with increasing noise.
How suprathreshold stochastic resonance compares with other forms of stochastic resonance
Conventional dynamical stochastic resonance: Many theoretical studies of suprathreshold stochastic resonance have been restricted to so called static threshold nonlinearities. Classical definitions of stochastic resonance are sometimes restricted to dynamical systems only. However, suprathreshold stochastic resonance has also been observed in arrays of dynamical neuron models, and consequently the dynamical or static nature of the nonlinearity is not what determines whether the effect occurs. Instead the central difference is that suprathreshold stochastic resonance does not rely on the input signal being entirely below a "threshold level," which is usually not the case for conventional stochastic resonance.
Aperiodic stochastic resonance: In nearly all published studies of suprathreshold stochastic resonance, the input signal is assumed to be an aperiodic random signal. In these cases suprathreshold stochastic resonance can be said to be a form of aperiodic stochastic resonance. However, since aperiodic stochastic resonance is a term usually reserved for subthreshold input signals. Furthermore, suprathreshold stochastic resonance can also occur for periodic input signals.
Array enhanced stochastic resonance: Due to the necessity for multiple noise sources, suprathreshold stochastic resonance is similar to array-enhanced stochastic resonance. The differences are (i) in array enhanced stochastic resonance, stochastic resonance occurs individually in each element of the array, which is not necessarily the case for suprathreshold stochastic resonance; (ii) array enhanced stochastic resonance assumes there is coupling between each element in the array.
Key theoretical results
The most studied model is the network shown in Figure 1. It consists of \(N\) identical threshold devices (i.e. single bit quantizers), each operating on a common signal \(x\) subject to independent additive noise. The overall output \(y\) is the sum of the \(N\) noisy binary outputs and is an integer between zero and \(N\ .\) The fidelity with which \(y\) can represent \(x\) is dependant on the noise distribution, the signal distribution, the input signal-to-noise-ratio, \(N\ ,\) and the threshold level.
Performance depends on the conditional distribution of \(y\) given \(x\ .\) For the assumption that each device is identical, this is given by the binomial distribution where the probability of single success depends on the cumulative distribution function of the noise evaluated at \(k-x\ ,\) where \(k\) is the common threshold level.
Description using mutual information
The most common metric for quantifying suprathreshold stochastic resonance is the mutual information \(I(x;y)\) between \(x\) and \(y\ ,\) although other measures such as mean square error (McDonnell et al. 2002), Fisher information (Rousseau et al. 2003), signal-to-noise ratio (Rousseau and Chapeau-Blondeau 2004) and signal detection theory (Zozor et al. 2007) have been used.
Assuming the threshold level is set equal to the signal mean, the mutual information will be exactly one bit per channel use in the absence of noise, will decay to zero as the input SNR goes to zero, and peak at a value approximately \(0.5\log_2(N)\) bits per channel use for an input SNR near zero dB.
Consequently, a curve of mutual information against input SNR is unimodal and therefore stochastic resonance is observed (Stocks 2000).
Only a few exact calculations for the mutual information are known, including (i) the case of a uniformly distributed signal and uniformly distributed noise with smaller variance than the signal (Stocks 2001b); (ii) the case of signal distribution equal to noise distribution (Stocks 2001a); (iii) a case where the distributions can be mapped to uniform noise and a beta (arcsin) input distribution (McDonnell et al. 2007).
However, calculation of the mutual information via numerical integration is straightforward in any other case (McDonnell et al. 2002). Demonstration of suprathreshold stochastic resonance results from plotting the mutual information versus, for example, noise intensity (defined as the ratio of noise standard deviation to that of signal standard deviation). Examples for various values of \(N\) for the case of a Gaussian signal and Gaussian noise, both with the same mean, and threshold levels equal to that mean, is shown in Figure 2.
Limiting behavior for large array sizes
Various different approximations have all verified that the mutual information scales with \(0.5\log_2(N)\) (McDonnell et al. 2007).
Description using signal quantization theory
Why is it that the presence of a range of noise levels allows a larger information rate per sample? The explanation for the array in Figure 1 is that the presence of independent threshold noise has the effect of distributing all \(N\) thresholds to different values, rather than the same value. Instead of modelling the noise as being added to the signal, it is equivalent to model the noise as being threshold noise, so that each threshold value is a random variable. When noise is present, effectively all threshold values will be unique.
This allows the output signal to take values other than zero and \(N\ ,\) and allows the output to become a \(\log_2(N+1)\) bit stochastic quantization of the input signal. For noise with a small variance compared to the signal, most of the output states will occur with a very small probability. However, when the noise variance is such that each output state is occupied with a probability that reflects the shape of the input PDF, a good approximation to the input is obtained.
Suprathreshold stochastic resonance as an optimal quantization scheme for large noise
Given that suprathreshold stochastic resonance in arrays of threshold devices, like that shown in Figure 1, can be described as stochastic signal quantization, an obvious question is the optimality of the quantization. That is, given the noise conditions of independent threshold noise, what are the threshold values that produce an optimal quantization? This question has been answered numerically, and it has been demonstrated that the case of all threshold values equal is optimal for sufficiently large input noise (i.e. about 0 dB) (McDonnell et al. 2006). A plot of the optimal thresholds against decreasing noise intensity exhibits bifurcations. The qualitative nature of this plot is not altered by changing the signal or noise distribution, or performance measure.
Suprathreshold stochastic resonance in arrays of non-thresholding nodes
Other theoretical studies have focused on arrays where the nodes are not "hard thresholds." This includes nodes with both static (Chapeau-Blondeau and Rousseau 2006) and dynamical (Duan et al. 2009) nonlinearities, with a focus on saturation effects.
Misconceptions about suprathreshold stochastic resonance
- Suprathreshold stochastic resonance is not a technique or a method. It is an observed phenomenon. That is, the name suprathreshold stochastic resonance describes the occurrence of an effect. The resonance part of the term is historical; a plot of performance against input noise intensity has a graph that is visually analogous to that for systems with a frequency resonance.
- A "hard threshold" is not necessary for suprathreshold stochastic resonance to occur. As with resonance, the term "suprathreshold" is used for historical reasons and to clarify that the effect is distinct from conventional stochastic resonance.
- Suprathreshold stochastic resonance is not best thought of as something that one might propose to deliberately introduce to a system, unless some other aspect of the system is unavoidably suboptimal. However, there are certainly many circumstances where optimality is impossible to achieve, and inducing suprathreshold stochastic resonance therefore can provide a benefit. Otherwise suprathreshold stochastic resonance can also be thought of as occurring in systems in which noise is unavoidably present. In such circumstances, there is a certain noise intensity which is optimal when compared to higher or lower noise intensities. An example where it might be beneficial to introduce suprathreshold stochastic resonance is a proposal to introduce randomness to cochlear implant stimulation strategies. The idea behind this is simply to reintroduce the noise conditions present in a normally functioning cochlear, under the hypothesis that the auditory system is optimised to cope with such random noise, rather than its absence. See below.
- Dithering: suprathreshold stochastic resonance is not the same as dithering in analog-to-digital converter circuits or image processing. In contrast to suprathreshold stochastic resonance, dithering involves deliberately introducing a random or pseudo random signal that is not independent for each threshold. Furthermore, dither signals generally have small and finite maximum amplitudes. In studies of suprathreshold stochastic resonance using mathematical models, noise signals can have any distribution, e.g. Gaussian or Cauchy, and are not restricted to small powers. Finally suprathreshold stochastic resonance is not restricted to "hard" threshold devices whereas dithering is.
Suprathreshold stochastic resonance in theoretical neuroscience
Populations of neuron models
The original work on suprathreshold stochastic resonance noted that the simple model analyzed has similarities to ensembles of sensory neurons. This, combined with the fact that neurons are capable of displaying stochastic resonance, is one of the motivations for studying such a system. It has been shown that suprathreshold stochastic resonance can still occur when the simplest model first considered is replaced by parallel arrays of FitzHugh-Nagumo model neurons (Stocks and Mannella 2001). Such work has since been extended to the case of leaky integrate-and-fire and Hodgkin-Huxley neuron models, with an emphasis on finding the optimal noise level for a large number of neurons (Hoch et al. 2003b). Suprathreshold stochastic resonance has also been demonstrated in integrate-and-fire neurons in the context of both input correlations (Durrant et al. 2011), and for other noise-based enhancement effects, and saturation (Blanchard et al. 2007).
Arrays of FitzHugh-Nagumo model neurons and integrate-and-fire model neurons have also been used to study the interplay between heterogeneity and noise (Hunsberger et al. 2014).
Detailed noise source modelling: ion channel fluctuations
Most work on suprathreshold stochastic resonance to date has used a high level of abstraction for the origin of the noise sources. A detailed model of how ion channel fluctuations can provide the noise sources essential for the affect to arise has been produced (Ashida and Kubo 2010), and incorporated into arrays of Hodgkin-Huxley neuron models.
Auditory nerve
In recent modelling studies suprathreshold stochastic resonance has been observed in a model of the electrically stimulated auditory nerve (Stocks et al. 2002). The auditory nerve model was developed from physiological studies that used the sciatic nerve of the toad Xenopus as an animal model of an auditory neuron. This suggests that, in principle, suprathreshold stochastic resonance could occur in the auditory system.
Applications
Cochlear implant stimulation strategies
Cochlear implants are prosthetic devices that enable profoundly deaf people to hear. The operation of cochlear implants requires direct electrical stimulation of the auditory nerve. Such stimulation requires sophisticated methods of signal encoding, and although hearing can be restored, patients still have difficulty perceiving speech in a noisy room, or music. A potentially very important proposed application where suprathreshold stochastic resonance would be caused to occur is a cochlear implant signalling strategy (Stocks et al. 2002). The idea is based on the fact that people requiring cochlear implants are missing the natural sensory hair cells that a functioning inner ear uses to encode sound in the auditory nerve. It is known that the stereocilia of these hair cells undergo significant Brownian motion, i.e. randomness, and the synaptic release of neurotransmitter introduces additional randomness. These sources of randomness lead to spontaneous firing in primary afferent auditory nerve fibres that is not normally present in deaf patients who benefit from cochlear implants. The hypothesis is that suprathreshold stochastic resonance induced by re-introducing this natural randomness to the encoding of sound could improve speech comprehension in patients fitted with cochlear implants.
Analog-to-digital converter circuits
At least two proposals have suggested how suprathreshold stochastic resonance might be usefully induced in analog-to-digital-converter circuits. First is a proposal to design the comparator component of a sigma-delta modulator (which is a specific type of analogue-to-digital converter that makes use of feedback) in a way that utilizes suprathreshold stochastic resonance (Olieau 2003). Second is a heuristic method for a flash-like analogue-to-digital converter, where the input signal is transformed so that it has a large variance and is bimodal, thus ensuring smaller output noise (Nguyen 2007).
Other applications
- Digital accelerometers (Olson and Carr 2007)
Related Concepts
Stochastic Pooling Networks
A stochastic pooling network is a network with the property that multiple parallel stochastic and compressed measurements of a common input signal are combined into a single measurement by a physical channel, in such a way that pooling of measurements causes no (or negligible) further loss of information about the network’s input signal, when compared with the best performance that could be achieved if all sensor measurements were available (McDonnell et al. 2009).
The simple binary quantizing network described above is a special case of a stochastic pooling network. Consequently, suprathreshold stochastic resonance is just one of several emergent phenomenon (as well as subthreshold signal stochastic resonance) that can occur in stochastic pooling networks.
The stochastic pooling network is useful for demonstrating generalizations of suprathreshold stochastic resonance, such as the fact that it can occur in networks of multi-bit quantizers.
A nanoscale electronics architecture inspired by suprathreshold stochastic resonance has been proposed for noise reduction and to improve reliability at the nanoscale. This architecture exploits stochastic pooling network-like arrays (Martorell and Rubio 2008).
Open research questions
Neuroscience
- Can it be verified that suprathreshold stochastic resonance occurs during information pooling in biological neuronal networks?
Engineering applications
- Can suprathreshold stochastic resonance be exploited in applications like sensor networks, analog-to-digital converters, and nanoscale electronics?
Theoretical questions
- Does suprathreshold stochastic resonance occur when the pooling of the measurements of each node is not a summation operation?
- Can suprathreshold stochastic resonance be observed when the network is optimized subject to constraints such as a maximum power for the input or output signal?
- Does suprathreshold stochastic resonance occur in arbitrary stochastic pooling networks, where for example, the network has a more complex topology such as the inclusion of feedback loops, adaptation, and cooperation between nodes?
- Can mathematical approaches predict the clustering and bifurcations (McDonnell et al. 2006) that are seen when the nodes of Fig 1 are optimized?
References
Internal references
- John Guckenheimer (2007) Bifurcation. Scholarpedia, 2(6):1517.
- James Meiss (2007) Dynamical systems. Scholarpedia, 2(2):1629.
- Eugene M. Izhikevich and Richard FitzHugh (2006) FitzHugh-Nagumo model. Scholarpedia, 1(9):1349.
- Lon A. Wilkens and Frank Moss (2008) Mechanoreceptors and stochastic resonance. Scholarpedia, 3(2):1635.
- Peter E. Latham and Yasser Roudi (2009) Mutual information. Scholarpedia, 4(1):1658.
- Rodolfo Llinas (2008) Neuron. Scholarpedia, 3(8):1490.
- Don H. Johnson (2006) Signal-to-noise ratio. Scholarpedia, 1(12):2088.
- Catherine Rouvas-Nicolis and Gregoire Nicolis (2007) Stochastic resonance. Scholarpedia, 2(11):1474.
Selected original theoretical papers
- Stocks N.G. (2000) Suprathreshold stochastic resonance in multilevel threshold systems. Physical Review Letters 84:2310-2313.
- Stocks N.G. (2001a) Information transmission in parallel threshold arrays: Suprathreshold stochastic resonance. Physical Review E 63:041114.
- Stocks N.G. (2001b) Suprathreshold Stochastic Resonance: An exact result for uniformly distributed signal and noise. Physics Letters A 279:308-312.
- McDonnell M.D., Abbott D., Pearce C.E.M. (2002) A Characterization of suprathreshold stochastic resonance in an array of comparators by correlation coefficient. Fluctuation and Noise Letters 2:L205-L220.
- Rousseau D., Duan F., Chapeau-Blondeau F. (2003) Suprathreshold stochastic resonance and noise-enhanced Fisher information in arrays of threshold devices. Physical Review E 68:031107.
- Rousseau D., Chapeau-Blondeau F. (2004) Suprathreshold stochastic resonance and signal-to-noise ratio improvement in arrays of comparators. Physics Letters A 321:280-290.
- Chapeau-Blondeau F., Rousseau D. (2006) Noise-aided SNR amplification by parallel arrays of sensors with saturation. Physics Letters A 351:231-237.
- Duan F., Chapeau-Blondeau F., Abbott D. (2009) Input-output gain of collective response in an uncoupled parallel array of saturating dynamical subsystems. Physica A 388:1345-1351.
- Patel A., Kosko B. (2009) Error-probability noise benefits in threshold neural signal detection. Neural Networks 22:697-706.
Performance limits of suprathreshold stochastic resonance
- McDonnell M.D., Stocks N.G., Pearce C.E.M., Abbott D. (2006) Optimal information transmission in nonlinear arrays through suprathreshold stochastic resonance. Physics Letters A 352:183-189.
- McDonnell M.D., Stocks N.G., Abbott D. (2007) Optimal stimulus and noise distributions for information transmission via suprathreshold stochastic resonance. Physical Review E 75:061105.
Suprathreshold stochastic resonance in computational neuroscience
- Stocks N.G., Mannella R. (2001) Generic noise enhanced coding in neuronal arrays. Physical Review E 64:030902(R).
- Masuda N., Aihara K. (2001) Bridging rate coding and temporal spike coding by effect of noise. Physical Review Letters 88:248101.
- Hoch T., Wenning G., Obermayer K. (2003a) Adaptation using local information for maximizing the global cost. Neurocomputing 52-54:541-546.
- Hoch T., Wenning G., Obermayer K. (2003b) Optimal noise-aided signal transmission through populations of neurons. Physical Review E 68:011911.
- Blanchard S., Rousseau D., Chapeau-Blondeau F. (2007) Noise enhancement of signal transduction by parallel arrays of nonlinear neurons with threshold and saturation. Neurocomputing 71:333-341.
- Sasaki H., Sakane S., Ishida T., Todorokihara M., Kitamura T., Aoki R. (2008) Suprathreshold stochastic resonance in visual signal detection. Behavioural Brain Research 193:152-155.
- Ashida g., Kubo M. (2010) Suprathreshold stochastic resonance induced by ion channel fluctuation. Physica D 239:327-334.
- Durrant S., Kang Y., Stocks N., Feng J. (2011) Suprathreshold stochastic resonance in neural processing tuned by correlation. Physical Review E 84:011923.
- Hunsberger, E., Scott, M., Eliasmith, C. (2014) The Competing Benefits of Noise and Heterogeneity in Neural Coding. Neural Computation 26:1600-1623.
Applications in cochlear implants
- Stocks N.G., Allingham D. Morse R.P. (2002) The application of suprathreshold stochastic resonance to cochlear implant coding. Fluctuation and Noise Letters, 2:L169-L181.
Applications in analog-to-digital-converters
- Olieau O. (2003) Stochastic resonance in sigma-delta modulators. Electronics Letters 39:173-174.
- Nguyen T. (2007) Robust data-optimized stochastic analog-to-digital converters. IEEE Transactions on Signal Processing 55:2735-2740.
Other applications
- Olson R. H. and Carr D. W. (2004) A digital accelerometer array utilizing suprathreshold stochastic resonance for detection of sub-Brownian noise floor accelerations. Sandia National Laboratories Technical report SAND2004-6441.
Related work
- Zozor S., Amblard P.O., Duchene C. (2007) On Pooling Networks and fluctuation in suboptimal detection framework. Fluctuation and Noise Letters 7:L39-L60.
- Martorell F., Rubio A. (2008) Cell architecture for nanoelectronic design. Microelectronics Journal 39:1041-1050.
- McDonnell M. D., Stocks N.G., Amblard P.O. (2009) Stochastic pooling networks. Journal of Statistical Mechanics: Theory and Experiment P01012.
- McDonnell M. D. (2009) Information capacity of stochastic pooling networks is achieved by discrete inputs. Physical Review E 79: 041107.
- Åkerberg O. A., Chacron M. J. (2009) Noise shaping in neural populations. Physical Review E 79: 011914.
Recommended reading
- McDonnell M.D., Stocks N.G. Pearce C.E.M., Abbott D. (2008) Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization, Cambridge University Press.
- McDonnell M.D., Abbott D. (2009) What is stochastic resonance? Definitions, misconceptions, debates, and its relevance to biology. PLoS Computational Biology 5:e1000348.
See also
Stochastic resonance, Mechanoreceptors and stochastic resonance, Neuronal noise, Stochastic dynamical systems
