User:Eugene M. Izhikevich/Proposed/Cybernetics
Cybernetics, as a scientific discipline has been named by Norbert Wiener (1894-1964). It was the title of his book with the subtitle Control and Communication in the Animal and the Machine (Wiener 1948). Cybernetics was a pluralistic theory and an interdisciplinary movement of a number of leading intellectuals. The term cybernetics goes back to Plato, when he explained the principles of political self-governance.
One of the supporting pillar of the emerging cybernetics is grew up from the paper entitled Behavior, Purpose and Teleology by Arturo Rosenblueth, Norbert Wiener, and Julian Bigelow (1943). It gave the conceptual framework of goal-directed behavior both in technological and biological context. Looking retrospective the paper is strange in several respects. It was published in Philosophy of Science, did not contain a single formula, figure or reference. In any case, the paper emphasized that purposeful behavior can exist both in engineered and biological systems without assuming the Aristotelian final cause. Purposeful behavior can be explained by present causes, but the causation acts in a circular manner.
Wiener himself emphasized the role of feedback mechanisms in the goal-oriented systems. While the physiologists already knew that the involuntary (autonomous) nervous systems control Bernard's internal milieu, he extended the concept suggesting that the voluntary nervous system may control the environment by some feedback mechanisms and searched for a theory of goal-oriented behavior. This theory promised a new framework to understand the behavior of animals, humans, and computers just under design and construction that time. Feedback control is an invisible thread in the history of technology, as studies on mechanical clocks, steam engines, aerodynamic and electronic devices show (Bernstein 2002). A device designed to measure and regulate the speed of a machine, called 'governor' was successfully built by James Watt. Interestingly, James Clerk Maxwell had a paper "On Governors" (Maxwell 1868), in which he anticipated some aspects of modern control engineering (Mayr 1971).
The other supporting pillar of cybernetics is the brain-computer analogy suggested by the spirit of the McCulloch-Pitts Neuron (MCP neuron). An MCP (McCulloch and Pitts, 1943) neuron is a formal model, and it can be identified as a binary threshold unit. A neuron iniziates an impulse if the weighted sum of their inputs exceeds a threshold, otherwise it remains in silence.
The MCP model framework wanted to capture the logical basis of neural computation, and intentionally contains neurobiological simplifications. The state is binary, the time is discrete, the threshold and the wiring are fixed. Chemical and electrical interactions are neglected, glia cells are also not taken into consideration. McCulloch and Pitts showed that a large enough number of synchronously updated neurons connected by appropriate weights could perform many possible computations.
John von Neumann (1903-1958) was certainly motivated by the MCP model when worked on the logical design of electronic computers. The all-or-none character of the neurons was isomorphic with that of the elementary computing units. The logical design of computers, and the techniques of switching theory used by von Neumann grew out from the MCP model. Analogy was assumed between computers and the brain, both at the elementary hardware level, and at the level of mathematics, as well. There is a similarity between the mathematical model of the brain (i.e. the MCP network model), and of the computer (i.e. the Turing machine). The difference lies in the fact that while neural networks contain finite number of neurons, the Turing machine contains a virtually infinite number of elements (i.e. the cells of the indefinitely extendible tape). The brain-computer analogy has obvious limits - it is not clear whether the brain executes algorithms, like the Turing machine. In any case, the computers were not build to be the model of the human brain.
The scope of cybernetics was not restricted to physical, biological and technological systems, the humanistic aspects were emphasized from the very beginning, even by Wiener itself.
Contents |
Two Founding Fathers
While there are several pioneers of cybernetics, two of them played special roles. McCulloch was the chairman of the fundamental conference series on cybernetics, while the term cybernetics appeared in the modern sense in the title of Wiener's book.
Warren McCulloch: a real pioneer of interdisciplinarity
Warren McCulloch (1898-1969) was one of the Founding Fathers of the movement and scientific discipline of cybernetics, who had a particular personality, a very original individual, a polymath. He learned philosophy, became an MD, and got education in mathematical physics and physiological psychology, as well.
His entire scientific activity was a big experiment to give a logic-based physiological theory of knowledge. McCulloch assumed that (1) the brain performs logical thinking (2) which is described by logic, therefore the implication is that the operation of the brain could and should be described by logic. (His papers are collected with the title Embodiments of Mind, McCulloch 1965.)
Norbert Wiener: another hero
Wiener was a well-accepted mathematician, who worked on functional analysis and on stochastic processes before Kolmogorov gave its systematic formulation. Wiener studied a model of the Brownian motion, a classical model in the theory of stochastic processes, which is called now as the Wiener process. The Wiener-Khintchine relationship helps to analyze stationary stochastic processes. It connects the temporal with the frequency domain, i.e. shows how to transform the autocorrelation function of a stationary time series to power spectrum by means of a Fourier transform. As a founder of cybernetics, he supported automatics, robotics and their role in improving human conditions.
Conway and Siegelman in their book (Dark Hero of the Information Age. In Search of Norbert Wiener, the Father of Cybernetics, 2005) analyzed how Wiener's personal history contributed to a break among the founding fathers of cybernetics, followed by the dissolution of cybernetic movement into other disciplines.
The Macy Conferences and their impact
The Macy conference series (ten meetings between 1946-1953) was organized to understand the feedback mechanisms in biological, technological and social systems, by the aid of concepts like circular causality and self-regulations. McCulloch served as the chairman of the conferences, which had a strong interdisciplinary character. While mathematicians/physicists and psychologists dominated the conferences, Wiener and von Neumann in particular made claims that their theories and models would be of utility in economics and political science, too. Margaret Mead (1901-1978) and Gregory Bateson (1904-1980) contributed very much to develop the humanistic aspects of cybernetical thinking (Bateson 1972).
The main topics of the conferences were (Dupuy 2000):
- Applicability of a Logic Machine Model to both Brain and Computer
- Analogies between Organisms and Machines
- Information Theory
- Neuroses and Pathology of Mental Life
- Human and Social Communication
Another pioneer, William Ross Ashby (1903-1972) attended only one, i.e. the ninth conference, and presented two papers; one on his 'homeostat' and the other on the prospects of chess playing automatons. He was probably first (Ashby 1952, Ashby 1956) to use the term self-organization and contributed very much to cybernetics and system science. One of his main conceptual achievements was to make a difference between an object, and a system defined on an object.
Basic concepts of cybernetics
Feedback and circular causality
Feedback is a process whereby some proportion of the output signal of a system is passed (fed back) to the input. So, the system itself contains a loop. Feedback mechanisms fundamentally influence the dynamic behavior of a system. Roughly speaking negative feedback reduces the deviation or error from a goal state, therefore has stabilizing effects. Positive feedback which increases the deviation from an initial state, has destabilizing effects. Natural, technological and social systems are full of feedback mechanisms.
The general principles of feedback control were understood by engineers, and autonomous control systems were used to replace human operators. This replacement can be done only up to a point, and consequently one is brought directly to face the question of the role of the human observer in technological systems.
Rosenblueth worked with Walter Cannon (who popularized the concept of homeostasis), and he considered living processes as self-regulated ones. Wiener and Bigelow were involved during the second world war in developing antiaircraft guns by using negative feedback.
Circular causality in essence is a sequence of causes and effects whereby the explanation of a pattern leads back to the first cause and either confirms or changes that first cause. Example: A causes B causes C that causes or modifies A. The concept itself had a bad reputation in legitimate scientific circles, since it was somehow related to use vicious circles in reasoning. It was reintroduced to science by cybernetics emphasizing feedback. In a feedback system there is no clear discrimination between causes and effects, since the output influences the input.
The mechanization of the mind
Cybernetics was built on the assumptions (based on Dupuy 2000, pp 3-4.), that
- Thinking is a form of computation. The computation involved is not the mental operation of a human being who manipulates symbols in applying rules, such as those of addition or multiplication; instead it is what a particular class of machines do — machines technically referred to as 'automata'. By virtue of this, thinking can be modeled within the domain of the mechanical.
- Physical laws can explain why and how nature — in certain of its manifestations, not restricted exclusively to the human world — appears to us to contain meaning, finality, directionality, and intentionality.
Second-order cybernetics
The keywords of the early cybernetics (identified, say, with the first five Macy meetings) were communication and control. The second order cybernetics (initiated by Heinz von Foerster (1911-2002) and Roger Ashby (1903-1972)) considered that the observer and the observed are the parts of the same system, and the result of the observation depends on the nature of their interaction. The observer is also a cybernetic system, so a cybernetics of cybernetics or second-order cybernetic was needed.
Heinz von Foerster served as the secretary of the last five Macy conferences. Between 1958-1975 he was the director of the Biological Computer Laboratory at the University of Illinois at Urbana-Champaign. Von Foerster constructed and defended the concept of second-order cybernetics. As opposed to the new computer science and control engineering, which became independent fields, the second order cybernetics emphasized the concepts of autonomy, self-organization, cognition, and the role of the observer in modeling a system. Cybernetic systems, such as organisms and social systems are studied by an other cybernetic system, namely by the observer. Von Foerster was a radical constructivist. According to this view, knowledge about the external world is obtained by preparing models on it. The observer constructs a model of the observed system, therefore their interactions should be understood by second-order cybernetics.
The dissolution of cybernetic movement
A set of cyberneticians, who felt the irreducible complexity of the system-observer interactions, abandoned to build and test formal models, and used a verbal language and metaphors. They were the subjects of well-founded critics for not studying specific phenomena (Heylighen and Joslyn 2001). With the termination of the Macy conferences and with the organization of the Dartmouth Summer Research Conference on Artificial Intelligence in 1956, the cybernetics movement lost its strong effects. While many concepts and principles of cybernetics survived and became indispensable elements of other fields, the power of the cybernetic movement has been strongly reduced.
At least four disciplines have crystallized from cybernetics (Arbib 1989, pp.6):
- Biological control theory
- Neural modeling
- Artificial Intelligence
- Cognitive psychology
Preliminary crystal seeds of these disciplines, of course, have already existed, e.g. Turing's paper on computing machinery and intelligence (Turing 1950) initiated artificial intelligence, or Rashevsky's continuous neural models (Rashevsky 1933).
The heritage and revival of cybernetics
The members of the next generation following cyberneticians, mostly just their students, shifted the emphasis from the structural approach to the functional one. The pioneers of the artificial intelligence (AI) research substituted McCulloch and Pitts' binary strings of zeros and ones by more general symbols. Procedures on physical symbol systems were viewed as the necessary and sufficient means for general (ie. natural and artificial) intelligent action. While the symbolistic paradigm became predominant, the perspectives of the cyberneticians and AI researchers did not separate immediately, but the debate became very sharply related to the Perceptron battle. The Perceptron is a mathematical construction of an adaptive neural network being able to learn and classify inputs. It was defined by Rosenblatt (Rosenblatt 1962) by extending the MCP rule by modifying synaptic weights. Minsky and Papert proved in 1969 (Minsky and Papert 1969) that a single layer Perceptron cannot solve the exclusive OR problem. Perceptrons were assumed to be able to classify only linearly separable patterns. The implication of the critique was the serious restriction on funding neural network research. However, the critique is not valid for multilayer neural networks.
Many concepts of cybernetics (very often tacitly) revived. Brain theory, cognitive science systems biology, and implemented ideas emerged by cybernetics.
The brain is a physical structure which is controlled and also controls, learns and teaches, processes and creates information, recognizes and generates patterns, organizes its environment and is organized by it. Furthermore, closed causal loops and self-referential systems implement the iterative nature of learning and interpretation.
Many concepts of cybernetics returned in a somewhat different context to cognitive science without being credited. McCulloch's embodied mind notion, von Foerster's critiques on the controversial computational paradigm of AI and cognitive science, and the concept of circular causality all returned related to the embodied cognition (see e.g. Ziemke 2005).
Systems biology is an emergent movement to combine system-level description with microscopic details. It might be interpreted as the renaissance of cybernetics and of system theory. The systems biological approach emphasizes the integration of components (mostly proteins and genes) by dynamical models. Internal control mechanisms exist which maintain the function of the system.
The analysis of dynamics and stability of large networks of elements interconnected by positive and negative feedback connections also grew up from ideas inherent in cybernetics. The strategy is applied in many disciplines, from chemical reaction networks via biochemical, cellular, ecological and epidemic networks, to many socioeconomic situations, such as international relations, economic trades etc.
Cybernetics has a great potential for future research (Umpleby 2007) by offering a common way of thinking and language to develop theories for machines, organisms, groups and societies.
References
Wiener N (1948) Cybernetics or Control and Communication in the Animal and the Machine, Paris, Hermann et Cie – MIT Press, Cambridge, MA
Rosenblueth A, Wiener N and Bigelow J (1943) Behavior, Purpose and Teleology; Philosophy of Science, 10 (18-24)
Bernstein DS: Feedback control (2002) an invisible thread in the history of technology IEE Control Systems Magazine, 2 (53-68)
Maxwell JC (1868) On Governors, Proceedings of the Royal Society, no. 100.
Mayr O (1971) James Clerk Maxwell and the Origins of Cybernetics, Isis, 62 (425-444)
McCulloch W and Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 7 (115-133)
McCulloch W (1965) Embodiments of Mind; MIT Press
Conway F and Siegelman J (2005) Dark Hero of the Information Age: In Search of Norbert Wiener, the Father of Cybernetics, Basic Books
Bateson G (1972) Steps to an Ecology of Mind. Chandler Publishing Company
Dupuy J-P (2000) The Mechanization of the Mind: On the origins of cognitive science. Princeton University Press
Ashby R (1952) Design for a Brain, Chapman and Hall
Ashby R (1956) Introduction to Cybernetics, Chapman & Hall
Heylighen F and Joslyn C (2001) Cybernetics and Second Order Cybernetics ; Encyclopedia of Physical Science & Technology
Arbib MA (1987) Brains, Machines and Mathematics, 2nd edition. Springer Verlag.
Turing, A.M. (1950). Computing machinery and intelligence. Mind, 59, 433-460.
Rashevsky, N. (1933) Outline of a physico-mathematical theory of excitation and inhibition, Protoplasma 20 pp. 42.
Rosenblatt F (1962) Principles of neurodynamics: Perceptrons and the theory of brain mechanisms; Spartan Books
Minsky ML and Papert S (1969) Perceptrons: An Introduction to Computational Geometry, MIT Press
Ziemke T (2005) Cybernetics and Embodied Cognition: On the Construction of Realities in Organisms and Robots. Kybernetes, 34 (118-128)
Umpleby SA (2007) Cybernetics. In: International Encylopeida of Organization Studies (Clegg SR and Bailey JR, eds.), Sage Publications, Inc.
