Harnessing stochasticity: How do organisms make choices? Open Access

How do organisms make choices? One very simple answer to this question would be that they do not. Following Descartes (1665),¹ the assumption would be that organisms are determinate machines. Despite their fiendish complexity, if we knew enough about the mechanisms involved, we would be able to predict their behavior to any arbitrary degree of accuracy.

Descartes actually excluded humans from this viewpoint, but that requires an assumption either that a non-material entity somehow intervenes in the case of humans, or that some non-determinate (stochastic) material process operates. Descartes chose the first option, which creates the difficulty that we have no way of representing how a material body could be so influenced. For example, would such an influence necessarily appear to be stochastic to scientific investigation, precisely because it would not be caused by any measurable physical events, and would have to appear to be stochastic in order to be indeterminate? Without making metaphysical assumptions beyond the possibility of scientific investigation, what we would find in this case simply collapses to the second possibility, at least insofaras we can investigate it objectively.² 

In this article, we will conclude that stochastic material processes are involved. Moreover, there is no reason to suppose that such processes do not operate in organisms other than humans. Since humans evolved from other organisms, we should expect both of these conclusions.

Moreover, at the micro-level, we now know that the material universe is fundamentally stochastic, whether it be by virtue of random kinetic energy producing the form of stochasticity observed in the Brownian motion of molecules or by virtue of quantum mechanical behavior at the level of particles. Organisms must be affected by such stochasticity. Neither animals nor humans can be fully determinate. But that leaves open the question how stochasticity is involved or used in living processes.

In recent articles, we have addressed the following issues which can be seen to be introductory to the focus of the present article.

1. Can stochastic and/or chaotic processes be used in organisms, rather than organisms being arbitrarily subject to them, i.e., can such processes be harnessed so that they become part of the necessary functional repertoire of organisms? This issue was addressed in Noble^{3 (p. 1)} and the answer is yes, organisms necessarily harness stochasticity.

2. Can we know whether organisms have agency, and does their behavior generate a form of directionality both in individual organisms, and at the level of populations so that the behavior can in turn influence the direction of evolution? This issue was addressed in Noble and Noble⁴ and the answer again is yes, organisms do have agency. Harnessing stochasticity is an essential part of the means by which they do so. As we will show later in this article, a fully determinate process (meaning completely predictable) would not satisfy the conditions for agency.

Those articles leave open the question how the harnessing of stochasticity and the possession of agency may be represented in empirical (i.e., experimentally testable) terms. As a test of what we propose, we will also ask the question whether any such representation can show why we cannot predict what we call free choice, yet can often account for it in rational terms in retrospect.

An agent acts, it does not just react in the way, for example, in which a billiard ball is caused by another ball to move. There are many levels of agency (Ref. 5, p. 32–40). Organisms are agents to the extent that they can interact socially with other organisms to choose particular forms of behavior in response to environmental challenges. Agency requires causal independence.⁶ It also requires intentionality, i.e., the sense of purpose, in order to be causally effective as a driving force.⁷ 

Inanimate objects can contain information. But it requires interpretation by an organism to become knowledge of what the information means. For example, rocks contain information, and that only becomes knowledge when organisms interpret it, e.g., to work out dates of events in the history of the earth. By this definition, DNA is also inanimate. It contains sequence information, but it does not contain knowledge. Until they are interpreted, DNA sequences are like uninterpreted hieroglyphics.

DNA information is interpreted by organisms. Outside a living cell, DNA is inert. A complete cell therefore is a minimal interpreter of DNA.

Knowledge about the world arises through organisms being creative in finding new solutions to environmental challenges. We can distinguish two types of knowledge:

This can be verified by those other than the organism that has the knowledge. In this sense, plants and bacteria have knowledge. Plants possess functional processes enabling them to use sunlight to create oxygen, and nutrients like sugars. We do not yet have that knowledge but wish we did! Note that this definition is not identical with Popper’s use of “objective knowledge.”⁸ 

Organisms that “know that they know” have this kind of knowledge. They can communicate this kind of knowledge to others through behavior and language.

Note. Many philosophers do not attribute knowledge to organisms unless they are conscious, e.g., Anthony Kenny,⁵ who refers to “capacity or ability” rather than “objective knowledge.” We acknowledge the difference of usage of “knowledge” but do not think that the conclusions of our article depend very much on which view one takes. Here, we simply note that resolving this question would depend on one’s view of animal consciousness; see, e.g., Ref. 9. In this article, we are not primarily concerned with this kind of knowledge, and we do not address questions of self-awareness and consciousness.

In this article, we refer to accounting for choice behavior in retrospect as being rational. What is meant is that it is possible to answer the question why an organism did what it did using the common sense meaning of rational, e.g., in terms of the organism’s presumed goals. This does not mean that the organism’s choice would be predicted by any particular version of Rational Choice Theory (https://en.wikipedia.org/wiki/Rational_choice_theory). Nor does it mean that the “rationality” does not contain an element of delusion. We will return to this question in the Discussion.

Interpreted as the inability to predict, stochasticity is a level-dependent property. Thus, molecular level stochasticity is compatible with higher level predictability, as is obvious from the predictability of thermodynamics. Stochasticity is therefore a relativistic concept. Whether underlying stochasticity can influence the overall behavior of a system must depend on whether the higher level is organized to enable it to do so. Organisms are high-level systems. In this article, we show that molecular stochasticity does not only cancel itself out at higher levels, as in the case of thermodynamics, it also becomes used in goal-directed feedback control processes. Higher-level organization can make that possible.

As many readers, particularly of this journal, will be aware, stochasticity and chaos are not identical. Chaotic sequences can be produced by determinate algorithms as first shown by Lorenz.^{10, 11} The difference is important because the variations in determinate chaos are constrained by an attractor, whereas genuine stochasticity is not. The difference can be made clear in phase plots. However, we doubt whether the difference between determinate chaos and stochasticity is relevant to the process we ascribe to choice behavior. If the attractor constraining a chaotic sequence is not itself an integral part of the organism’s control networks, the variations will appear random to the choice process.

An important basis for our paper is that organisms are open systems in which causation operates between multiple levels. That they are open systems is obvious: they exchange matter and energy with their environment and engage in social interactions with other organisms. Multi-level causation is not, however, universally accepted in biology. We follow the argument that causation must be multi-level. The demonstration that this is the case is mathematical. Even if we try to imagine that only molecular level mechanisms are causative, we are faced with a fundamental difficulty when we try to solve the differential equations for those mechanisms. There is no solution unless we introduce boundary conditions that represent the causative action of higher levels and scales. This is the mathematical basis of the principle of biological relativity.^{12, 13} The principle states that there is no privileged level of causation. But it is important to note that the upward and downward forms of causation are not necessarily of the same form. Causation by setting the boundary conditions for lower level processes is more like a constraint on the forms of organisation that the lower level elements may take.¹⁴ These causal interactions can occur between any of the levels of organization and are the reason why downward is causally effective. Indeed, in purposive behavior, it is primary since it will only be at the higher levels that the purposive organization may be evident.⁴ This is the general causal basis for the choice process that we will now present.

For an empirically testable theory of choice to be possible, we need to know at which stages in the process experimental interventions could test its validity. At first sight, that may seem impossible. How can we specify a process that is necessarily unpredictable but which can be given an at least apparently rational justification once it has happened? Our previous work provides a clue to that problem. In Ref. 4, we analyzed agency by comparing it to the purposive behavior of the immune system. The immune system solves what we can best characterize as a template puzzle: given a new invader with an unknown chemical profile (shape of template), what is the best way to find the key (an anti-template, i.e., the antibody) to lock onto and neutralize the invader? The answer in the case of the immune system is one of the most remarkable forms of the harnessing of stochasticity. In response to the new environmental challenge, a feedback loop activates a massive increase in mutation rate in a highly targeted region of the immunoglobulin DNA sequence.¹⁵ 

The process of choice in organisms can be viewed as analogous to the immune system. The process can be represented as follows:

1. Influences from the environment (boundary conditions) and the organism’s history (initial conditions) lead to defining the problem facing the organism. This will be the state of the organism in which the environmental challenge has occurred but not yet a solution in the organism’s reactions. We conjecture that such a problem can be viewed as a puzzle analogous to the form of a template for which a match is needed. The configuration of these conditions might be a routine one, in which case what we normally characterize as a reflex response may be adequate. But it is precisely such responses that we would not characterize as involving a choice. We say that a choice occurs when there is no automatic reflex response possible. The challenge facing the organism then is what could fit the puzzle template?

2. Instead of an automatic response therefore, the organism must search amongst existing stored possible fits to the problem template. By analogy with the immune system, this is equivalent to finding that the DNA sequence for the correct immunoglobulin shape already exists. It is precisely when no such solution exists that hypermutation is triggered. We hypothesise that a comparable process occurs generally in choice situations in organisms.

3. In which case the organism can spin (i.e., activate) stochastic processes within itself to generate further possible new solutions. This is where novelty arises. These processes can be of any biological kind. For cognitive problems in organisms with highly developed nervous systems, these will be primarily neural. Note also two important characteristics of this stage of the process. First, the organism triggers the resort to stochasticity but no longer controls it, just as the immune system does not directly control which mutations occur. Second, the options at this stage are effectively infinite. In the case of the immune system, the number of possible sequences for the variable part of the immunoglobulin must be larger than the total number of particles in the universe. That is also true for the number of interactions between the 20 000 or so genes in a human.¹⁶ Stochasticity and/or chaos in the nervous system must make even more options available.¹⁷ 

   Neural processes are extensively stochastic—at all functional levels, from the opening and closing of ion channels via action potential generation, spontaneously or through synaptic transmission in neuronal networks, up to cognitive functions including decision making (8 chapter 22).^18–22 As pointed out in Braun,²³ the reason may be found in the functional organization of living systems composed of a manifold of nonlinear feedback loops that often are adjusted to operate in the neighbourhood of bifurcations where it can essentially depend on random effects of what will happen next, e.g., whether an ion channel is opened or remain closed or whether an action potential is generated or not—what even may decide the choice between leaving the bar and going home or having another drink.

4. The organism returns to direct control at the next stage, which is to compare what is thrown up by the stochastic process with the problem template to determine what fits. “Template” and “fit” here are used metaphorically, in much the same sense in which a logical answer can be said to “fit” (i.e., answer to) the problem posed by a question. This is the essential choice process, needing a comparator. The comparator therefore forms part of what we call the interpreter (see definitions). This is the stage at which we can say that the organism knows that it has found a possible solution.

5. The final stage is to implement the discovered action to solve the problem.

   This is an idealized process, but it clearly helps one to explain an apparent paradox regarding the predictability or otherwise of what we call a free choice. Step 4 ensures that, in retrospect (and only in retrospect), the choice may be what in the case of humans we call rational. There may be a complete logic to why it was made. The logic lies in the fit between the problem template and the solution template. But step 3 ensures that the choice was unpredictable since we cannot predict what stochasticity will throw up. So, free choice is both rational and novel. (See also Parallels in the work of Karl Popper below.)

   This hypothetical process is open to empirical tests at all stages since it makes significant assumptions about what is actually happening within the organism. The kind of knowledge the organism has is what Popper characterised as objective knowledge (as distinct from subjective knowledge) and is fully open to observational test.

For example, the existence of stage 3 naturally explains why problems leading to the necessity for making a choice may lead to what we can call the puzzled state. Before stage 3, there is no solution in sight. Only after stage 3 might there be a solution that can lead to rational action. There will therefore be a period during which the organism does not know the solution. In the case of humans, we can communicate such states in language (“I havn’t a clue”). Other organisms can communicate by behavior: frustration, depression, displacement activity, etc.

We are far from being the first to favor active agency as an explanation of the behavior of organisms and to favor the role of choice in the direction of evolutionary change. The arguments about the active role of organisms have their origins in a long tradition in which deterministic and non-deterministic views of life have been pitted against each other. As noted in our Introduction, the two threads were present in the same philosopher in the case of Descartes who in the seventeenth century struggled to reconcile his determinist interpretation of animal behavior with his conviction that this could not be true of humans. How else could he have written his great works? It would have taken a monkey billions of years to manage by chance to type just a single sentence of Descartes’ work (Ref. 4, p. 1). (The relevant combinatorial mathematics is given in Ref. 16; see https://en.wikipedia.org/wiki/Weasel_program.)

The existence of creativity shouts out loud and clear that the universe cannot be simply deterministic, and since the early 20th century revolutions in physics, we have the proof that it is not. Yet, this revolution had surprisingly little effect on biology, which continued with deterministic interpretations of life and its evolution throughout the century. It was thought that indeterminacy at microphysical scales could hardly be relevant to processes at physiological scales. The proof that it is relevant came with the discovery of the hypermutation mechanism in the immune system. As we have shown in previous articles, the harnessing of stochasticity at a molecular level is precisely what enables organisms to be creative. The immune system serves as a model, which can be generalized (Ref. 4, p. 4–5). Given the nature of the universe, uncertainty is inevitable. Choice necessarily involves dealing with uncertainty. Low-level stochasticity is the clay from which high-level novelty can develop.

We wish to credit two more recent predecessors for major influences on our ideas: Patrick Bateson and Karl Popper. Patrick Bateson’s work on the active role of behavior in evolution^24–28 was pursued throughout his career and has been summarized in a book published just before his death in 2017.²⁷ He was a careful historian as well as a great biological scientist. He documented the development of the ideas of active agency through from Darwin, through Spalding and Baldwin to his phrase the “adaptability driver” to describe the active nature of organism agency.²⁶ His phrase captures the directionality of agency in organisms.

Amongst fore-runners of the ideas explored in this article is the outstanding work of Karl Popper. In 1986, Popper gave a lecture to The Royal Society in London in which he laid out his “New Interpretation of Darwinism.”²⁹ In that lecture, he distinguished between “passive Darwinism” and what he called “active Darwinism.” His “passive Darwinism” is more or less identical with classical neo-Darwinism: the theory that random genetic variation and natural selection are entirely sufficient (allmacht in Weismann’s words)³⁰ to explain evolution. Popper wrote: “I shall attempt to turn the tables completely on passive Darwinism … I shall claim that the only creative element in evolution is the activity of living organisms.”^{29,(p. 119)} “Active Darwinism” is therefore equivalent to the theory that organisms have agency and make choices, which is the main theme of our paper. Those choices include choosing niches (niche selection theory) and which other organisms they interact with (including sexual selection), and more recently, the discovery of aversion to cheating behavior in populations of dogs³¹ and monkeys.³² 

Popper regarded the “metaphor of ‘natural selection’” as “a theory of error elimination”^{29 (p. 120)} rather than being creative of novelty itself. He saw it as a filter eliminating errors. To understand this point, we should remember that Darwin contrasted natural selection with artificial selection, which is clearly choices made by organisms (the selective breeders). When Darwin realised that sexual selection is more like artificial selection, he therefore faced a problem. Sexual selection is clearly an activity of organisms determining their evolution. The problem is that this blurs the distinction he was drawing. Sexual selection is therefore a form of active Darwinism to use Popper’s terminology. Specifically, he wrote “sexual selection is a refutation of natural selection.”^{29 (p. 128)}

Popper saw that complete determinism was incompatible with viewing organisms as agents making choices. He would therefore have seen the importance of the role of stochasticity in our paper. In The Open Universe, Popper demonstrated that indeterminism is a necessary but not sufficient condition for emergence and openness.^{29 (p. 70)}

In the same exposition of Popper’s ideas leading up to his Royal Society lecture, Niemann²⁹ presents some other points that correspond well to the ideas of our paper. Summing up Popper, he repeated that “all life is problem solving. Acquiring new knowledge is always purposeful activity.”^{29,(p. 90)} He insisted that “in all cases the activity comes from outside of the DNA. The former ‘centre of life’ is rather a dead place.”^{29,(p. 96)} That it is the cell that divides, not only the DNA.^{29,(p. 98)} And that it is “The cell … also managing the genome.”^{29,(p. 101)} This insight resembles that of Barbara McClintock, the discoverer of natural genetic engineering³³ in saying that “the genome is an organ of the cell.”³⁴ 

Finally, there is his point that “influences (on action) [are] traceable in hindsight … we are unpredictable but not irrational” (Ref. 29, p. 110). Popper therefore arrived at many of the points we are making here.

It would therefore be surprising if he had not also seen the obvious implication, which is that organisms harness stochasticity; otherwise, choice behavior would not be possible. We are grateful to Hans-Joachim Niemann for directing us to Popper sources preceding his Royal Society lecture where he does clearly draw the correct conclusion. Some of the relevant texts occur in his dialogue with John Eccles The Self and Its Brain.³⁵ Popper writes “New ideas [in statu nascendi] have a striking similarity to genetic mutations” and continues “describing ‘the process with respect to new ideas and to free will decisions’ (Ref. 35, p. 540). As randomly produced proposals followed by selection based on standards coming from the world” (cf. Ref. 36, Secs. 31–33). Popper arriving at this conclusion is all the more remarkable for the fact that it required him to abandon his earlier (1973) conclusion that “indeterminism is not enough.”^{37, 38}

The main difference is that while he envisaged “the cell … also managing the genome,” (Ref. 29, p. 101). He does not seem to have arrived at the details of the comparison with hypermutation in the immune system. Perhaps, this is attributable to the fact that the discovery of some of the detailed molecular mechanisms of somatic hypermutation occurred in 1999 after his death in 1994.^{39, 40} There may also have been a puzzle regarding the molecular mechanism of hypermutation. Increasing the natural mutation rate by a factor of up to 10⁶ must have seemed implausible. But this is also roughly the order of magnitude difference between the natural mutation rate in DNA copying before and after repair by cellular editing mechanisms. Mismatch DNA repair is indeed suppressed during somatic hypermutation.⁴¹ 

Our main conclusion is that it is possible to construct an account of choice behavior using stochastic processes by analogy with the way in which the immune system harnesses stochasticity to discover novel solutions to new challenges. There are several predictions and implications.

One of the implications is that it could be important in investigations of choice behavior in animals to include tests for signs of delay or other behavioral signs attributable to stage 3 in our choice process. These could include hesitation (time taken to decide), displacement activity, or other signs of puzzlement. Just as an example, we could take from many good and interesting studies of animal choice; a study of risk-taking behavior in primates⁴² was successful in showing varying degrees of risk-taking in the different primate species but did not include any parameter that would answer this question. Most studies on choice in animal behavior seem to be assuming that animals behave as though they solve a calculation of probability. Thus, in the cited paper, we find: “Any agent, in order to successfully navigate a world of possibilities, needs to strike the right balance between these factors, utilizing mechanisms that when confronted with risky choices, lead to decisions, which optimally combine the probability of receiving a reward multiplied by the amount of the reward.” Animals may not actually be “calculating” in quite the way this quote implies. If we are correct, no calculation or its equivalent, using, e.g., forms of Rational Choice Theory, could represent all of what is happening. That is particularly true when extrapolation to human behavior is involved. To quote the same source: “Based on our findings, we propose that decision-making in the great apes provides a promising context for the interpretation of decision-making in humans, the fifth great ape species.” We agree with this conclusion, but note that it will be particularly important to consider the role of stochasticity in both animals and humans.

Krupenye et al. have in any case shown that humans and animals display departures from Rational Choice Theory which they characterize as biases in choice behavior dependent on whether decisions involve losses or gains.⁴³ The involvement of stochastic processes does not of course exclude biases.

Rosati and Hare have shown that chimpanzees and bonobos can distinguish between risk and ambiguity in choices presented to them.⁴⁴ They write “Importantly, apes’ divergent preferences for risk and ambiguity diminished with time: although apes chose the risky option more frequently than the ambiguous option in the first session; by session two they showed no difference. One possibility is thus that the apes are able to rapidly incorporate new information about previously ambiguous options into their decision strategies: after choosing the ambiguity option and receiving some feedback about what it provided, they may have treated the ambiguity and risk option as equivalent because the functional outcome was the same.” The stochastic choice process we describe here would account for this form of learning. By analogy with the immune system model, once a novel challenge has been met, it becomes part of the standard repertoire.

Santos and Rosati have written a valuable review of this field.⁴⁵ They write “we now know that human choice is often not as rational as one might expect.” We see two ways in which this statement can be interpreted. First, within the context of our Choice Process, there is obviously no guarantee that a stochastic process will throw up a fully rational solution. Partial success is what would be expected most of the time. The same is true of the immune system. All it needs to do is to come up with a “good enough” template match. It does not have to be the perfect match. If a key fits the lock, it does not really matter whether it is an exact fit.

Second, that leaves the question how it happens that, nevertheless, most of the time, we and others can give a “good enough” rational explanation of a choice, at least in retrospect. That seems to be true however partial the “fit” seems to be to the problem. A possible solution to that problem could be what Santos and Rosati call the endowment effect. Animals and humans privilege retaining what they already own. Could the same effect operate in the case of decisions? Do we and perhaps other animals “own” decisions. It seems plausible at least.

Observations of primates in their natural environments have extended our knowledge of choice behavior in ways that enable us to obtain important insights into subjective knowledge.

That organisms may know that others have subjective knowledge is itself an important factor in objective knowledge and is part of situational logic or behavioral cognition. Such knowledge necessarily carries with it a great deal of uncertainty. An animal must predict that the other knows and how they might act on such presumed knowledge. This is manifest in both human and non-human animal behavior.

Spinning the wheel as a creative process therefore occurs not solely at a physiological level, but also at a social and cultural level. The evolution of language allows sophisticated and abstract problem solving. Language allows a cultural spinning of the wheel. Thus, chimpanzees use communication that distinguishes private from public interaction. What they know that others may not know is a part of their objective knowledge. Chimpanzees employ signals with a sensitivity to the public/private nature of information, by adjusting their use of signal types according to social context and by taking into account potential out-of-sight audiences.⁴⁶ 

The written and recorded word, together with artistic representation, allows problem solving across many generations—a repository of social wheel spinning, and to “see” the world in different ways. Solutions to problems can differ from group to group depending on context and cultural history. This is evident in the use of tools by chimpanzees to crack nuts. The use of stones to crack nuts has to be “introduced” to the group and is learned by others in the group. Furthermore, the stones are modified to better crack the nuts. Tools may be shared or hidden and kept for later use. This demonstrates creative decision making in practice.

A further prediction is that choice behavior should depend on the processes of plasticity since the ability to store and retrieve the results of stochastic variation requires such plasticity. Tang and Guo⁴⁷ and van Swinderen^{48, 49} showed that choice behavior in Drosophila is strongly affected by mutations that lead to defective short-term memory. The behavior that remains is then rigid optomotor responses. As van Swinderen expresses it, “a strong and non-distractable optomotor response, as seen in the dnc and rut mutants, may reflect failure of an interacting attention-like mechanism designed to periodically alternate among competing percepts of variable salience.” Alternating between competing outcomes of stochastic processes is precisely what must be involved in the choice process.

D.N. would like to thank Professor Michael Joyner for hosting his visit to the Mayo Clinic in Rochester, MN, while this article was being written. R.N. is an Honorary Senior Lecturer at the Institute of Women’s Health at University College London. We thank Hans Braun, Michael Joyner, Anthony Kenny, Hans-Joachim Niemann, and the two journal referees for valuable comments on early drafts of this article.