Modeling Population Dynamics & Evolutionary Cooperation with Game Theory

Steven Xu, Tri Vinh Truong, Emma Wong, Quinten Bennett

Abstract

Since the development of game theory, it has been used to model behaviours exhibited by individuals in situations involving strategy optimization involving mathematical concepts and calculations. When applied to the field of evolutionary biology, evolutionary game theory is derived to model the competitive strategies and evolutionary decisions found in many species of organisms. For example, the Hawk-Dove game analyzes payoffs and uses mathematical models to simulate the development of aggression between species; the Prisoner’s Dilemma models selection for mutual cooperation or defection by transforming the corresponding payoff matrices; plants investigate and evaluate between growing by mass or effectiveness and follows the mathematical model that gives the best outcome. Analysis of these models and the corresponding effects on competitive and cooperative evolution provide a better understanding of the strategic decisions made by species along their evolutionary pathway.

Introduction to Evolutionary Game Theory

Game theory traces its origin back centuries to ages when discussion of optimizing strategies in chess—along with other games of chance—began, and later developed into ideas for which the basis of modern game theory as mathematical interpretation could be applied. However, the foundation of the mathematical structure of game-play-strategies wasn’t clearly defined or formulated until the early 1900s—logician Ernst Zermel published a paper in 1913, using the set theory to prove that an optimal chess strategy can be determined (Larson, 2010). Soon after, mathematician John von Neumann published “On the Theory of Games of Strategy” (1928), finally establishing game theory as a unique field. Following this paper, a few books had been published by Neumann who initially intended to invest game theory for optimizing individual’s strategic decisions using mathematical manners (Leonard, 2010), but it turned out that game theory has much more potential.

Game theory underwent rapid extensive development and found its applications in other fields as more studies were initiated and pursued, as described by John Maynard Smith in his book Evolution and the Theory of Games (1973): “The last decade has seen a steady increase in the application of concepts from the theory of games to the study of evolution.” (p. vii). Since game theory provides methods to quantify and rationalize strategies and decision-making processes, the concept of the theory extends to fields of study that involves optimizing decisions; to mention a branch of it, evolutionary game theory is developed combining the concepts of game theory and evolutionary biology.  

The essence of game theory is the concept of “games” itself. In the scope of game theory, a “game” is defined as the situations that the decision-making individual, referring to as the “player”, may maximize its gains through anticipating and evaluating the possible strategies of other “players” (Ross, 2021). This definition extends the concept of “games” beyond how it is commonly known as a form of entertainment to a wide range of situations from the playful disputes to the major conflicts between individuals. Resultantly, the broad concept of “games” can be further divided into two major categories: non-cooperative and cooperative. Cooperative games indicate that the commitments each player makes are binding and enforceable—that is, contracts and agreements made by cooperative players are strictly enforced by external measures while non-cooperative individuals may act to their own advantage, and the agreements between players must be maintained by themselves (Harsanyi, 1966). In the context of evolutionary game theory, many natural competitions are non-cooperative games where mutual agreements between animal species are not governed or enforced by external forces even if such agreements could exist.

Another important concept of the game theory biological studies is Nash equilibrium, proposed by John Forbes Nash Jr. (1950). It describes the existence of a stable equilibrium within an n-player non-cooperative game, where each player selects a strategy from its pool of finite strategies with definite payoff (Nash, 1950). Nash equilibrium is achieved with each player selecting their best strategy, under the condition that they aim to maximize their own payoff while taking account of possible strategies other players would pick given their goals are identical. One of the conditions for Nash equilibrium is that all players must make rational decisions when evaluating their choices—a condition that would be extremely difficult to meet in a natural environment. However, the model of evolutionarily stable strategy (ESS) shows that the evolutionary decisions in animal competitions follow the mathematical model of Nash equilibrium (Smith, 1974). Since then, many studies have been made on the topic of analyzing complex evolutionary behaviors using game theory and Nash equilibrium as important mathematical tools. The evolutionary game theory can now be examined in greater detail along with different natural mechanisms.

The Hawk-Dove Game

The most common game used to model population dynamics within a species of animal takes the form of the Hawk-Dove game. In this game, pairs of animals will meet each other and contest a resource. A given animal may adopt two possible strategies: they may either behave as “Doves,” and back down in the face of a conflict for resources, or they may behave as “Hawks,” who fight and compete for resources (Eldakar, 2020). In the simplest form of this game, when two doves meet, they will share and distribute the resource amongst themselves evenly, resulting a payoff V/2 for each dove, where V represents the resource. When a dove meets a hawk, it will back down and gain no resources, while the hawk takes all the resources for itself, resulting in a payoff  for the hawk, and a payoff R for the dove. And finally, when a hawk meets another hawk, they will fight each other for the resources, incurring a cost to themselves due to the energy spent while fighting, resulting in a payoff (V – C)/2, where C represents the cost of fighting. Once the resource is obtained, the hawk or dove will either die or reproduce, depending on the resources it gained in one round of the “game” (Eldakar, 2020). These interactions can be described in a payoff matrix as shown in Figure 1:

Using this payoff matrix, it is possible to determine the average fitness W of an individual based on the number of resources gained in an interaction, weighed by the probability of that interaction happening. If we let p to be the fraction of individuals employing the “dove” strategy, and correspondingly (1-p) to be the fraction of individuals employing the “hawk” strategy, then the average fitness for a dove may be calculated as follows:

W =\text{ (probability of encountering a Dove) ∗ (payoff of encountering a Dove) + (probability of encountering a Hawk) ∗ (payoff of encountering a Hawk)}

The average fitness for a dove:

W_{D} = p \frac{V}{2} + (1-p)(0)

The average fitness for a hawk:

W_{H} = p(V) + (1-p)( \frac{V}{2} - C)

Setting V= 1, graphing out these fitness equations shows the following (Fig. 2, 3):

To find at what costs C allow for the coexistence of doves and hawks, we set W_d = W_H to find values of p at which neither doves nor hawks are favored, obtaining the following equation:

p (\frac{V}{2}) = p(V) + (1-p)(\frac{V}{2} - C)

p = 1 - V/2C

This effectively allows us to determine the fraction of doves that is able to coexist with a population of hawks, depending on the value of C. For example, setting C = 1 and V = 1 means that Nash equilibrium will be obtained when p = ½, as shown in Figure 2. On the other hand, setting C = 0 and V = 1, as shown in Figure 3, ensures that there will be no fraction p for which W_d = W_h, resulting in a prisoner’s dilemma, wherein all individuals choose to be hawks, or defectors, rather than cooperate, or behave as doves (Eldakar, 2020).

Additionally, there is no value C for which the W_d is always larger than W_h; in other words, the dove strategy will never become an ESS, no matter the value of C. This effectively means that if a single hawk were to invade a population of doves, it would eventually dominate the overall population of individuals, no matter the costs incurred by infighting amongst hawks. On the other hand, the hawk strategy can become an ESS, provided V/2 > C; if a dove were to invade a population comprised exclusively of hawks in this scenario, the dove population would be unable to grow and displace the hawk population. This imbalance can be explained by the fact that hawks will tie with hawks, and dominate over doves, while the best doves can hope for is a tie with another dove (Eldakar, 2020).

Given the significant disadvantage that cooperators face against defectors how can cooperators possibly gain the upper hand? This dilemma can be addressed upon investigation of the various mechanisms of evolutionary cooperation.

Transformations of the Prisoner’s Dilemma Matrix

Natural selection often leads to cooperation between competing individuals. As discussed earlier, individuals may choose to cooperate or defect when facing another “player.” The Prisoner’s Dilemma states that a defector will receive the highest payoff if they are up against a cooperator, while the cooperator will receive the lowest payoff. In the cases where competing individuals make the same decision, mutual cooperation has a significantly greater payoff than mutual defection. These payoffs can be compiled into a matrix as a standard framework for evolutionary dynamics (Fig. 4).

The fitness of an individual depends on the frequency of different strategies stemming from phenotypes in a population, where the ideal outcome is reproductive success (Taylor & Nowak, 2007). Evolutionary game dynamics can describe frequency dependent selection through variations of the Prisoner’s Dilemma payoff matrix. Local competition of strategies involved in evolutionary dynamics is therefore represented by transformations of this standard framework matrix. Here, four mechanisms for evolution of cooperation will be explored.

Mechanisms of Evolutionary Cooperation

Direct Reciprocity

In the context of a repeated Prisoner’s Dilemma, direct reciprocity can emerge from repeated encounters between the same two individuals, ultimately leading to evolution of cooperation (Taylor & Nowak, 2007). Individuals participating in direct reciprocity will choose between cooperation and defection—defectors always defect, but cooperators first start with a cooperation, then imitate the other player’s decision in the previous move. Direct reciprocity is represented by a transformed matrix, where w is the probability there is another round and 1 – w is the probability the game is over (Fig. 5). The evolutionarily stable strategy (ESS) always lies in defection—only in special cases will both cooperation and defection be able to dominate. The inequality described in Figure 6 is the minimum requirement for evolution of cooperation, given that there are sufficient many rounds. If the inequality holds, the game is coordinated and both defection and cooperation are ESS. If it does not hold, defection dominates cooperation.

Indirect Reciprocity

  Although it arises from direct reciprocity, indirect reciprocity differs from the previous mechanism by happening in the presence of “interested audiences.” By this, it implies that an individual’s decision is based off its knowledge of the reputation of competitors from previous encounters (Taylor & Nowak, 2007). Each conflict between two players is witnessed by surrounding individuals, who absorb the information for future reference. Resultantly, there are two possible strategies organisms can apply: defection from defectors, and cooperation from cooperators unless they are aware that their competitor is a defector, in which case they will likewise defect. Indirect reciprocity follows the exact same analysis as the payoff matrix for direct reciprocity, but here the probability to know the reputation of another individual is q and the probability that a cooperator will cooperate with a defector is 1 – q. The transformed matrix for indirect reciprocity is similarly developed (Fig. 7). To achieve evolution of cooperation from indirect reciprocity, the probability to know the reputation of another player must exceed the threshold value represented by the inequality in Figure 8. Otherwise, the defection will dominate, and the game will not be coordinated.

Kin Selection

Different species will participate in competitive interactions, such as the constant struggle between predator and prey to seize the advantage over one another. However, it is often observed that similar evolutionary games are played between genetically related individuals. In the case of kin selection, an individual’s survival is promoted by the gene encoding altruistic behavior towards another individual if the recipient of the altruistic act also possesses the gene (Taylor & Nowak, 2007). A population with an average genetic relatedness (r) between interacting players given by 0 < r < 1 can be considered. The payoff received by the other player is added to the individual’s payoff multiplied by r. To keep the total payoff constant between players, this sum is divided by 1 + r and the following transformed matrix is obtained (Fig. 9).

The matrix for kin selection payoff presents two new inequalities, where r_C is the relatedness of the cooperator, r_D is the relatedness of the defector, and r is the relatedness between the interacting players. Cooperation is ESS if the inequality stated in Figure 10 holds, while defection is ESS is the inequality in Figure 11 holds.

The relative ranking of r, r_C and r_D determines the evolutionary outcome of the related species. Three pathways defined by the parameters,

1. R + P > T + S
2. R + P < T + S
3. R + P = T + S

can be further broken down into several possible outcomes.

If R + P > T + S, then r_D > r_C and the following three outcomes may occur:

1. r_D > r_C > r (defectors dominate),
2. r_C > r > r_D (both cooperators and defectors are ESS),
3. r > r_D > r_C (cooperators dominate)

Similarly, if R + P < T + S, then  r_C > r_D and three more outcomes are possible:

1. r_C > r_D > r (defectors dominate)
2. r_C > r > r_D (neither cooperators nor defectors are ESS)
3. r > r_C > r_D (cooperators dominate)

The third pathway is degenerate, where R + P = T + S and r_C = r_D. This is known as “equal gains from switching,” since bi-stability between cooperators and dominators is impossible, so dominance will constantly switch between them (Taylor & Nowak, 2007). In this special case, the inequalities in Figure 10 and Figure 11 lead to Hamilton’s rule, r > c/b.

Group Selection

Competition may occur between groups of organisms, not only individuals. Generally, groups will divide at rates proportional to the average fitness of the individuations constituting it (Taylor & Nowak, 2007). Cooperator groups have the constant payoff R, and defector groups have the constant payoff P. A new payoff matrix can be obtained for selection between competing groups (Fig. 12.).

Sometimes, groups within the same population may compete. Consider the number of subdivisions of a population to be m, with a maximum group size n. The maximum population size is therefore mn. Using these defined parameters, the overall dynamics on two levels of weak selection between groups of the same population (w << 1, large m and n) can be described by a single matrix (Fig. 13.). This matrix is the sum of the standard Prisoner’s Dilemma payoff matrix (Fig. 4) multiplied by group size n, and the matrix described in Fig. 12. multiplied by the number of groups m.

There are two inequalities to consider in order to determine which group has the evolutionarily stable strategy (Fig. 14., 15). Similar to kin selection, there are also two parameter regions,

R + P > T + S \text{ and } R + P < T + S

 with the same possible outcomes, but with different threshold values.

Game Theory Models for Plant Ecology

Game theory has been integrated into several plant ecology models. Most models for a plant’s adoption of specific traits, which can be referred to as a strategy, consider what is best for the plant in the context of certain external biotic and environmental factors. However, few consider how this strategy may be influenced by other plants (McNickle & Dybzinski, 2013). Plant growth models based on game theory allow for the explicit inclusion of other individuals, resulting in a “game” situation between two organisms and their competing strategies (McNickle & Dybzinski, 2013).

Game theory models within plant ecology vary greatly based on the traits of focus as well as their approach. An individual’s strategy could be determined as its’ roots’ biomass (McNickle & Brown, 2012), and a plant’s fitness could be considered by comparing the number or size of fruiting bodies it produces (Maina et al., 2002). As well, certain assumptions can be made, such as unlimited access to light and carbon, to focus the model on the competition between organisms concerning certain resources (McNickle & Brown, 2012). What most models share is the idea of the Evolutionarily Stable Strategy (ESS) (McNickle, & Dybzinski, 2013). While the ESS outside of a controlled experiment shifts constantly due to changes in the environment, each individual and species will continue to adapt towards it (Fig. 16.) (McNickle, & Dybzinski, 2013).

Over generations, the chosen strategy can shift towards the ESS through natural selection, as individuals whose fixed strategy lies closest to the ideal are best fitted to reproduce. However, if a plant is capable of phenotypic plasticity, it would be capable of changing its own strategy depending on the actions of other plants (McNickle, & Dybzinski, 2013). This results in “games” that last a single organism’s lifetime. While the mechanisms by which plants can read others in their surroundings and react remains to be clearly understood, this activity has been observed (McNickle & Brown, 2012).

One model by McNickle & Brown (2012), considers the fitness of plants growing in a shared space with a constant availability of soil nutrients regardless of population density. The model considers root biomass (the strategy), root cost, as well as the effectiveness of root nutrient collection and assumes unlimited carbon and light access. The model uses the concept of a best response curve to find the ESS of two competing plants. The curve demonstrates the best strategy a plastic plant can take in response to their neighbour’s, with the ESS corresponding to the point where the curves meet (Fig. 17.).

The model considers two situations: competition between intra-species plants and inter-species plants at differing resource availabilities. At low resource levels, root propagation remains low as both individuals focus on improving root efficiency over growth (McNickle & Brown, 2012). As availability increases, root proliferation increases in both cases (McNickle & Brown, 2012).

In the case of inter-species competition, the plant with the lower root cost or higher root effectiveness is predicted to have the greater root biomass. Their opponent admits defeat and instead focuses on improving efficiency (McNickle & Brown, 2012). However, in the case of intra-species competition, the ESS is the same strategy for both plants. This would result in much greater root proliferation of both individuals than in the inter-specific case, as they attempt to gain an advantage in an equal game despite the increased investment (Mcnickle & Brown, 2012). This situation can be considered as a tragedy of the commons. A tragedy of the commons is an event where individuals in a community over-allocate resources to a specific task at a deficit, only doing so to remain competitive (McNickle, & Dybzinski, 2013). An example of this is the extreme height of tree canopies (McNickle, & Dybzinski, 2013). While the community could maintain itself at much lower levels of resource allocation, such as only growing a certain height, with the same or greater benefit, any individual that commits more would be at a distinct advantage over others. Everyone must continue to waste resources to remain competitive. As a tragedy of the commons occurs due to a game to minimize losses to defectors, it can be considered a Prisoner’s Dilemma.

The results described above parallel the findings of a study conducted by Maina et al. (2002) on the Kenyan variant of the Phaseolus varigaris bean. Two pairs of pots were filled with soil of similar nutrient levels. The roots of a sprouting bean plant were placed either in one of the two pots by themselves (“owners”), or into both pots of the pair, shared with another sprout (“fence-sitters”). By the end of the study, while both plant types shared similar above-ground biomass levels, the fence-sitters had more than 150% the root mass as owners. These roots grew into areas containing their “opponent’s” roots to steal nutrients away from them. However, the isolated owners showed greater fitness as they produced more and larger pods, with a total 90% more pod-mass than the fence sitters (Fig. 18.) (Maina et al., 2002). In this scenario, participation in the game resulted in a tragedy of the commons, as more resources were allocated to the strategy of remaining competitive as opposed to reproduction.

Like most mathematical models of natural phenomena, the use of game theory cannot completely describe a plant’s growth with respect to its community. However, its implementation can aid in illustrating interactions between competing organisms. Ideally, overtime organisms will adopt better strategies culminating in the ESS.

Conclusion

Game theory has numerous applications including several in biology. Using game theory, interactions between organisms can be characterized as “games” in which each organisms chooses a strategy, the best of which optimizes gains and minimizes losses. By using these games as a framework, it is then possible to explain the evolution of certain animal behaviors, such as the evolution of group dynamics and cooperation. The Hawk-Dove model provides a basis for the relation of strategy frequency and population dynamics. Differing cooperation mechanisms that arise between organisms to reduce competition and losses can be mathematically integrated into the game via a transformation of the game’s matrix. Game theory models for developing plant ecology have shown parallels to games generated via experimentation, sometimes both sometimes resulting in a tragedy of the commons. Integrating game theory into models describing biological phenomena can immensely aid in our understanding of them.

References

Eldakar, O. T. (2020). Hawk-Dove Model, The. In V. Zeigler-Hill & T. K. Shackelford (Eds.), Encyclopedia of Personality and Individual Differences (pp. 1905-1910). Springer International Publishing. https://doi.org/10.1007/978-3-319-24612-3_1645

Harsanyi, J. C. (1966). A General Theory of Rational Behavior in Game Situations. Econometrica, 34(3), 613-634. https://doi.org/10.2307/1909772

Larson, P. B. (2010). Zermelo 1913. In E. Zermelo, H.-D. Ebbinghaus, C. G. Fraser, & A. Kanamori (Eds.), Ernst Zermelo – Collected Works/Gesammelte Werke: Volume I – Set Theory, Miscellanea / Band I – Mengenlehre, Varia (pp. 260-273). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-79384-7_9

Leonard, R. (2010). Von Neumann, Morgenstern, and the Creation of Game Theory: From Chess to Social Science, 1900–1960. Cambridge University Press. https://doi.org/DOI: 10.1017/CBO9780511778278

Maina, G. G., Brown, J. S., & Gersani, M. (2002). Intra-plant versus Inter-plant Root Competition in Beans: avoidance, resource matching or tragedy of the commons. Plant Ecology, 160(2), 235-247. https://doi.org/10.1023/A:1015822003011

McNickle, G., & Brown, J. (2012). Evolutionarily stable strategies for nutrient foraging and below-ground competition in plants. Evolutionary Ecology Research, 14, 667-687.

McNickle, G. G., & Dybzinski, R. (2013). Game theory and plant ecology. Ecology Letters, 16(4), 545-555. https://doi.org/https://doi.org/10.1111/ele.12071

Nash, J. F. (1950). Equilibrium points in <i>n</i>-person games. Proceedings of the National Academy of Sciences, 36(1), 48-49. https://doi.org/doi:10.1073/pnas.36.1.48

Roos, D. (2021, September 21). Game Theory. The Stanford Encyclopedia of Philosophy. Retrieved November 25, 2021, from https://plato.stanford.edu/archives/fall2021/entries/game-theory/

Smith, J. M. (1974). The theory of games and the evolution of animal conflicts. J Theor Biol, 47(1), 209-221. https://doi.org/10.1016/0022-5193(74)90110-6

Smith, J. M. (1982). Evolution and the Theory of Games. Cambridge University Press. https://doi.org/DOI: 10.1017/CBO9780511806292

Taylor, C., & Nowak, M. A. (2007). Transforming the Dilemma. Evolution, 61(10), 2281-2292. https://doi.org/https://doi.org/10.1111/j.1558-5646.2007.00196.x

von Neumann, J. (1928). Zur Theorie der Gesellschaftsspiele. Mathematische Annalen, 100(1), 295-320. https://doi.org/10.1007/BF01448847