Sleeping in Flight: Adaptations of a Bird’s Brain

Ziyad Bouslama, Young Chae Han, Stephanie Makhlouf, Catherine Xu

Introduction

Flying has required birds to develop a plethora of tools to make it as efficient as possible. These biological tools range from different body sizes, body weights, wingspans, feather numbers and their distribution, to an increase in brain to body size ratio to accommodate more developed functions. Multiple types of birds including swifts, sandpipers, songbirds, and seabirds can fly non-stop for days, weeks, and even months without landing in response to numerous ecological demands and limitations. This induces a certain capacity for these birds to sleep or rest while in the air. From avoiding the waste of resources needed to find a place to land and rest, especially when migrating over large bodies of water, to avoiding predators, as birds have few to no flying predators, the ability for a bird to rest while flying brings many advantageous features.  

What adaptations make ‘sleeping on the wing’ possible?  Brain lateralization and the functional discrepancies between the two hemispheres in birds gives them the ability to sleep asymmetrically, with one hemisphere of the brain staying more awake than the other, allowing them to keep one eye open. There are two main types of sleep: slow-wave sleep and REM sleep. In birds, slow-wave sleep can happen both bihemispherically (i.e., both hemispheres are asleep) and unihemispherically (i.e., one hemisphere stays awake while the other sleeps). Meanwhile, REM sleep happens bihemispherically and is accompanied with twitching and a reduction in muscle tone. In addition to the ability to sleep unihemispherically, unique mechanisms in birds allow for a degree of muscle tone even during bihemispheric sleep, which may help with aerodynamic control when sleeping in flight.

A Bird’s Brain

Flying required birds to adapt their brains accordingly: their brain’s visual system and hemispheres have grown larger to accommodate the need for better visual evaluations at fast speeds and for powered flight (i.e., flapping of their wings) (Husband and Shimizu, 1999). The main parts of a bird brain are the medulla oblongata, cerebellum, hypothalamus, optic lobes, olfactory lobes, and the two cerebral hemispheres.

The medulla oblongata is the part of the brainstem that mainly connects the brain to the spinal cord of the bird; its neurons are responsible for heat rate control, blood pressure and respiration (Ritchison, n.d.). The spinal cord serves as the pathway for the brain to send and receive messages to and from the sensors and receptors of organs and muscles. However, the shape of the spinal cord differs between animals. Indeed, a bird’s spinal cord is larger at the neck level, as it is the connection point to the wings, which require a lot of processing power from the brain (Ritchison, n.d.).

The cerebellum is responsible for precise and coordinating actions between muscles — mainly in the wings — to guarantee efficient flight. It is significantly large and folded, with over 10 folds, which lets the cerebellum house a more important number of neurons (Husband and Shimizu, 1999). 

Their brains also possess a hypothalamus, which is the mediator between the nervous and the endocrine system — which is responsible for hormones — and an optic lobe on each side of the cerebellum responsible for any visual information. The optic lobes are linked by an optic chiasma, a region where the optic nerves cross to go to the opposite eye. This suggests lateralization of bird brains, as each optical lobe is responsible for a different eye (Ritchison, n.d.).

Lastly, a bird’s brain contains two cerebral hemispheres with a small olfactory lobe each. The cerebral hemispheres, part of the cerebrum, have many functions depending on the bird, ranging from singing to food searching to nesting. The olfactory lobes are responsible for the sense of smell of the bird, which is usually weak or unnecessary (Ritchison, n.d.).

Birds and reptiles have similar brains, as they share a common ancestor: the dinosaurs. Birds have, however, developed larger brains compared to their body size, with bigger cerebral hemispheres and cerebellum, similar to mammals. Due to the nature of their needs, birds also evolved to have larger optic lobes, which translates into better sight and smaller olfactory lobes, thus suggesting a weaker sense of smell (Husband and Shimizu, 1999).

Nonetheless, for over a century, it was thought that a bird’s brain mainly consisted of basal ganglia with almost no malleable or flexible parts — grey matter, — which translates to a bird acting solely on instinctual behaviors. 

In addition, previously and for a long time, bird’s cerebrum has been thought to be mainly composed of striatum, which is the part of the brain responsible for motor functions — along with the cerebellum, — response stimulus, and learning, which plays a role in the development of reflexes and a reward system based on actions taken. All of these are linked to behaviors based on instinctive actions. In comparison, the human brain’s upper layers are made of pallium cells, which are nerve cells (also referred to as grey matter). The latter are responsible for storing memory of various things and can easily change and form new nervous links (Jarvis et al., 2005). 

The white part, referred to as white matter, in the human brain consists of axons, extensions of the nerves cells that connect them to other nerve cells or other parts of the brain and in which messages of the nervous system pass through. Axons are white from the myelin that covers them; myelin, in part, accelerates the speed of nervous system messages flowing through the fibers (Jarvis et al., 2005).

Recent research and efforts by the Avian Brain Nomenclature Forum have shown that a bird’s brain, like a human’s, is largely made of pallial cells. A new part of the brain has also been identified in birds and mammals in general: the pallidal cells (Jarvis et al., 2005). These cells are mainly involved in the smoothness of all voluntary actions and act unconsciously, without needing to think about it. This can be observed for humans when walking as the movement made by the different parts of the legs are very precise and regular without the need to focus on them. They are also responsible for the learning of new reflexes and movements (Gillies et al., 2017). This new approach in viewing the bird’s brain helps with possible explanations about how birds have learned to spend long durations in the air.

Lateralization of the Brain

In the human brain, there are two cerebral hemispheres: the left hemisphere and the right hemisphere. The left hemisphere is responsible for the right side of the body whereas the right hemisphere is responsible for the left side of the body. These two hemispheres are also particularized in different functions: the left-side regulates logic and language while the right-side controls spatial ability and creativity (Corballis, 2014). The differential use of each hemisphere is due to the functional specialization of the brain called “lateralization”.  Such characteristic is also present in birds, where the distinct functional differences are obvious. Just like the human brain, the brain lateralization in birds displays major functional discrepancies. 

Left Hemisphere/ Right Hemisphere

As mentioned above, there are distinct functional and structural differences between each hemisphere of a bird’s brain. Similar to the lateralization in the human brain, the specific nerve arrangement in birds will cause the left hemisphere to process information from the right side of the body and vice versa (Birkhead, 2012). This means that each hemisphere is specialized in a particular function since they can process different types of information. The brain lateralization and asymmetrical brain function in birds reduces conflicts, giving them a major advantage in neural processing and efficiency. In other words, birds may perform and complete various tasks simultaneously due to brain lateralization.

Asymmetrical visual lateralization is very significant when discussing the topic of lateralization in birds. This is mainly due to the fact that the eyes of a bird are placed laterally (Fig. 4), meaning that its brain will process each stimulus in different ways depending on which side of the head it was seen. In studies of this topic, the relation between light exposure and lateralization was found. In 1980, Lesley Rogers and Gisela Kaplan determined the significance of light exposure to birds during their embryonic development stage. They found out that a chick embryo turns its head sideways during the last days of incubation, covering its left eye from the light. This means that the light can only reach the right eye by going through the shell and the membranes, which will then create asymmetry in the role of each eye. Later, Rogers and Kaplan conducted an experiment where they incubated eggs in the dark and found out that the eyes of the hatched chicks showed no difference in the role of each eye. This proves that the visual lateralization is not determined genetically, and the role of each eye is reversible by modifying the light exposure during their formative stages (Rogers and Kaplan, 2019).

Asymmetries and Lateralized Biases

The most challenging part about visual asymmetry in birds is that the preferences for the left or the right side cannot be generalized; they show no preferences in using one eye over the other. This is because the bias in the role of each eye can take place in different ways for different birds. As such, bias does not exist in the population of birds as a whole, and each bird can have its own preferences. Interestingly, studies show that the more biased the bird is, the less proficient and competent it will be in solving complicated problems (Birkhead, 2012). 

Just like the human brain, each brain hemisphere of a bird is specialized in a particular function. Birds will develop their own bias based on the functions controlled by each hemisphere. Table 1 below provides a summary of biases recorded in different bird species.

Sleep During Flight

Because of the way bird brains are lateralized and other specific mechanisms in the nervous system, birds sleep differently compared to humans and most mammals. Many bird species are even able to sleep during flight.

Unihemispheric Sleep

Birds alternate between two types of sleep: slow wave sleep (SWS) and rapid-eye movement (REM) sleep. In contrast to SWS, REM sleep is characterized by random rapid movement of the eyes and low muscle tone throughout the body.

An electroencephalography (EEG) — a monitoring device that measures brain waves — of REM sleep reveals irregular but overall high-frequency brain waves, whereas slow-wave sleep is characterized by continuous, high-amplitude, and low-frequency (0.75-4.5 Hz) EEG readings (Rattenborg et al., “Evidence that birds sleep in mid-flight”). Another difference between these two types of sleep is that slow-wave sleep can occur in one or both brain hemispheres at a time whereas rapid-eye movement sleep solely happens bihemispherically (meaning in two hemispheres at the same time) (Rattenborg, 2006).

Sleep is more asymmetric in flight than when on land. In fact, during long flights, birds may sleep in a unique way called unihemispheric slow-wave sleep (USWS), also known as asymmetric slow-wave sleep. As its name indicates, USWS is a type of sleep where one brain hemisphere rests while the other hemisphere remains alert (Lapierre et al., 2007). Brain waves during USWS can be visualized electrographically: unihemispheric sleep consists of an EEG composed of low-frequency and high-amplitude waves (slow waves) recorded in the sleeping hemisphere and an EEG typical of wakefulness, composed of high-frequency and low-amplitude waves in the other hemisphere (Mascetti, 2016).

According to Fuller (Mascetti, 2016), some neural groups promote awakening by activating the cortex and subcortical structures while simultaneously inhibiting neural groups that promote sleep. In USWS, these neural groups are stimulated according to the need of each hemisphere. Therefore, a high activity of neural groups that promote awakening are found in the non-sleeping hemisphere and vise-versa. Furthermore, USWS is regulated by neurotransmitters; particularly, the differential release of the neurotransmitter acetylcholine, which has been linked to hemispheric activation. Therefore, the maximal release of acetylcholine is lateralized to the awake hemisphere. In contrast, a minimal release of acetylcholine is found in the hemisphere exhibiting an EEG of SWS (Lapierre et al., 2007).

Sleep during powered flight may also be possible since stereotypical locomotor movements such as walking or flapping are controlled by spinal reflexes. In fact, since slow-wave sleep can occur in one hemisphere at a time or in both hemispheres simultaneously during flight, researchers concluded that birds do not need unihemispheric sleep for aerodynamic control. That said, unihemispheric sleep is beneficial for birds during flight because the eye connected to the awake hemisphere can remain open, allowing birds to visually navigate during sleep and avoid collisions with other birds (Dvorsky, 2016). For example, frigatebirds sleep mostly while circling in rising air currents and keeping the eye connected to the awake hemisphere and facing the direction of flight open, suggesting that they use unihemispheric sleep to watch where they are going.

On average, during flight, birds sleep for 42 minutes per day meaning they spend less than three percent of their time asleep whereas, on land, the bird would sleep for about 12 hours a day. Despite what one might believe, the reduced sleep quantity does not present limits to the birds’ behavioral or health level. In fact, researchers believe that birds can compensate for the lack of sleep thanks to their efficient immune system, brain plasticity, thermoregulation, and restoration of brain metabolism (Mascetti, 2016).

Bihemispheric Sleep – Muscle Tone

Birds have a trait that makes them very unique from mammals: they can maintain a degree of muscle tone during REM sleep. Some mammals can keep control of certain muscles during slow-wave sleep, such as horses who often stand while sleeping. However, all mammals completely lose skeletal muscle tone — the muscles that require voluntary control — when they enter REM sleep, forcing them to lay down (Rattenborg et al., “Local Aspects of Avian Non-REM and REM Sleep”). On the other hand, multiple species of birds have proven to be free from this limitation that mammals face, as muscle tone seems to be regulated at a local level. For example, when geese sleep with their heads facing forward, despite their head dropping a bit during REM sleep, electromyography recordings show continuous muscle tone in the neck. In addition, many birds can stand during both non-REM and REM sleep, some even on one foot (Rattenborg et al., “Local Aspects of Avian Non-REM and REM Sleep”) (Fig. 9). This ability is often attributed to the ‘automatic digit flexor mechanism’, which causes tendons to pull a bird’s foot closed when relaxed, and the ‘digital tendon locking mechanism’ which is made up of interlocking ridges in the tendons and tendon sheaths that help keep the digits closed. However, birds that die in their sleep will fall from their perch, indicating that some muscle tone is still required to balance while standing, especially if standing on one foot. It is likely that the mechanism involved in the local muscle tone control in birds’ necks is the same one that allows birds to stand during REM sleep (Rattenborg et al., “Local Aspects of Avian Non-REM and REM Sleep”). One could conclude that this is the same mechanism that allows for birds to maintain aerodynamic control when they enter bihemispheric sleep mid-flight, a behavior observed in frigatebirds and that likely happens in other sleep-flying birds as well (Rattenborg et al., “Evidence that birds sleep in mid-flight”).

Case Study – Frigatebirds

Frigatebirds (Fig. 10) are a large species of seabirds that spend most of their time flying over tropical and subtropical oceans throughout North and South America (“Magnificent Frigatebird Identification” | All About Birds | Cornell Lab of Ornithology, n.d.). These birds have long wings, barely webbed feet, and reduced feather waterproofing that make take-off from water very difficult. In addition, during both the day and night, frigatebirds must follow ocean eddies to look for foraging opportunities. As an adaptation to these ecological demands, frigatebirds can fly above the ocean for weeks to months without landing, sleeping both unihemispherically and bihemispherically ‘on the wing’ (Rattenborg et al., “Evidence that birds sleep in mid-flight”).

Rattenborg et al. (“Evidence that birds sleep in mid-flight”) are the first, and only so far, to do a comprehensive study and provide concrete evidence of birds sleeping during flight. The researchers recorded EEGs, showing electrical activity of the brain from the hyperpallium — a visual area — of both hemispheres of frigatebirds. They also attached a 3-dimensional accelerometer to each bird’s head to track head movement – sway, surge, and heave. The researchers observed three main patterns in brain activity: alert wakefulness, slow-wave sleep – bihemispheric and unihemispheric – and REM sleep.

On an EEG, alert wakefulness is characterized by low-amplitude high-frequency waves, which were observed in frigatebirds as they glided during the day. These low-amplitude high-frequency waves were frequently interrupted with high-amplitude signals that happened in conjunction with rapid head movements, shown on the accelerometer, likely indicating visual processing in the hyperpallium as the bird looked for foraging opportunities (Rattenborg et al., “Evidence that birds sleep in mid-flight”). At night, the EEGs continued to mostly show wakefulness. However, when flapping at night, the wakefulness was punctuated by isolated high-amplitude slow waves, and when gliding at night, wakefulness was periodically replaced with continuous high-amplitude slow waves (Fig. 11.a).

As previously mentioned, continuous high-amplitude slow waves are indicative of SWS. When the frigatebird was in flight, the EEGs showed that SWS happened mostly asymmetrically or unihemispherically. In flight, about 72 % of SWS was asymmetric and about 47 % of the asymmetric SWS (ASWS) was completely unihemispheric, allowing the bird to keep the eye connected to – and opposite to – the more awake hemisphere open. Fig. 11.b shows examples of EEG recordings of bihemispheric SWS and unihemispheric SWS (Rattenborg et al., “Evidence that birds sleep in mid-flight”).

Being large soaring seabirds, frigatebirds repeatedly circle over thermals to gain altitude then glide down in a straight line to conserve energy. The studied frigatebirds typically had episodes of SWS during their circling climbs and were awake during their descents. Rattenborg et al. (“Evidence that birds sleep in mid-flight”) compared ASWS patterns with data from the accelerometer and gamma activity (30-80 Hz) patterns, which are related to visual activity, and found some correlations with the direction of circling.

The birds are more likely to circle towards the side with slower wave activity (SWA) (Fig. 12. c and d). For example, when there was more SWA in the left hemisphere, the birds circled to the left about 65 % of the time and to the right about 8.8 % of the time. Similarly, the birds circled more often towards the hemisphere with less gamma activity (Fig. 12.e). For instance, when left gamma is less than right gamma, the bird circled to the left about 62 % of the time and to the right about 8.6 % of the time. In other words, the bird prefers to circle towards the side that is opposite to the hemisphere that is more awake and exhibiting more visual activity. Because each hemisphere is connected to the eye on the opposite side of the head, the mentioned correlation indicates that frigatebirds usually keep the eye facing the direction of the turn open (Fig. 13.), supposedly to watch where they are going (Rattenborg et al., “Evidence that birds sleep in mid-flight”).

Frigatebirds are also able to sleep bihemispherically while flying. Indeed, about 28 % of SWS during flight was bihemispheric and these episodes could last for several minutes (Rattenborg et al.; “Local Aspects of Avian Non-REM and REM Sleep”; “Evidence that birds sleep in mid-flight”). As previously mentioned, it can be concluded that unihemispheric sleep is not needed for aerodynamic control during soaring or gliding flight, and that it likely serves other uses, such as keeping an eye open to get visual input of their surroundings. 

The last type of brain activity observed was REM sleep, which happens bihemispherically as well. In the frigatebirds, REM sleep looked a lot like wakefulness, with the EEGs from both hemispheres displaying low-amplitude high-frequency waves, except REM sleep was accompanied by twitching and dropping of the head. This type of sleep only appeared in brief bouts during SWS (Rattenborg et al., “Evidence that birds sleep in mid-flight”).

Like many other species of birds, despite having the ability to sleep on the wing, frigatebirds actually sleep very little during flight compared to when they are on land: they slept for about 2.9 % of their time spent in the air and about 53 % of their time spent on land (Rattenborg et al., “Evidence that birds sleep in mid-flight”) (Fig. 14.a). Even when they were soaring or gliding and sleep was possible, the frigatebirds only slept for about 20 % of the average 9.7 hours spent in these flight styles. In addition, the intensity of both bihemispheric and unihemispheric SWS was lower during flight compared to when on land (Fig. 14.b). The duration of SWS episodes was also shorter in flight: SWS periods averaged at 11 seconds in flight versus 28 seconds on land. This means that ecological demands for attention — such as the need to track ocean eddies and look for foraging opportunities — heavily limit the length and depth of frigatebirds’ sleep during flight (Rattenborg et al., “Evidence that birds sleep in mid-flight”).

Conclusion

Brain lateralization is a fascinating and yet a mysterious topic. The asymmetry in brain function maximizes the brain capacity for neural processing by reducing the conflict of each hemisphere. This characteristic increases the overall efficiency and performance of the brain. Each brain hemisphere is responsible for specific functions, but these functions are not genetically determined and can be altered during the embryonic development stage by changing light exposure. Furthermore, the functional differences and biases can differ depending on the bird species; the functions controlled by the left and right hemisphere cannot be generalized for all birds.  We can see that when some birds, such as frigatebirds, start circling on rising air currents, they switch into slow-wave sleep for a couple of minutes while remaining airborne. In fact, as discussed previously, birds’ brain lateralization allows them to perform asymmetric sleep as well as completely unihemispheric sleep. This unique pattern of sleep allows one hemisphere to rest while the other hemisphere connected to the open eye stays awake, which is very useful for navigating while still having the opportunity to rest. Additionally, both hemispheres can go into slow-wave sleep at the same time while aloft since birds may maintain aerodynamic control even when their entire brain is asleep. This suggests that the main purpose of unihemispheric sleep is related to navigation and getting visual information – and not aerodynamic control. On rare occasions, going into REM sleep for a couple of seconds while they continue flying is also possible but unfavorable likely due to reduction in muscle tone. It is also important to note that, despite having the ability and many opportunities to sleep unihemispherically while flying, birds sleep very little during flight compared to on land and yet they do not experience the same negative effects that mammals like humans experience when they are sleep deprived. Thus, in response to ecological demands that limit the length and depth that a bird can sleep while flying, such as how frigatebirds must continuously navigate ocean eddies to keep up with foraging opportunities, these sleep-flying birds have somehow adapted to function well even when sleep-deprived.

References

Birkhead, T. (2012). Bird Sense: What It’s Like to Be a Bird: Walker. Retrieved from https://books.google.ca/books?id=pt2Swm2lH8YC

Cavanagh, P. (n.d.). Magnificent Frigatebird. In. Audubon Photography Awards: National Audubon Society. Retrieved from https://www.audubon.org/field-guide/bird/magnificent-frigatebird

Corballis, M. C. (2014). Left Brain, Right Brain: Facts and Fantasies. PLoS Biology, 12(1), e1001767. doi:10.1371/journal.pbio.1001767

Dvorsky, G. (2016). We Finally Know How Birds Sleep During Flight Without Dying. Retrieved from https://gizmodo.com/we-finally-know-how-birds-sleep-during-flight-without-d-1784760623

fotocorn. (n.d.). A racing pigeon poses in front of the lens of the camera. In. 123RF. Retrieved from https://www.123rf.com/photo_115733992_a-racing-pigeon-poses-in-front-of-the-lens-of-the-camera.html

Gabler, M. (n.d.). Unihemispheric Sleep and Serotonin. Retrieved from https://sites.google.com/site/wcubiol537neglectedstate/change-the-banner/molly-gabler-unihemispheric-sleep-and-serotonin

Gillies, M. J., Hyam, J. A., Weiss, A. R., Antoniades, C. A., Bogacz, R., Fitzgerald, J. J., . . . Green, A. L. (2017). The Cognitive Role of the Globus Pallidus interna; Insights from Disease States. Experimental Brain Research, 235(5), 1455-1465. doi:10.1007/s00221-017-4905-8

Güntürkün, O., Ströckens, F., & Ocklenburg, S. (2020). Brain Lateralization: A Comparative Perspective. Physiological Reviews, 100(3), 1019-1063. doi:10.1152/physrev.00006.2019

Husband, S. A., & Shimizu, T. (1999). Efferent projections of the ectostriatum in the pigeon (Columba livia). Journal of Comparative Neurology, 406(3), 329-345.

Jarvis, E. D., Güntürkün, O., Bruce, L., Csillag, A., Karten, H., Kuenzel, W., . . . Butler, A. B. (2005). Avian brains and a new understanding of vertebrate brain evolution. Nature Reviews Neuroscience, 6(2), 151-159. doi:10.1038/nrn1606

Kaouri, H. (2015). Half Sleeping Bird. In H. S. Bird.jpg (Ed.), (Vol. 2,215 × 1,578, pp. A young House Sparrow Passer domesticus exhibits Unihemispheric slow-wave sleep.). Wikimedia Commons. Retrieved from https://commons.wikimedia.org/wiki/File:Half_Sleeping_Bird.jpg

Lapierre, J. L., Kosenko, P. O., Lyamin, O. I., Kodama, T., Mukhametov, L. M., & Siegel, J. M. (2007). Cortical acetylcholine release is lateralized during asymmetrical slow-wave sleep in northern fur seals. The Journal of neuroscience : the official journal of the Society for Neuroscience, 27(44), 11999-12006. doi:10.1523/JNEUROSCI.2968-07.2007

Mascetti, G. G. (2016). Unihemispheric sleep and asymmetrical sleep: behavioral, neurophysiological, and functional perspectives. Nature and science of sleep, 8, 221-238. doi:10.2147/NSS.S71970

Ornithology, C. L. o. (n.d.). Magnificent Frigatebird Identification. Retrieved from https://www.allaboutbirds.org/guide/Magnificent_Frigatebird/id

Pan, S.-T., Kuo, C.-E., Zeng, J.-H., & Liang, S.-F. (2012). A transition-constrained discrete hidden Markov model for automatic sleep staging. Biomedical Engineering Online, 11(1), 52. doi:10.1186/1475-925X-11-52

Peeden, J. (2007). Sleeping ducks. In. Flickr. Retrieved from https://www.flickr.com/photos/jeffreypeeden/1794193417/in/album-72057594133670606/

Rattenborg, N. C. (2006). Do birds sleep in flight? Naturwissenschaften, 93(9), 413-425. doi:10.1007/s00114-006-0120-3

Rattenborg, N. C., van der Meij, J., Beckers, G. J. L., & Lesku, J. A. (2019). Local Aspects of Avian Non-REM and REM Sleep. Frontiers in Neuroscience, 13(567). doi:10.3389/fnins.2019.00567

Rattenborg, N. C., Voirin, B., Cruz, S. M., Tisdale, R., Dell’Omo, G., Lipp, H.-P., . . . Vyssotski, A. L. (2016). Evidence that birds sleep in mid-flight. Nature Communications, 7(1), 12468. doi:10.1038/ncomms12468

Ritchison, G. (n.d.). Avian Brain & Senses. Retrieved from http://people.eku.edu/ritchisong/birdbrain.html

Rogers, L. J., & Kaplan, G. (2019). Does Functional Lateralization in Birds Have any Implications for Their Welfare? Symmetry, 11(8), 1043. Retrieved from https://www.mdpi.com/2073-8994/11/8/1043