"This is a preprint of an article published in
Vol. 5, No. 4, 2000, pp. 34-43, copyright 2000."
"This preprint is posted for personal or professional use only but not for commercial sale or for any systematic external distribution by a third party (e.g., a listserve or database connected to a public access server)."
Larry S. Liebovitch and Daniela Scheurle
Larry S. Liebovitch earned a B.S. in physics, a Ph.D. in Astronomy, worked for 15 years in Ophthalmology Departments, and is now a Professor at Florida Atlantic University in Boca Raton, Florida with appointments there in the Center for Complex Systems and Brain Sciences, Center for Molecular Biology and Biotechnology, Department of Psychology, and Department of Biomedical Science. He has applied nonlinear methods to analyze molecular, cellular, physiological, and psychological systems. He is author of 75 articles and book chapters and presented 130 lectures throughout the world. He is the author of Fractals and Chaos Simplified for the Life Sciences (1998) and coauthor of Fractal Physiology (1994), both published by Oxford University Press.
Daniela Scheurle earned a B.S. and an M.A. in Biology, and a Ph.D. in Pharmaceutical Biology at the Friedrich Alexander University in Erlangen in Germany. After having worked as an Assistant Professor at the Department of Anatomy in Regensburg, Germany, she moved to Florida Atlantic University where she first joined the Center for Complex Systems and Brain Sciences as a postdoctoral fellow for two years and is now a postdoctoral fellow at the Center for Molecular Biology and Biotechnology.
We used to think that a good measurement is characterized by its mean and variance and that a good theory is characterized by its ability to predict the values measured in an experiment. The properties of nonlinear systems called fractals and chaos have now taught us that this isnt necessarily true. Data from fractal systems extend over many scales and so cannot be characterized by a single characteristic average number. Data from chaotic systems do not repeat the same time series of values, even if they are started very close to the same initial conditions. This means that a valid mathematical model will not be able to predict the values of the time series.
Erwin Chargaff, who discovered that the amounts of A (adenine) and T (thymine) are paired with the amounts of G (guanine) and C (cytosine) in DNA, wrote that it is not the goal of science to tear at the tapestry of the world and pull out the threads and determine the color of each thread. It is the goal of science to see the picture on the tapestry. Here, we would like to step back a bit from the tapestry of nonlinear systems and try to make clear two important lessons: 1) how we use statistics to analyze our experimental data and 2) how we judge the validity of our mathematical models. These lessons are often stated in technical terms, but here we want to make their dramatic implications for everyday science as clear as possible.
FRACTALS AND CHAOS
"Fractals" and "chaos" are two examples of nonlinear approaches to analyze and understand the properties of complex systems.
A fractal is a object in space that has an ever larger number of ever smaller pieces. It is self-similar, meaning that the smaller pieces are reduced copies of the larger pieces. For some fractals, the smaller pieces are exact copies of the larger pieces. For most fractals in nature, the smaller pieces are kind-of-like the larger pieces. A tree is such a fractal. It has an ever larger number of ever smaller branches. A fractal can also be a process in time. There can be an ever larger number of fluctuations of ever smaller amplitude. A fractal can also be a set of numbers from experimental data. There can be an ever larger number of ever smaller numbers.
The word chaos was chosen to describe another type of nonlinear system. It used to be thought that complicated results must be produced by very complicated systems. Chaos means that some nonlinear, but quite simple systems, can produce very complicated results. These systems have the surprising property that we can completely predict the values of the system over brief times, but we are unable to predict their values over long times.
We will see how fractals and chaos lead to two important lessons for how we handle data and models in science.
STATISTICS: A LIVING NOT A DEAD SCIENCE
In the history of science, each new way of analyzing data has led to new insights about the natural world. Many people think statistics is pretty cut and dry. It is certainly taught that way in college statistics courses. In fact, over the last four hundred years, statistics has changed quite a lot. These changes have altered how we analyze data and even how we perform experiments. Fractals are the latest development in statistics. An appreciation of the properties of fractals is changing the most basic ways we analyze and interpret data from experiments and is leading to new insights into understanding physical, chemical, biological, psychological, and social systems.
WHY WE MEASURE MORE THAN ONCE
Presently, the most commonly used statistical procedure is to repeat an experiment a number of times and then take the average of the results. We are so used to doing this that we do this automatically. In fact, this procedure is only about two hundred years old. Before this, scientists did an experiment only once, as accurately as possible, and got the "right" answer. They would think were crazy for repeating the same experiment many times.
Our present day methods of handling experimental data have their roots about four hundred years ago. At that time scientists began to calculate the odds in gambling games. From those studies emerged the theory of probability and subsequently the theory of statistics. These new statistical ideas suggested a different and more powerful experimental approach. The basic idea was that in some experiments random errors would make the value measured a bit higher and in other experiments random errors would make the value measured a bit lower. Combining these values by computing the average of the different experimental results would make the errors cancel and the average would be closer to the "right" value than the result of any one experiment.
It doesnt seem to be recognized today that the average is a useful and more accurate way to characterize the data only if the data have some very specific mathematical properties. Namely, the variation in the measurements must be due to the fact that there are a large number of small, independent, random error sources. This is sometimes illustrated by an exhibit in a science museum consisting of a vertical triangular array of pins sandwiched in a thin glass case. Small steel balls are dropped on the apex of this triangle. At each pin the ball has equal probability of falling to the right or the left of the pin. Each row of pins deflects the ball a small amount. To reach the bottom at the far left or far right the ball must suffer all left or all right deflections at all the pins. Since that is pretty unlikely, most of the balls at the bottom wind up near the center and much less wind up further from the center. The deflection at each pin is random and independent of the deflection at every other pin. There is equal probability at each pin that the ball is deflected to the left or the right. Thus, there are a large number of small, independent, random error sources. When many balls have fallen through the pins they accumulate at the bottom. They form vertical rows. The height of these rows as a function of their horizontal distance forms a Binomial distribution. (The number of ways of reaching each horizontal position k at the bottom of the n rows of pins is the number of combinations of n things taken k at a time.) When there are lots of balls and lots of pins, this distribution is well approximated by the Gaussian distribution which is also called the bell curve and the normal distribution.
THE NORMAL DISTRIBUTION AND THE REAL WORLD
This normal distribution is the paradigm, the view of the world, that has molded how we analyze experimental data over the last two hundred years. As shown in Figure 1 we have come to think of data as having values most of which are near an average value, with a few values that are smaller, and a few that are larger. The probability density function, PDF(x), is the probability that any measurement has a value between x and x + dx. We suppose that the PDF of the data has a normal distribution. What an interestingly loaded way to describe a scientific hypothesis, the "normal" distribution.
Figure 1. The values of a set of data are represented by the diameters of these circles. The probability density function, PDF(x), is the probability that any measurement has a value between x and x + dx. Top: We are used to expecting that the data have a PDF that is a "normal" distribution. That is, most of the values of the data are near an average value with some a bit smaller and some a bit larger. Bottom: However, much of the data from the natural world consists of an ever larger number of ever smaller values. The PDF of such data has a fractal distribution which is a straight line on a plot of Log[PDF(x)] versus Log(x).
In fact, much of nature is definitely not "normal." It consists of objects having an ever larger number of ever smaller pieces. There is no single number, such as an average, that can adequately characterize such objects. A tree has an ever larger number of ever smaller branches. A mountain range has an ever larger number of ever smaller hills. An archipelago has ever larger number of ever smaller islands. There is no meaning to the average diameter of these branches, the average height of these hills, or the average area of these islands. Such objects are called "fractals." As shown in Figure 1, the data from a fractal object in space or a fractal process in time consists of a few large values, many medium values, and a huge number of small values. There is no single number, such as an average, that adequately characterizes such data. The PDF is not normal. Typically, it is a straight line on a plot of Log [PDF(x)] versus Log(x), indicating that is has the form Ax-a, which is called a "power law."
WHEN THE MEAN IS MEANINGLESS
The mean of a sample of data is called the sample mean. When the PDF of the data has a normal distribution, then as we collect more data, the sample means approach a limiting value that we identify as the "right" value of the average, called the population mean. This is shown in Figure 2.
Figure 2. Top: For data with a "normal" distribution of values, as more data are collected, the means of those samples approach a limiting value that we identify as the "right" value of the data, called the population mean. Bottom: For data with a fractal distribution of values, as more data are collected, the means of those samples continue to either increase or decrease. They do not approach a finite, nonzero, limiting value. There is no unique value that characterizes the data. The population mean is not defined.
However when the PDF has a power law fractal form, the averages measured depend on the amount of data analyzed. The sample means do not approach a limiting value as more data are collected. As shown in Figure 2, the averages measured will either increase or decrease with the amount of data analyzed. There is no single value that we can identify as the "right" value of the average. Therefore, the population mean does not exist.
Whether the sample means increase or decrease depends on the relative number of small values compared to the large values in the data. If many small values are included as more data are analyzed, then the average will decrease. If there are a few very big values that are included as more data are analyzed, then the average will increase. Which one of these two cases will happen depends on the relative number of small values in the data compared to the large values. That is characterized by the parameter called the "fractal dimension." For fractal objects the fractal dimension, d, describes the number of new pieces, N, of an object that are found when the object is viewed at a finer resolution r, namely, d = Log(N) / Log(1/r). For fractal data, the fractal dimension quantifies the relative number of small values compared to the large values. It is related to the slope a of the power law form of the PDF.
The statistical methods that assume that the PDF of the data has a normal distribution provide meaningful measures, namely the mean and variance, to characterize that type of data. However, when those methods are applied to data that have a fractal rather than a normal distribution, the results are not meaningful. For a fractal distribution both the mean and variance will depend on the amount of data analyzed. We need to use appropriate fractal measures, such as the fractal dimension, to characterize fractal data in a meaningful way.
The failure of the mean and variance to characterize fractal data is is a radical lesson for those who think that the mean and variance can always be determined experimentally and always have a useful meaning. Interestingly, these statistical properties of fractals have been known for over three hundred years by mathematicians who specialized in this type of statistics. But these properties did not become woven into the fabric of statistical theory that became popular in the natural sciences. It is only now, with the rise of interest in fractals, that this previously obscure mathematics is being rediscovered and its implications for the analysis and interpretation of experimental data is being appreciated.
TWO GAMES OF CHANCE
Two coin toss games illustrate the different properties of normal and fractal distributions of data. First, consider a normal coin toss game. Toss a coin. If it lands TAILS you win nothing. If it lands HEADS you win $1. On average, for each toss, you will win the sum of the probability of each outcome multiplied by the payoff for that outcome, which is equal to (1/2)x($0) + (1/2)x($1) = $0.50. So, if you go to a fair casino, they should say to you, "Please put up $0.50 each time to play this game." This also seems fair to you too because 1/2 of the time you will win nothing and 1/2 of the time you will win $1.
Now consider a three hundred year old fractal coin toss game proposed by Niklaus Bernoulli in Russia and published by his cousin Daniel Bernoulli in Germany. In this game you toss a coin until it lands HEADS. If it lands HEADS on the first toss you win $2. If it lands TAILS on the first toss and then HEADS on the second toss you win $4. If it lands TAILS on the first toss, TAILS on the second toss, and then HEADS on the third toss, you win $8, and so on. The probability of having a HEAD on the N-th toss is 2-N. The payoff for that outcome is $2N. On average, for each toss, you will win the sum of the probability of each outcome multiplied by the payoff for that outcome, which is equal to (1/2)x($2) + (1/4)x($4) + (1/8)x($8) + ... = 1 + 1 + 1 ... = . The average winnings per game does not approach a finite, limiting value. Because Niklaus lived in St. Petersburg, this became known as the St. Petersburg paradox. It's a paradox because you argue, correctly, that half of the time you win $2. Therefore, the median winnings of the game is $2, so it is fair for you to wager twice that amount, $4, on each game. But the casino argues, equally correctly, that the mean winnings per game is infinite, so that the casino asks you to put up more than all the money in the universe to play even one game.
The PDF(x), how often you would win between x and x+ dx amount of money per game for the normal coin toss game is shown in Figure 3. It is a normal distribution. The average winnings, which is equal to $0.50, is a good value that characterizes the distribution of values. Sometimes the winnings per game are a bit less than $0.50 and sometimes a bit more. But, most of the time, after you have played many games, the winnings are pretty close to $0.50 per game.
Figure 3. Probability density function, PDF(x), how often you would win between x and x+ dx amount of money per game for two types of coin toss games. Top: The PDF for the normal coin toss game is a normal distribution. The PDF shown here is for 100 plays of the game. Bottom: PDF for the fractal St. Petersburg coin toss game is power law, namely a straight line on a plot of Log[PDF(x)] versus Log(x).
The PDF for the fractal coin toss game is also shown in Figure 3. It is a power law distribution of the form Ax-a, which is a straight line on a plot of Log[PDF(x)] versus Log(x). Most of the time the winnings are small. There is a high probability of winning a small amount of money. Sometimes, you get a few TAILS before that first HEAD and so you win much more money, because you win $2 raised to the number of TAILS plus one. Therefore, there is a medium probability of winning a large amount of money. Very infrequently you get a long sequence of TAILS and so you win a huge jackpot. Therefore, there is a very low probability of winning a huge amount of money. These frequent small values, moderately often medium values, and infrequent large values are analogous to the many tiny pieces, some medium sized pieces, and the few large pieces in a fractal object. There is no single average value that is the characteristic value of the winnings per game. The winnings spans a large range of values.
Figure 4 shows what happened when we both games played 10 million times, using the random number on a computer to toss the coins. First we played the normal coin toss game. For a while, we were lucky, our winnings per game were greater than $0.50. But as time went on, our luck ran out. Our average winnings per game rapidly approached exactly $0.50. It behaved just like the statistics they taught you in college, as more data were included the sample means approached the population mean. Then we played the fractal coin toss game. The more we played, the more our winnings per game kept increasing. Every once in a while we got a string of TAILS before a HEAD and therefore we won a huge jackpot. That huge jackpot increased the average winnings per game. The more we played, the ever bigger were those ever rarer jackpots and the ever more they increased the winnings per game computed over all the games. There was no defined average winnings. The average depended on how many games we had played. The sample means kept increasing. They did not converge to a finite limiting value. There was no population mean. There was no single average value that characterized the winnings per game.
Figure 4. The normal and fractal St. Petersburg coin toss games were each played for 10 million games. The average winnings per game is plotted versus the number of games played. Top: As more games were played, the average winnings per game of the normal coin toss game approached a limiting value. The "right" value of the average winnings per game was $0.50. Bottom: As more games were played, the average winnings per game of the fractal St. Petersburg coin toss game continued to increase without bound. There is no single average value that characterized the winnings per game.
LOTS OF NATURE ISN'T NORMAL
Until recently, it has been assumed that the things that we measure in the natural world have PDFs that have a normal distribution and therefore can be meaningfully characterized by their mean and variance. However, many things in the natural world consist of an ever larger number of ever smaller pieces and so have a fractal PDF that cannot be meaningfully described by a mean and variance.
For example, we have studied the timing of heart attacks. An episode of very rapid heart rate can lead to a loss of the organized pattern of contraction of the heart and its ability to pump blood. To prevent such an episode from leading to death, a small computer, called a "cardioverter defibrillator" can be implanted in the chest. If the heart beats too fast this device produces an electrical shock to kick the heart back into a slower, safer rhythm. Since it is a small computer, it can record the times when it was triggered and this record can be played back with a radio transceiver when the person returns to the hospital.
We analyzed the times between the events of rapid heart rate that triggered the device . As shown in Figure 5, the PDF of the time, t, between these events is a straight line on a plot of Log[PDF(t)] versus Log(t), indicating that it has a fractal, power law form. These times ranged from seconds to weeks. Most of the time it was a short time between these events. Less often it was longer. Very infrequently it was very long. There is no average time defined between these events. The average measured from each sample of data depends on the length of time covered by the sample. The number of events per day will be different if it is measured over one day, or one week, or one month, or one year.
Figure 5. The PDF of the times between episodes of the onset of rapid heart rate measured in patients with implanted cardioverter-defibrillators from the work of Liebovitch et al. . Most often the time between episodes is brief. Less often the time is longer. Infrequently it is very long. There is no single average time that characterizes the times between these events. The PDF has a power law form that is a straight line on a plot of Log[PDF(t)] versus Log(t), similar to the PDF of the fractal St. Petersburg coin toss game in Figure 3.
The realization that these times are not normal and cannot be characterized by their mean and variance is an important advance over previous work which had tried, unsuccessfully, to find the average rate of these events. There will be many events close together when the times between the events are very short and very few events close together when the times between the events are very long. The apparent bunching of these events led some to conclude that there were "storms" of episodes of rapid heart rate . Our work showed that the entire range of different times between events, from very short to very long, are all part of the same PDF. This suggests that the same underlying biological mechanism, rather than different mechanisms, cause both the short times and the long times between events . Our work demonstrated that the mean and variance of the rate of these events are not meaningful measures of this data. A meaningful measure is the fractal dimension, which describes how much more often there are small times between the events than long times. The fractal dimension can be determined from the slope of the plot of Log[PDF(t)] versus Log(t). We hope that using the fractal dimension to characterize these data will lead to better ways of diagnosing heart disease and measuring the effectiveness of different drugs used to treat it.
We often analyze experimental data with the preconceived bias that events in nature should occur in a uniform way. Even when events occur at random, they are often bunched together. And the bunches have bunches which have bunches. Many events in nature occur in this type of fractal pattern. The failure to understand that nature often has this pattern means that we may misplace our attention on the high frequency events rather than the overall pattern of how the number of events depends on the time between the events. That is, we may strive to find the non-existant average of the high frequency events rather than the fractal dimension of the entire pattern of events in an effort to give us clues to the mechanisms that generated the events.
Fractals are common in nature. "Clouds are not spheres. Mountains are not cones. Lightning does not travel in a straight line." . Perhaps we think the computer generated pictures of fractals are beautiful because they strike a chord in us that resonates with our experience with natural things. A number of recent books describe fractals in the physical , biomedical [6-9], and social sciences [10,11]. In biomedical systems alone, for example, fractal objects in space include: the alternations of purines and pyrimidines in the nucleotide sequence that encodes genetic information in DNA, the areas of genetically similar cells in genetic mosaics, the blood vessels of the circulatory system, the blood flow in the muscle of the heart, the diameter of airways in the lung, the dendrites of neurons, the edges of growth of bacterial colonies, the surfaces of proteins, the surfaces of cell membranes, the spaces between the cells that line the lung, and the texture of bone. Fractal processes in time include: the timing of action potentials in nerve cells, the durations of consecutive breaths, the chemical kinetics of ion channel proteins in the cell membrane, the electrical activity of the heartbeat, the reaction rates of biochemical reactions limited by diffusion, the vibrational energy levels in proteins, the voltage fluctuations across the membrane of T-lymphocytes, the volumes of consecutive breaths, and the washout kinetics of substances from the blood.
LESSON #1: THE MEAN AND VARIANCE ARE NOT ALWAYS A MEANINGFUL WAY TO CHARACTERIZE DATA
Articles in scientific journals and presentations at scientific meetings today almost exclusively characterize data by a mean ± variance. When the PDF is normal, then the mean and variance are meaningful measures. But, when the PDF is fractal, then there is no single average value that characterizes the data and the mean and variance are not meaningful. In that case, the fractal dimension is the meaningful measure. It describes the relative number of small values compared to the large values. Since there are many fractals in physical, chemical, biological, psychological, and social systems it means that the data from many natural systems cannot be meaningfully characterized by a mean and a variance. We are about to witness a dramatic change in our most basic tools of descriptive statistics which will change how we analyze and interpret experimental data.
Weve seen what weve learned from fractals, now lets see what we can learn from chaos.
THE CLOCKWORK UNIVERSE
About 350 years ago ideas from science, technology, and philosophy lead to a new view of nature. At that time, Newtons theory of gravitation had successfully explained the motions of the planets around the sky, the tides that wash up on the shore, and the shape of the earth itself. Increasingly sophisticated technology had made it possible to construct impressive mechanical devices. The Enlightenment proposed that we had the knowledge and daring to understand nature. The view that emerged was that the universe itself was a machine, perhaps like a gigantic, complex clock. If we could understand the motions and connections between all its wheels and gears we could predict their future positions throughout all of time.
Prediction became an important touchstone for scientific theories. If a theory could predict the values later measured in an experiment, then the mechanisms modeled by the mathematics of the theory should be the "right" explanation of what was happening in the experiment.
It makes perfect sense that if we completely understand how a machine works that we can predict how it will behave in the future. For example, consider the machine that is the solar system, the earth and other planets that revolve around the sun. If we understand how force of gravity depends on mass and distance, and we know the masses and the starting positions and velocities of all the planets, then we should be able to predict their positions for all eternity. Can we really do that? To mark his 60th Birthday on January 21, 1889, Oscar II, King of Sweden and Norway, proposed to give a prize to the best mathematical essay on this question. Of the 12 essays received, the winning essay was submitted by the French mathematician Henri Poincaré . He showed that if we have three bodies in space, even though we know the law of gravity, and the conditions under which the three bodies start, we still cannot necessarily predict the future locations of the three bodies. It is as if we have examined all the gears in our great grandfather clock in the parlor and so we know how each one works and how each is connected to all the others. We set the time on the face of the clock. But we still cant correctly predict the time that will be on the face of the clock the next day. About 70 years later this behavior was given the name "chaos."
Chaos is a bit of a confusing choice of words for this behavior. In normal usage, "chaos" means disordered, without form. Here "chaos" means a simple, nonlinear mathematical system of equations that is highly ordered. It is a deterministic dynamical system, meaning that the variables at one point in time can be computed from their values a little earlier in time. Yet, in the long run, the values of the variables cannot be computed from their values a long time ago. The word "chaos" was chosen to emphasize this unpredictability, not to indicate that the system of equations lacks order.
EXQUISITE SENSITIVITY TO INITIAL CONDITIONS
When we start an experiment with pretty much the same experimental conditions, we are used to thinking that we should get pretty much the same experimental results. Lets say we take a dish of cultured cells from our incubator on Tuesday and put about a 10% solution of a chemical on them and they turn blue. We expect that if we take the adjacent dish of cells from our incubator on Wednesday and put about a 10% solution of the same chemical on them they should also turn blue. This intuition is based on our experience with linear systems. For linear systems small changes in the initial conditions produce equivalently small changes in the final output. Why is our intuition based on such linear systems? Since we could only solve the mathematics of linear equations we used them to represent all of nature. But much of nature is not linear. Small changes in how we start the experiment can result in large changes in the results. If the cells behave chaotically, then the cells on Wednesday could turn neon green instead of blue in response to the chemical.
About 40 years ago Lorenz developed a very simplified model of the motion of air in the atmosphere that displays this "sensitivity to initial conditions" . The air is heated from below and cooled from the top. Hot air rises and cold air falls. The air moves in rotating cylinders which bring hot air up on one side and cold air down on the other side. This motion of the air mixes hot and cold air reducing the temperature difference driving the motion of the air. Yet, the air is still being heated from below and cooled from the top. So, when the cylinders of air slow down to a complete stop, then they start rotating again. But sometimes they start rotating in the opposite direction. That is, if they were originally rotating clockwise then they switch to rotating counter-clockwise. As time goes on, the rotation of the cylinders of air speeds up and slows down, and it keeps changing between rotating clockwise and counter-clockwise. Lorenz modeled the motion of the air by three variables in three coupled equations. One variable, that he called X, is the rate of angular rotation of the cylinders of air. When X > 0, the cylinders are rotating clockwise and when X < 0, the cylinders are rotating counter-clockwise.
Figure 6 shows the values of X as a function of time, t, for two computations of the motion of the air in the model. In the top computation the air was started with the initial value of X=1. In the bottom computation the air was started with X=1.0001, not very different. The values of X a short time after the start are pretty similar for both computations. But after a while the values of X of the first computation are radically different from those in the second computation. At the same time that the value of X in the first computation is greater than zero and the air is rotating clockwise, the value of X in the second computation is less than zero and the air is rotating counter-clockwise, in the opposite direction. Our intuition based on linear systems does not apply to this nonlinear, chaotic system. We can never start two experiments exactly the same. We are used to thinking that if we start two experiments almost the same that the results of both experiments will be the same. Here, even though we start both computations almost the same, after a while, the results are very different.
Figure 6. Computation of the time series of the rate of rotation, X, of cylinders of air in the Lorenz model of the atmosphere as a function of time, t. Top: The computation was started with an initial value of X=1. Bottom: The computation was started with an initial value of X=1.0001. Even though both computations were started with almost the same initial value, after a while, the time series are markedly different. This behavior is called "sensitivity to initial conditions." Since an experiment cannot be started with exactly the same initial values, an experimental system that is sensitive to initial conditions will produce a different time series every time the experiment is repeated.
THE "REAL" THINGS ARE THE INVARIANTS
The concept of an "invariant" is an important one in physics. We can explain this concept by the following example. A pencil rests flat on a horizontal plane. The first time that you look you see that the eraser of the pencil is at the origin of the x-y axis and that the point of the pencil is at the coordinates x=3 and y=4. The next time that you look you see that the eraser of the pencil is still at the origin but the coordinates of the point of the pencil are now x=4 and y=3. It could be that the length in x and the length in y of the pencil has changed. But, more likely, you probably think the length of the pencil is the same, but that it has rotated a bit. The length, L, of the pencil is equal to (x2 + y2)1/2. Even though the values of x and y change, the length remains the same. Initially L = (32 + 42)1/2 = 5 and later L = (42 + 32)1/2= 5. Even though the measurements of x and y change, the "real" thing, the length, remains the same. These "real" things are called "invariants." Sometimes we can measure the value of the invariant directly in an experiment. Sometimes we can only determine the invariant from combinations of measurements from the experiment.
About two hundred years ago people began doing detailed measurements of electricity and magnetism. The results were a bit strange because moving electric charges produced magnetic fields and moving magnets produced electrical currents. The amounts of electricity and magnetism that converted into each other were fully described by mathematical equations derived by James Clerk Maxwell about 135 years ago. But even though the equations described the amounts it didnt make any sense. How can one thing magically change into a different amount of another thing? Then, about 95 years ago, Albert Einstein showed that there was a "real" thing, an invariant, the electromagnetic tensor. Under some conditions this "real" thing appears as electricity and under other conditions it appears as magnetism. (Specifically, Einsteins Special Theory of Relativity showed how constant speed changes the electric and magnetic components of the electromagnetic tensor.)
The history of science is basically the search for these "real" things, the invariants. The not so real things look different under different conditions, but the "real" things are invariant when we know how to look at them.
The lesson here is that for chaotic systems the time series is not an invariant. Our intuitive notion that we should get the same results if we rerun the same experiment isnt true. We can never restart the second experiment with exactly the same values of all the conditions. Because of the sensitivity to initial conditions, this means that the results of the first experiment and the second experiment can be completely different.
We are used to thinking that the test of a mathematical model is to predict the outcome of an experiment. If the model does this then we believe that the mechanisms modeled by the mathematics are the "right" explanation of what happened in the experiment. But this does not work for chaotic systems. We cannot determine which is the best mathematical model by comparing their predictions to the results measured at different times from an experiment.
The values measured at different times in an experiment are called a "time series." The time series of a chaotic system is sensitive to the initial conditions. Each time the experiment is run, the time series will be different. The time series is not the "real" thing, it is not the invariant.
A common and powerful trick in mathematics to solve a hard problem is to transform it to an equivalent easy problem and then solve the easy problem. In his pioneering work a century ago, Poincaré showed that it is very hard to understand some properties of systems by studying their time series. So he used this trick. He showed how the values of a times series can be transformed into an object. The topological properties of this object, which were easier to figure out, would reveal properties of the time series that were harder to figure out.
This is quite a change in perspective. We are so used to measuring data from experiments that we think that the time series of the measurements is the "real" thing. For chaotic systems, the time series is not the "real" thing. The "real" thing, the invariant, is the attractor. The attractor is found by transforming the time series into what is called a "phase space." It is the attractor, not the time series that is reproducible from experiment to experiment. It is the attractor, not the time series, that gives us the information about the mechanisms that are the "right" explanation of what happened in the experiment.
Figure 7 shows the time series from two computations of the Lorenz model of the atmosphere. Since this is a chaotic system each time series is different. Each of these time series, X(t), can be transformed into an object in a phase space. Take the first two values of the time series. Make believe that the first value, X(t1), is the value of a x-coordinate and that the second value, X(t2), is the value of a y-coordinate, and plot a point at that position: (X(t1),X(t2)). Then make believe that the second value, X(t2), is the value of a x-coordinate and that the third value, X(t3), is the value of a y-coordinate, and plot a point at that position: (X(t2),X(t3)). And so on. This is equivalent to making a plot of X(t) versus X(t+(delta)t). Although each time series in Figure 7 is different, they just form overlapping parts of the same attractor in the phase space in Figure 8. The time series is different in each experiment. The attractor is the same in each experiment. The attractor, not the time series, is the invariant, the "real" thing.
Figure 7. Two computations of the time series of the rate of rotation, X, of cylinders of air in the Lorenz model of the atmosphere as a function of time, t. The computations were started at different initial values and each time series is different. The time series is not the "real" thing, the invariant of the system.
Figure 8. The two time series (+ and 6) from Figure 7 were each transformed into a phase space by plotting X(t+(delta)t) versus X(t). Even though the time series from Figure 7 are quite different, those two time series actually form overlapping parts of the same attractor. The attractor is the "real" thing, the invariant of the system.
LESSON #2: THE TIME SERIES OF DATA FROM AN EXPERIMENT IS NOT NECESSARILY THE "REAL" THING
Articles in scientific journals and presentations at scientific meetings today almost exclusively characterize data that is changing in time by a time series. When a system is not chaotic, then the time series is a meaningful measure. But when a system is chaotic, then the time series measured from each experiment will be different. The time series is then not the "real" property, not the invariant of the system. The meaningful measures are found from the properties of the attractor that can generated from a transformation of the time series. Even though the time series will be different from each experiment, the attractor will be the same from each experiment. Since there can be chaotic systems in physical, chemical, biological, psychological, and social systems this means that the data from many natural systems may not be meaningfully characterized by their time series. We are used to trying to find mathematical models that most accurately predict the time series from experimental data in order to discover the mechanisms that are the "right" explanation of what was happening in the experiment. For chaotic systems, we need to find the mathematical model that most accurately predicts the attractor in order to discover the mechanisms that are the "right" explanation.
Much work in science involves the discovery of ever more arcane and complex facts. It is therefore easy to get the impression that progress in science depends on discovering the ever finer details of the world. This is a false impression. The most important progress in science involves changes in the basic way we see the world. These are changes in the simplest things that we thought were so obvious that they required no assumptions at all. Yet, it turns out that we realize that these simplest things did depend on assumptions, and that those assumptions were false.
We presented two lessons that change our view about two of the simplest, basic, most naive concepts that we take for granted in science. First, we have shown how the mean and variance are not necessarily meaningful measures to characterize experimental data. When that is the case, the meaningful way to characterize data is to use the fractal dimension, which describes the relative number of smaller values compared to larger values in the data. Second, we have shown that a time series from an experiment is not necessarily reproducible and therefore that predictability of the time series is not necessarily a meaningful way to test the validity of mathematical models. When this is the case, then the attractor in the phase space is reproducible and the topological properties of the attractor can be used to test the validity of mathematical models.
For such a long time we thought that most data must have a normal distribution and therefore that the mean is meaningful. With the perfect vision of hindsight, this is a bit odd. Much of the world around us is not normal. It consists of fractals, for example, mountain ranges, river basins, and broccoli. These fractals have an ever larger number of ever smaller pieces. Yet, we insisted on modeling many things as if all of their pieces had only one size. Similarly, it is also a bit odd that we thought for so long that if we knew the rules of the world that we could completely predict its future. The endless variety of chess games, the patterns of snowflakes, and the complexity of human relationships might have suggested something different to us.
The point is not that we have been so wrong for so long. The point is that it is so difficult to see the simplest things as they really are. We become so used to our assumptions that we can no longer see them or evidence against them. Instead of challenging our assumptions we spend our time in studying the details, the colors of the threads that we tear from the tapestry of the world. That is why science is hard.
Science is hard because it is hardest to see what is most obvious.
1. Liebovitch, L.S.; Todorov, A.T.; Zochowski, M; Scheurle, D.; Colgin, L.; Wood, M.A.; Ellenbogen, K.A.; Herre, J.M.; Bernstein, R.C. 1999. Nonlinear properties of cardiac rhythm abnormalities, Physical Review E59: 3312-3319.
2. Credner, S.C.; Klingenheben, T.; Mauss, O.; Sticherling, C.; Hohnloser, S.H. 1998. Electrical storm in patients with transvenous implantable cardioverter-defibrillators, Journal of the American College of Cardiology 32:1909-1915.
3. Wood, M.A.: Ellenbogen, K.A.; Liebovitch, L.S. 1999. Letters to the editor: Electrical storm in patients with transvenous implantable cardioverter-defibrillators, Journal of the American College of Cardiology 34:950-951.
4. Gleick, J. 1985. The man who reshaped geometry, New York Times Sunday Magazine Dec. 8, p. 64 et seq.
5. Bunde, A.; Havlin, S. eds. 1994. Fractals in Science. New York: Springer-Verlag.
6. Liebovitch, L.S. 1998. Fractals and Chaos Simplified for the Life Sciences. New York: Oxford University Press.
7. Bassingthwaighte, J.B.; Liebovitch, L.S.; West, B.J. 1994. Fractal Physiology. New York: Oxford University Press.
8. Iannaccone, P.M.; Khokha, M. eds. 1995. Fractal Geometry in Biological Systems. Boca Raton FL: CRC Press.
9. Dewey, T.G. 1997. Fractals in Molecular Biophysics. New York: Oxford University Press.
10. Batty, M.; Longley, P. 1994. Fractal Cities. New York: Academic Press.
11. Peters, E.E. 1994. Fractal Market Analysis. New York: John Wiley & Sons.
12. Barrow-Green, J. 1997. Poincaré and the Three Body Problem. Providence RI: American Mathematical Society.
13. Lorenz, E.N. 1963. Deterministic nonperiodic flow, Journal of the Atmospheric Sciences 20: 130-141.