(This is an excerpt from my master thesis: Please cite as; S.-S. Poil, Temporal correlations and criticality in models of neuronal networks, M.sc. thesis, University of Copenhagen & VU University Amsterdam, 2007 and/or link to http://www.poil.dk/s/self-organized-criticality-soc/55 )

See part 1 of “Introduction to self-organized criticality” here

Criticality

The term criticality comes from statistical physics, but has been adapted to other fields. Most sand pile models show criticality by a scale-invariant probability distribution of avalanche sizes [Bak, 1997]. A general SOC medium is in a critical state if an energy-dependent observable has a scale-invariant probability distribution (within the limits of finite-size effects), this would, e.g., be the avalanche size of topplings in a sand pile model, or the size of a forest fire in a forest fire model [Bak, 1997; Christensen and Moloney, 2005]. It is difficult to give an example of an energy-independent observable, but it is any observable that does not change owing to the flow of energy. Criticality is only defined from energy-dependent observables, because in SOC systems the critical state should only be dependent on the existence of a flow of energy through the system, if not, the system is considered to be fine-tuned [Bak, 1997; Flyvbjerg, 1996a].

What is scale-invariance?

A scale-invariant or scale-free probability distribution is most often a power-law distribution;

Where s is the observable, β is the power-law exponent or scaling exponent. A power-law distribution is scale-invariant, because, as the word says, no scale is important – no feature exists at one scale that makes that scale stand out. We also say the power-law characterize a fractal or self-similar pattern. This is seen if we consider the two sizes s1 and s2, and find their relative probability

if we then consider another scale, i.e., we multiply the two sizes with a constant k, and see the relative probability did not change

We see that independent on which scale we look, we find patterns having the same quantitative relations. This is for example seen with Koch curves or the coast of Norway, where only finite-size effects limits the conclusion [Bak, 1997; Mandelbrot, 1983]. If we consider a scale-free time signal – a 1/f-noise signal – we see a similar pattern. If we find the power spectral density PSD at one frequency f, we will at the double frequency 2f find the PSD has decreased with a factor two. If we continue, and double this frequency to 4f, the PSD has decreased with yet a factor two. Thus, we can zoom in on our time signal and find fluctuations on all time scales (where it is scale-invariant) with an equal pattern [Linkenkaer-Hansen, 2002].

Characteristics for SOC systems

SOC systems are usually slowly driven steady-state non-equilibrium systems. This leaves us with a huge set of possible systems, because most systems in nature are non-equilibrium systems and often steady-state. Equilibrium systems are the good old theoretical systems, which we meet in the textbooks (e.g., the Ising model). For these systems, we can calculate any thermodynamic property from the free energy. Many processes in nature, however, relax slowly to equilibrium, – if ever – and only allow estimates to be found of, e.g., differences in free energy using Jarzynski’s equality [Jarzynski, 1997]. Moreover, non-equilibrium systems often react unpredictably and are, thus, not easy to ’fine-tune’ to criticality. The term steady-state tells that the in-flux and average out-flux of energy is equal. Energy, is here a wide term defined as being the conserved ’unit of potential’ for causing disturbance in the medium. The timescale of the steady-state is defined to be long, compared with the timescale of the in-flux energy. The term non-equilibrium means the system at the boundaries for energy flow is not in equilibrium — and the energy flow, thus, causes an increase in entropy. This also means the response to an external perturbation is not always linear, as it would be in an equilibrium system [Christensen and Moloney, 2005].

SOC systems often have the following characteristics:

• Local interaction among elements in the system cause the dynamics.
• Activity propagates in avalanches, i.e., energy is dissipated intermittently
• Insensitivity to initial conditions
• High susceptibility to perturbations
• Power-law scaling in the temporal dynamics and spatial structure (Spatial and temporal correlations), i.e., no characteristics scales exist (only finite-size effects).
• Many meta-stable states
• Intrinsic memory or thresholds exists in the medium
• Old, i.e., the system has evolved for a long-time relative to its size
• The system is robust to local failures or random perturbations

Systems that show self-organized criticality will most often consist of local units that interact. These interactions will spread in avalanches of activity on all scales with an upper and lower cut-off depending on the typical scales in the systems (e.g., the system size). Owing to the dynamics of a critical state, self-organized systems have a high susceptibility to perturbations. The scaling statistics are, however, robust to random perturbations or local failures, because the critical state is an attractor. This argument also applies to another characteristic, namely the insensitivity to initial conditions. This robustness is, as we will see, an important property of self-organized criticality systems.

Self-organization in nature – the nature of self-organization

We should understand from the driving force definition that, e.g., the critical state of the Ising model is not SOC. Because this state only appears if the system is fine-tuned into it, i.e., the physicist is setting the temperature. In nature, however, it is difficult to see how systems could be ’fine-tuned’. From a philosophical view, we could claim that all biological systems have been ’fine-tuned’ through evolution, and the specific environment they live in. A distinction between reorganization in the system owing to external forces that push the system in a certain direction, and self-organization that is only dependent on the interaction among elements in the system is essential to make.(see Figure 2).

Sketch of the difference between external and internal driving. Sketch of the difference between (A) external driving forcing the system (the circle) in a certain direction, and (B) internal ’interaction’ among elements that push the system to a critical state.

Biological systems organize because of internal interactions among elements in the systems, but on the long-time scales, they have evolved because of influence from an interaction with the environment. This influence changes the local rules that govern the interaction among the elements. As an example, we can consider the neuronal networks in the brain; the local rules are the mechanisms that support the interactions among neurons, these rules (supported by the expression level of different genes) do not change on the short time scales (seconds, minutes) – but are dependent on specific external influence on the medium-long time scales (days, generations). This influence is equal to a biased driving force, and it cannot be considered SOC – because if the influence had been different, the mechanisms would have evolved in another direction. Nevertheless, it seems that organization to criticality and scale-invariance is ubiquitous in nature [Buchanan, 2000]. Or as Chialvo [2006] state it (on the development of neuronal networks as sensors),

An animal in a subcritical world would not make any use of sensors with large dynamic range, because everything would be equally steady and frozen; indeed, it is hard to conceive of life itself existing under such conditions. Going to the other extreme, sensors would be useless in a supercritical world, because things would be changing too fast all the time. Instead, in the real world, senses are needed to gather information from a wide range of energies, where nothing is steady and uniform, where extreme events exist, and where probabilities often have long tails. (Reprinted by permission from Macmillan Publishers Ltd: Nature Physics Chialvo 2006, copyright 2006)

Indeed, it is the optimal character of the critical state that on the long timescale, is the ’fine-tuning’ of the organization towards criticality in biological systems. On the short timescales, it is internal interaction, and thus self-organized criticality, that attract the system to the critical state. Johnson [2001] has written an excellent book on the topic of emergence.

Separation of time scales forms criticality

From a physical view it is clear that a system that builds an internal structure should somehow export entropy to its surrounding [Haynie, 2001]. To export entropy, an SOC system needs a separation in the time scales of in-flux and out-flux of energy; i.e., the in-flux of energy should happen on a longer time-scales than the out-flux of energy. This idea might be understood using an analogy. Consider my collection of articles, each day I receive many new articles, but each day I also delete many articles; i.e., this is the in-flux and out-flux of energy to my system. Let us assume that articles are either good or bad, and that I of course only want good articles in my collection (this will make a steady-state impossible – i.e., my collection size is always increasing). We can, therefore, view the out-flux of energy to consist only of bad articles, and the in-flux to consist of both good and bad articles. It should be clear the way of keeping the collection ’in order’ is to have a slow rate of incoming articles and a fast rate of deleting bad articles (A fast-fast in- and out-flux is also good in this example, but it would not be good if the system was in a steady-state, because then no build-up would be form in system). If the opposite was used (i.e. fast-slow or slow-slow in- and out-flux), the collection will on average consist of a huge number of both good and bad articles, and no high order would be found. The flow of energy in an SOC system is dependent on local thresholds in the medium. The local threshold will store energy, and ensure the energy dissipation happens intermittently and not continuously. The driving should be slow. Where slow is defined to be, slow compared to the typical time of an energy avalanche propagating in the system. This just states the system should be allowed to evolve (i.e., relax to a state) owing to the local interactions in the system, and that this dynamics should not be disturbed by a fast driving force.

 References

P. Bak. How nature works. Oxford University Press, 1997.

M. Buchanan. Ubiquity. Weidenfeld and Nicolson, The Orion Publishing Group Ltd., 2000.

D. R. Chialvo. Are our senses critical. Nature Physics, 2:301–302, 2006.

K. Christensen and N. R. Moloney. Complexity and Criticality. Imperial College Press, 2005.

H. Flyvbjerg. Simplest possible self-organized critical system. Phys. Rev. Lett., 76:940–943, 1996a.

D. Haynie. Biological Thermodynamics. Cambridge University Press, 2001.

C. Jarzynski. Nonequilibrium equality for free energy differences. Phys. Rev. Lett., 78, 1997.

S. Johnson. Emergence. Scribner, 2001.

K. Linkenkaer-Hansen. Self-Organized Criticality and Stochastic Resonance in the human brain. PhD thesis, Helsinki University of Technology Laboratory of Biomedical Engineering, 2002.

B. Mandelbrot. The Fractal geometry of Nature. Freeman, 1983.