Showing posts with label systems. Show all posts
Showing posts with label systems. Show all posts

30 November, 2020

Dharma and Ergodicity

In the last couple of years, I have spent a lot of time trying to interpret the notion of dharma and traditional Indian worldview using systems theory. You can check out older posts with the label "Theory of Being" and also my book on Amazon called "Theory of Being". 

In this post, I put forth some more working ideas towards building a more comprehensive theory of systems using Indian thought. The ideas presented here are work in progress-- meant to provoke thought and get feedback. Ideas presented in this blog are subject to future revisions as I get more clarity. 

*~*~*~*~*~*

When we study systems we are first taught that a system is an ensemble of interacting parts. The interaction between the parts or the system dynamics, is what characterises a system-- without which, it would just be a collection of different parts, and not a system. 

Systems are everywhere-- in fact, a non-system is much harder to find and/or create, than a system. Even an inanimate piece of matter, like a stone, is a system of tightly interconnected molecules interacting in specific patterns that keep the solid matter intact. 

When we study systems in today's world, we are taught about two kinds of systems called as: linear and non-linear systems. The latter is also sometimes called "complex" systems. 

Linear systems are so called because their dynamics can be represented in the form of linear differential equations. In an intuitive sense, they represent an "open loop" interaction with their environment where inputs from the environment and the outputs to the environment do not confound one another. 

Almost all machines adopt a linear design model. Even with complex machinery like say an airplane, the overall flight dynamics is still linear. What this means is that the input that goes into the airplane (the air in front of the engines) is separate from the output (or the exhaust) that comes from the airplane. Airplanes in their current design would find it very hard to fly, if the exhaust and the turbulence it creates, were to somehow come back to the front of the engine. Indeed, when planes fly too close behind other large airplanes, they are known to even crash from the exhaust turbulence, also called the wake turbulence. 

But most natural systems are non-linear in nature. Human societies, ecosystems, weather, etc. all display rich forms of "closed-loop" interactions where the output of some unit in the system comes back to affect it subsequently. We see that in human societies all the time. Our output or actions have consequences that can come back to haunt us, in the (wrongly) called "law of karma".

Non-linear systems are known to be very hard to model, understand and predict. The term "butterfly effect" comes from the study of non-linear systems, representing what is called "sensitivity to initial conditions"-- which means that a small change in the initial conditions of a non-linear model, can result in vastly different ways in which its dynamics would pan out over time. Colloquially, it is said that, a butterfly flapping its wings in one part of the world, can potentially create a chain of events causing thunderstorms in some other part of the world. 

While non-linear systems are hard to model and understand, not all non-linear systems pose a hopeless case. Ludwig von Boltzmannn, who was one of the pioneers in statistical mechanics-- or the study of large number of interacting entities using statistical techniques, had this hypothesis about complex systems: large systems of interacting particles settle down to a stable state where the time average along a single trajectory equals the ensemble average.

This hypothesis was later found to be false, and not applicable universally. But it was true of a specific class of interacting, non-linear systems that has now come to be called "ergodic" systems and the above hypothesis is now called the Ergodic Hypothesis.

To understand ergodicity, let us take an example. Consider a music troupe-- my favourite is ABBA-- (I know, I am old!) who are famous and have a large fan following. They are known for their superlative performance that draws a huge audience whenever they perform. But each time they sing, their performance would not be an exact replica of previous performances. The same song sung at different times would be a little different from one another. But this difference will not be arbitrary-- most of the performances would hover around an expected set of metrics about music quality, beats, emotions, etc. that have made them famous. It would hardly ever be the case that the first time they sing a given song, it would be sublime and the next time it would be downright jarring. 

Suppose, in order to "quantify" ABBA's performance, we develop a set of metrics like highest musical pitch, number of frenzied fans, number of beats per minute, and so on, and we measure them painstakingly across each of their songs. Now, we can see the ergodic property that in a single long performance, the average of these metrics would likely be the same as the average of these metrics obtained from a large number of independent concerts. In other words, to get a good idea of an ABBA performance, we could either take small samples from several of their performances, or sit long enough in a single performance. In both cases, we would get a pretty good idea of an ABBA performance. 

Sometimes the troupe may try to experiment with their performance and change some fundamental stylistic elements. Most likely, such fundamental changes from their signature performances would not have gone down so well with the audience. In other words, the more the troupe changes, the more they would remain in the same neighbourhood of activities that made them what they were.

The ergodic property of systems points to a number of interesting implications. One, it indicates the presence of an "invariance" or a stable set of characteristics of the system that regulates and keeps each individual instance of the system around this stable region. It is almost as if, the system-- the music troupe in our example-- has its own characteristic sense of "self" that is expressed in its performances. Just like when we express ourselves as individuals, we display both a signature set of invariant properties that characterises us, as well as a randomised but bounded distribution of each individual expressions. If a single musician (my favourite now is M S Subbulakshmi) were to sing a given song (my favourite is Bhaja Govindam and here are two different performances of the same song: Performance 1 and Performance 2), each rendition would be somewhat different from each other-- yet all of them have the signature tune of the singer, whom we can recognise instantly. 

Consider again, our music troupe. Suppose that they hire a manager to promote their music, and the manager starts insisting on various changes to their styles according to what he thinks is the latest fad. Each new idea from the manager pulls the troupe's style in a different direction. But one day, if they grow weary of the manager and fire him, the troupe would then almost immediately revert back to their original characteristics. The "invariance" property again. Or shall we say the "elastic" nature of ergodic systems? That is, subject to minor distortions, the system reverts back to how it was, once the distorting influence is removed. 

Suppose also that the favourite troupe was once just a garage band when they started out. As they become rich and famous, they lifestyles change. They move on to bigger houses, buy cars and travel, and even invest in better, high quality instruments and audio systems. Despite all this, their signature distribution would still remain pretty much the same. The sound and video quality of their music may vastly improve, because of which, their impact may become even higher than before. But yet, the better "capability" they now have due to better instruments and money, only serves as a catalyst-- and not as a replacement or an alternative-- for their signature brand of music. 

If you have read about the Theory of Being, the above characteristics would have sounded very familiar. 

Dharmic thought that developed in India, was primarily based on observing life forms and its variegated characteristics. This is in contrast to the physics and mechanics developed in ancient Greece, that models the world in terms of inanimate matter. 

Dharmic philosophers observed living beings and were able to understand the invariance property of living beings. They could see that, despite the variegated behaviours of an animal-- these behaviours were not completely arbitrary. There was something latent and invariant behind the rich set of behaviours, which they could identify as the self. 

It was also apparent that it is our sense of self that drives us, and this is what we strive to preserve, and this is what sustains us. This characteristic of sustaining our sense of self is what is called dharma. They could also see that not every kind of interaction is conducive to dharma, which leads to the concept of adharma. We know this today as not every kind of non-linear interaction is ergodic, and there are indeed non-ergodic systems. 

Dharmic philosophers could also see that they could extend this concept of self and dharma to systems of being, like human societies, ecosystems, climate, and so on. Indeed, they could also use this method of inquiry to even understand the behaviour of metals and annealing, and in understanding several forms of systems of inanimate objects and their interactions. 

This lead to the development of a complete abstract theory of "being" which characterised Eastern thought for several thousands of years and sustained a vibrant civilisation under this thought umbrella. In Indian society, dharma is seen as an essential property for sustenance of life, since thousands of years. Life indeed is ergodic and to preserve life (and human societies and ecosystems), we need to preserve its ergodicity. 

Yet, if we see current day public rhetoric and self-help "motivational" books, many times they fail to recognise this important characteristic. 

"Travel and discover yourself" they say, and keep urging people to spend their money on "experiences" rather than "things". Travelling all the time helps us discover something alright-- we discover that we have spent most of our money, have no place we can call home, have no sense of belongingness anywhere, and so on. Travelling indeed does help us discover new things about the world and about ourselves-- provided we can do this in a sustainable fashion. If our passion for travel uproots us from our homes and our sense of identity and belongingness to somewhere, we have indeed paid a very high price. 

Indian wisdom on the other hand, always advised us to invest our money on building strong relationships and develop new ideas, rather than the dichotomy between experience and things. The underlying reason was of course, dharma

Similarly, "live outside your comfort zone" they say. And I'm tempted by Sheldon's quote in the Big Bang Theory: "It is called comfort zone, for a reason". Our best performance may indeed be outside of our comfort zone. But if our life motto is about living outside our comfort zone, we are most likely neglecting an important part of what sustains us. The fact that we feel comfortable in some settings says something about ourselves. It is one thing to just enjoy the comfort of our comfort zones, and yet another thing altogether to discover what does our comfort zone say about ourselves and how can we use it for self actualisation. 

Indian wisdom does not compel us to either be comfortable or uncomfortable. It instead urges us to meditate and observe ourselves, understanding what drives us, what is it that we seek, and by what do we feel disspirited. Our best performance comes not from being comfortable or otherwise, but from a deep sense of self awareness.

Let me stop at this note for now. There are several more interesting characteristics of ergodicity, and the dharmic way of life. More on them in some future posts.

03 March, 2017

Theory of Being -- IV: Prana and Entropy

This is the fourth in the series called Theory of Being. A clearer understanding of this post is possible when the reader has read through the earlier posts in this series.

To summarize the main elements, we started by noting that the universe as we know it can be broadly divided into two realms -- the energy realm and the information realm. Everything in the physical universe can be reduced to energy, and everything in the conceptual realm is fundamentally built from information.

We also started to develop a new theory of the universe, inspired by the dharmic worldview. Here, the building block of the universe is an entity called "Being". A being is a template that can host energy and information. The "state" of being is the information content in being. Beings tend to settle down into "stable" or "sustainable" states, which is called its dharma. These are states that correspond to robust optimality given its operational neighbourhood (called its Vidhi).

Beings can compose with one another to form bigger beings, with the entire universe itself forming the ultimate Being.

*~*~*~*~*~*

The tendency of beings to settle down into stable states is treated as axiomatic, as it is based on ample empirical evidence -- be it in the quantum dynamics of sub-atomic particles, or the elasticity of solids, or the dynamics of fluids or the phenomenon of homeostasis in living beings.

The philosophy of dharma that characterizes Eastern thought is built on the concept of sustainability of beings. This in in contrast to "particles" and "mechanics" that forms central elements of Newtonian worldview, that can be traced all the way to ancient Hellenic Greece.

It is not that Western thought has not recognized stable states or Eastern thought has not recognized mechanics. For instance, the relatively recent discipline of Game Theory, also dubbed as "A Beautiful Math" is predominantly based on understanding equilibria and stable states. Game Theory is applied as much to biological and social phenomena as much to physical phenomena.

The question we are now posing is to understand the way in which mechanics feature in the hermeneutics of dharma.

Even though stability and sustainability form the fundamental building blocks of the dharmic worldview, there is another important concept that we need to consider. This is the concept of Prana.

Prana is variously translated as "force", "energy", "life", etc. Of course, none of them capture its essence exactly. The closest working definition of Prana that we will be using to develop our theory is Prana as "vitality".

Prana is basically the fundamental element of vitality in the universe. It is what makes beings seek different levels of dharma or stable states.  A being with low levels of prana settles down in a "lower" state of dharma and a being with high levels of prana can reach "higher" states of dharma.

So how do we understand, and perhaps quantify this "lower" and "higher" states of dharma?

"Lower" and "higher" states of dharma can be distinguished by their "information content" or entropy.

Consider a society of beings each of whom are trying to maximize their survivability. (Note that, a society of such beings is also a being which is trying to maximize its sustainability.)

Each being has some needs for its survival, which requires it to build connections with its environment, made up of other beings. Let us say that we have one such society with low levels of "Prana". The beings do not have much vitality and have just enough resources to build just one connection. The beings are focused solely on their survival and do not have the wherewithal to process complex notions like social fairness and such. The beings just connect with one another so that they can get to everybody else, as quickly as possible. So in what kind of a stable state, does such a society end up?

Some truly awesome research efforts have shown that, the resultant "dharma" for such a society, looks somewhat like this:

Star graph: Image source: Wikipedia

A "star" network is an emergent stable property (or an "equilibrium" in Game Theory parlance) resulting from individual beings exercising their Prana to maximize their sustainability.

As we can see, a star network is optimal in the sense that, anyone can reach anyone else in the network, in a maximum of 2 hops. Given very little Prana of just making one connection, this is the class of network with the shortest separation, that can be built. No one imposed a star graph on this society -- it emerged as a stable property from individual beings seeking to sustain themselves! This network now becomes a "being" of the "star" variety. We are already talking evolution here!

So how does the "star being" fare with respect to its sustainability? For one, it is quite efficient, since anyone can reach anyone else in a maximum of two hops. The "star being" is also robust against random failures. If every node is equally likely to fail, and there are n nodes in the network, then, for (n-1) possible failures, the rest of the network (and the "star being") continues to function. It is only one critical failure (of the central node) that can kill the "star being".

However, the "star being" is also innately vulnerable because the central node is heavily loaded in comparison with all the other nodes. It has to manage (n-1) connections, while every other node manages just one connection. The heavy load on the central node makes it most vulnerable to burnout and collapse, which would in turn, bring down the entire "star being".

The star network also has very low information content or "entropy". If we have to describe a star network over a given set of n nodes, all we need to specify is which node is the central node. The rest of the network can be reconstructed just by knowing who is the central being.

We can make this society reach a higher level of dharma by infusing more "Prana" into the constituent beings. We can do this either by empowering them with more resources (by providing enough energy to make more connections), or by empowering them with richer information constructs like "fairness".

Let us take the second case to understand the concept of Prana. As we can see, Prana is not just energy. Increasing awareness also increases Prana. The vitality of a being towards sustaining itself is based not just on its energy, but also on its awareness.

When beings are aware of the network that they are part of, and understand concepts like fairness, they would be able to see that the star network that they created is not very sustainable. The network is its own adversary -- by overloading the central node and by giving it an unfair amount of load.

So how would a collection of beings operate towards sustainability, when they are empowered with the concept of fairness? They form what is called as a "scale-free network" or a "hub and spoke model" as shown below: 
Scale-free network. Image source: Wikipedia
A scale-free network has several "hubs" with different levels of "centrality". No node in the network is so overloaded that it has to manage all communications between all nodes. But a few nodes are slightly more central and manage more load than others. Failure of these nodes, do damage the network, but none of them will damage the network to such an extent that the failure of the central node in a star network does, where the entire network gets totally broken.

The "scale-free being" is more sustainable than the "star being" even though it is less efficient. In the "star being" anyone could communicate with anyone else in a maximum of just two hops, which is not the case here. But the "star being" was also inherently unsustainable, as the very nature of the being made it vulnerable for a fatal internal failure (of the central node).

The "scale-free being" also has more information content, or "entropy" as compared to the "star being". In order to describe the network we need much more information than just specifying who is the central node.

So basically, increase in Prana increases the information content of the stable state that the being settles down in -- a "higher" form of dharma, if you will.

Prana is not just about "living" beings -- in fact, there is no difference between living and non-living beings in dharmic thought. Prana is the "cosmic energy" that permeates everything in the universe.

The concept of annealing shows how infusion of Prana (heat energy) in metals can make them change their stable state, and change the characteristic nature of the metallic compound. Although annealing has its roots in metallurgy, the process itself is generic and is applicable in various other domains.

Now think of a computer that is fresh off the factory and just has the inbuilt firmware. When we switch it on, it settles down to a state where it can perform a very basic set of operations. Its Prana is very low. Now, add more Prana to this computer in the form of an operating system like Linux (ok, even Windows or Mac 😋), and suddenly it settles down in a state with much higher information content, and capable of doing much more things!

Dharma (the teleology of sustainability) and Prana (the teleology of vitality) form the basic building blocks of the universe!

Models of Intelligence

Today, whenever we speak of AI, we mean a specific model of AI based on large language models and diffusion, both of which, undergo foundati...