3. Two Dirty Words

Reductionism and Holism

After having sorted out what Models are, we can now discuss two complementary problem solving strategies (or perhaps meta-strategies). They are in many ways each other’s opposites, but the classification can become an argument about meta-levels and definitions. I will initially pretend the division is clear and obvious, and will elaborate later.

Reductionism is the Use of Models

In this blog series we will use exactly the above definition of the word “Reductionism”. If you look up the definition elsewhere you may find that some sources divide the strategy into sub-strategies. They also seem to miss the most important sub-strategy, which we’ll discuss later. But what all these sub-strategies have in common is that they all provide ways to simplify observations of fragments of our rich mundane reality into much simpler Models, which we use for reasoning, computation, and sharing.

Reductionism is so central to how we do science — the heavy reliance on Models, such as theories, equations and formulas, in physics, chemistry, etc. — that we can speak of “Model Based sciences” or “Reductionist sciences” where such Model Making is easy and effective. And this classification excludes those sciences, like psychology, where such Model making is difficult and less often rewarded with reliable results.

After considering all the advantages of Models we might wonder why we even bother discussing it. To many people, especially those with a solid STEM (Science, Technology, Engineering, and Mathematics) education, it may well look like the only choice. But there is also the other strategy:

Holism is the Avoidance of Models

This is where the questions start. This is where the paradoxes surface. This is where your worldview may get shaken up. Seriously. Especially if you are a scientist or engineer with a solid STEM education and decades of professional success using science and Models.

In some sense, the goal of this entire blog series is to demonstrate that we need to use both problem solving strategies when creating our artificial Intelligences. Because that is what it is going to take. We need Holistic Understanding; we established that in the first chapter. As a sample of the new ideas that we will have to deal with I will just mention

• Reasoning is Reductionist
• Understanding is Holistic
• Neural Networks are Holistic
• Holistic systems can jump to conclusions on scant evidence
• Holistic systems can themselves know what is important and what isn’t
• Holistic systems can solve problems we ourselves cannot (or don’t care to) Understand
• Holistic systems are “Model Free”. They do not use any a priori Models of any problem domain
• Reasoning systems inherit all problems and benefits of Reductionism.
• Understanding systems inherit all problems and benefits of Holism
• Humans are born Holistic
• Humans each solve thousands of little problems every day, and we are solving almost all these problems Holistically, using Understanding, and without a need to Reason at all. This includes fluent language use.
• A STEM education instills a strict Reductionist discipline in order to mitigate problems with fallibility of Holistic human minds
• All intelligences are fallible

These claims all deserve individual treatments, and we’ll get to all of them in later blogs. But the major theme is clear:

Humans are mainly Holistic problem solvers. The same must be true for our Artificial Intelligences

We had several reasons for focusing on Reductionist Methods (Models) and Reasoning during the first 60 years of AI. Our computers were too small to make Neural Networks work at all. But there were also ideological reasons. AI was born out of the math and computer science departments of our universities. And therefore most of the people working on AI were solidly oriented towards the goal of creating a logic based Reductionist infallible artificial mind. To build early AIs, like expert systems, we entered rules or programmed in lots of facts to Reason about. But this was building Reductionist castles in the air, comprised of un-anchored facts that didn’t tie to any Understanding whatsoever. The troubles with classical AI, such as brittleness (the tendency to make spectacular and expensive mistakes at the edges of their competence), can be directly traced to the lack of foundational Understanding to support these attempts at Reasoning.

Understanding Machines will not suffer from this brittleness, but will fail gracefully at the edges of their competence, much like humans. Most of the time they will “know” the answer; beyond that they will guess. And the guesses they make are based on a lifetime of experience (gained through learning from a large corpus) and so they have a good chance of being at least a workable choice, if not perfect.

How can anyone solve problems without using Models?

A lot of people coming from a STEM background cannot even imagine how to solve problems without using Models. But it’s not hard, once you understand the difference. Mostly it’s a matter of doing what worked last time. The problem is now figuring out whether we are in a situation that’s similar enough that it will work again. This is mostly a pattern matching problem. More later.

What’s the result? The Holistic answer is a quick guess at the best action, based on experience with similar situations. Most of the time it’s correct, sometimes it’s a little wrong, and every now and then, there’s a noticeable mistake. And if we get things a little wrong, we may notice the outcome and correct the action. We learn from our mistakes. If we practice something a lot we will start doing it effectively and perfectly every time. Do we learn faster if we make more mistakes? Should we make mistakes on purpose? More later.

In situations where you cannot use Models, which are more common than many realize, the Holistic guess may also be your only option. Conversely, if you have an adequately well working Model based solution, just use that.

My video “Model Free Methods Workshop” demonstrates how the group solves four different problems, at a high level, using both Reductionist and Holistic Methods.

Why are these “Dirty words”?

Well, they are not dirty to Epistemologists.

Reductionism has been the default problem solving paradigm because it’s the one that has to be taught. We are born with a holistic problem solving apparatus. But Reductionist science doesn’t come naturally; therefore it has to be taught in schools, practiced, and carried out according to certain rules. Perhaps that’s why the sciences are called “disciplines”, because following the ideal scientific method requires practice and constant vigilance.

J. C. Smuts’ book “Holism and Evolution” (1926) established the terminology in the Epistemological literature. And Erwin Schrödinger wrote “What is Life” (1944), questioning the power of physics to provide useful explanations to the life sciences. Pirsig wrote “Zen and the Art of Motorcycle Maintenance” (1974) that contrasts something very Holistic, Zen Buddhism, with something very Reductionist, Motorcycle Maintenance. So the chasm between the strategies was identified a long time ago.

The strategies are each other’s opposites. Holism based strategies for Understanding can handle many important kinds of complexity and can quickly provide a guessed answer, but these guesses are fallible, and often more expensive to compute. Reductionist education and strategies brought benefits of cheap Model reuse and formal rigor to improve correctness, but cannot handle complexity and is therefore dependent on an external Understander to determine applicability in real-world complexity-rich situations. And as part of that education, we are told that Holistic methods (such as jumping to conclusions on scant evidence) are bad… in spite of the fact that our brains use Holistic methods thousands of times each day to successfully Understand the environment we live in.

We can all use either strategy as appropriate; if we don’t have a STEM education we will still sometimes make naïve Models. But sometimes there is a choice and different people may prefer one or the other. When playing pool, some people estimate and compute bouncing angles and some people shoot “by feel”.

But we have our preferences, and it might be tempting to label a person with an overly strong preference as “a Holist” or “a Reductionist”. This is sometimes received badly, if perceived as a limitation. Some dictionaries even flag “Reductionist” as derogatory; and yet, some people use it as a self-assigned label. I try to use these terms only as shorthand for “a person with a stated strong preference for Holistic (or Reductionist) Methods”.

The two terms were very useful in Epistemology. But then someone invented the concept of “Holistic Medicine”: Instead of just treating a single medical problem, you analyzed the patient’s entire situation, attempting to account for diet, exercise, sleep, work, habits, stress levels, allergies, family, friends, and environmental poisons. A good idea, in general. But the wide scope was unmanageable by the (traditionally Reductionist) medical establishment and the idea faded away. Instead, the whole idea of Holism became tainted as woo-woo when the term “Holistic Medicine” became associated with woo-woo merchants selling crystals and aromatherapy.

As explained above, “Holism is the avoidance of Models”, or better phrased, “Holism is the meta-strategy of avoiding a priori Models of the problem domain”. That extra precision rarely matters. There’s nothing woo-woo about it. It does say “science not required”, but…

You can make breakfast without Reasoning

It is important to note that Holistic methods are based on a lifetime of experience (in humans) and a training corpus’ worth of experience (in Neural Networks). When you are making breakfast, you are relying on this experience, mostly repeating whatever worked yesterday.

Some people claim they use Reasoning while making breakfast… but they can make their breakfast while speaking to someone else on the phone and as they hang up, they find themselves suddenly sitting at the breakfast table with their coffee and hot oatmeal. Same thing when driving to work; you may get lost in thought, and then you find yourself parked at work. You didn’t need to Reason, since all subproblems that occur in driving had been solved multiple times during years of driving. Subconscious Understanding is used for simple things like sequencing our leg muscles as we walk; you have no idea how you are doing that, it just works. Same thing with vision. You Understand that you are looking at a chair, but you do not have conscious access to the fifteenth rod/cone/pixel to the left of your center of vision, and have no idea how this Understanding works. Same thing with understanding and generating language. You do not have any explanation for how you are able to understand the meaning of this sentence. Understanding is Subconscious and Holistic.

So for the majority of things we do every day, we do not need Reasoning or Reductionist Methods. Some people would like to think they are “logical thinkers”, immune to most cognitive fallacies, but whether they are or not, at the lower levels, everyone is solving most of their problems Holistically.

I claimed that “Reductionist Reasoning requires Holistic Understanding”. In other words, I need to Understand the problem domain at hand before I can create and/or re-use Models to enable me to Reason about the domain.

So Holistic Understanding is much more important than Reductionist Reasoning because it is the most used strategy, by far, and the former is also a prerequisite for the latter. But the fallibility of Holistic Understanding forced us to create Reductionist science and to teach it in STEM educations. It is as if the purpose of science is to keep Holistic guessing in check. But this aversion to fallibility has a cost, because it means complexity-bound and “irreducible” problems cannot be solved. Like language Understanding, global resource allocation, and social interactions.

Reductionism and Model based science appeared around 1650 after a century of gestation. Excluding minor romantic interludes, it has held its position as the dominant paradigm for about 400 years. This is changing.

The Reductionist train is running out of track

The remaining hard problems facing humanity are problems of irreducible complexity in domains where Reductionist Model Based Methods simply cannot work. Whether we like the idea or not, we need to accept these Holistic methods into our AI toolkits, starting now. We will use these methods either in their raw form, as Model Free Methods, or as Understanding Machines at any level from component to robot co-worker.