The Knowledge Illusion

Introduction: Ignorance and the Community of Knowledge

• In fact, most knowledge is little more than a bunch of associations, high-level links between objects or people that aren't broken down into detailed stories.
• The ability to think vastly increases the effectiveness of action. In that sense, thought is an extension of action.
• Our thought processes, our language, and our emotions are all designed to engage causal reasoning to help us to act in reasonable ways.
• Thought is masterful at extracting only what it needs and filtering out everything else.
• The mind is not built to acquire details about every individual object or situation. We learn from experience so that we can generalize to new objects and situations. The ability to act in a new context requires understanding only the deep regularities in the way the world works, not the superficial details.
• The study of the mind cannot be reduced to the study of the brain. Cognitive science is not the same as neuroscience.

1: What We Know

• This is the illusion of explanatory depth. Before trying to explain something (eg, how a toilet works, or how a bike's parts fit together), people feel they have a reasonable level of understanding; after explaining, they don't.
• Human beings are not warehouses of knowledge.
• Most of cognition consists of intuitive thought that occurs below the surface of consciousness. It involves processing huge quantities of information in parallel. When people search for a word, for example, we don't consider one word at a time sequentially. Instead, we search our entire lexicon - our mental dictionary - simultaneously, and the word we're looking for usually rises to the top. That's not the kind of computation that von Neumann and Turing had in mind in the early days of computer science and cognitive science.
• Looking closer at things (is like zooming in on a fractal) just raises more questions. There is always more to understand.
• Adults forget how complex things are and decide to just stop asking questions. Because we are not conscious that we have made this decision to stop probing, we end up thinking we understand how things work more deeply than we do.

2: Why We Think

• Hyperthymesia, highly superior autobiographical memory, is extremely rare, affecting only a handful of people, who can remember in copious detail everything that has happened to them.
• Almost all animals have brains. The neuron as one of the earliest adaptations when animals branched off from other organisms. Even animals that don't have fully structured brains have nervous systems, networks of neurons that work together to process information. Plants on the other hand do not have brains. No plant evolved cells that can organize into networks to process information.
• As soon as animals developed neurons and nervous systems, the complexity of their actions exploded and developed at a remarkable rate. This happened because the neuron is the building block of a flexible system that evolution can use to program more and more complex information-processing algorithms.
• Bring smart is all about having the ability to extract deeper, more abstract information from the flood of data that comes into our senses. Instead of just reacting to the light, sounds, and smells that surround them, animals with sophisticated large brains respond to deep, abstract properties of the world that they are sensing.
• Remembering everything gets in the way of focusing on the deeper principles that allow us to recognize how a new situation resembles past situations and what kinds of actions will be effective.
• Storing details is often unnecessary to act effectively; a broad picture is generally all we need.

3: How We Think

• We are familiar with a huge number of mechanisms that produce effects.
• For each mechanism we're familiar with, we understand enough about how it works to know what must be true for the mechanism to have the effect we expect and what must be false for the mechanism not to have its effect.
• We're not good at everything. What we do excel at is reasoning about how the world works. We're gifted causal reasoners, and rats, as it happens, are too. What could be more useful if you're an animal who has evolved to operate in the world?
• We infer all kinds of things that we're not told and we don't observe directly.
• We may make inferences all the time, but those inferences are not based on textbook logic; they are based on the logic of causality.
• Rather than thinking in terms of propositional logic, the logic that tells us whether a statement is true or false, people think in terms of causal logic, the logic of causation that incorporates knowledge about how events actually come about in order to reach conclusions.
• Our ability to project our thoughts into the distant future is also a kind of causal reasonng.
• Predicting the effects of our actions on other people is sometimes hard, but again, we do it all the time, mostly successfully.
• Forward reasoning is thinking about how causes produce effects.
• Reasoning backward is reasoning from effects to causes. Doctors do it to diagnose the cause of symptoms and mechanics do it to diagnose what's wrong with your car.
• Backward reasoning from effect to cause may be hard, but it's also what makes humans special; it's not clear that any other organism has the capacity or interest to figure out the causes of what has happened.
• People don't worry about alternative causes when thinking about the likelihood of an effect given a cause.
• When reasoning diagnostically, from effect to cause, our respondents didn't neglect alternative alternative causes.
• Perhaps the most common way that people pass causal information to one another is by storytelling.
• One tale from the Bible discusses the root cause of everything, how the world was created. Many biblical stories tell us about the consequences of our actions and why, therefore, certain actions are right and other wrong.
• Fairy tales and urban legends tend to teach us about what we should avoid, what's dangerous, and how we determine who to trust. Stories about heroic acts tell us about the surprising extent of our own potential.
• A good story has a moral that applies jot just this world but also to other worlds that we might find ourselves in.
• Storytelling helps us to imagine how the world would be if something were different. This is clearest in science fiction.
• Thinking about alternative possible worlds is an important part of being human. It is called counterfactual thought, and you can see that it depends on our capacity to reason causally. Why do we do this? Why do we so naturally tell stories that require reasoning about counterfactual worlds? Perhaps the main motivation is that it allows us to consider alternative courses of action.
• Psychologists have proposed that stories make up our identities, both our individual identities and the identities of the groups we are a part of. We tell stories about the past - we reminisce and romanticize. We tell stories about the future - we predict and fantasize. And we tell stories about the present - we construct who we are and daydream. All of this is about identifying causes and foreseeing effects. How did we come to be? Where are we going? What actions should I take right now?
• Stories are about the causal relations in the world, however much we simplify them. So individuals need a cognitive system that can understand the causality inherent in whatever the story they are telling us about. We need a cognitive system that can understand what protagonists and antagonists want to achieve, how obstacles prevent achievement of those goals, and how those obstacles are overcome (or not, as the case may be). These are all causal notions involving agents trying to cause the world to be a certain way. It is no coincidence that storytelling, the most natural mode of human discourse, depends on the very sam resource - causal knowledge - that allows thinking to produce more effective action.

4: Why We Think What Isn't So

• Thought is full of guesswork and approximation.
• For the small range of conditions we encounter, shallow causal reasoning is all we need to get by.
• People excel at causal reasoning; we just tend to do it superficially outside our area of expertise.
• Intuition vs deliberation:
  • Intuition provides answers that come to mind quickly, spontaneously, and effortlessly - An animal beginning with E? Elephant!
  • Deliberation is when you're conscious not only of the response, but also how you got there. [but the example, of a hardish anagram, does not feel very deliberative to me - I feel I'm still using deliberation - It's just that I'm concentrating attention on the anagram for longer to force intuition to keep returning responses to me until the right one comes along]
• Intuitions and passions are not the same, but there is a close link between them.
• Sometimes the conclusions we come to intuitively - quickly and effortlessly - are overruled by conclusions that we come to more slowly and effortfully, via deliberation. Intuition leads to one conclusion, but deliberation makes us hesitate.
• In certain Hindu and yoga traditions, there are seven chakras:
  • The first is lowest down and has to do with rooting the to the earth.
  • The sacral chakra is slightly below the navel and is associated with sexuality and related entities and drives.
  • The third is slightly above the navel and relates to fire.
  • The fourth is near the heart and has to do with love.
  • The fifth is in the throat and is related to communication
  • The sixth, the Ajna chakra, is between the eyebrows. It is like a third eye and is associated with visual imagery and intuition, the thoughts that come automatically, shaped by unconscious processes.
  • The seventh, the Sahaswara chakra is at the crown of the head and is associated with intelligence and consciousness - ie deliberation.
• A team of people cannot intuit together, but they can deliberate together.
• The illusion of explanatory depth is a product of the intuitive mind; we think about how things work automatically and effortlessly. But when we deliberate about our knowledge, our illusion is shattered.
• When asked a problem in the Cognitive Reflection Test, most people intuit the answer (wrongly). A small proportion of people check their intuitive answer, realize that it is wrong and then calculate the right answer. These reflective people tend to suppress their intuitive responses and deliberate before responding.
• More reflective people are more likely than less reflective people to prefer dark chocolate to milk chocolate. they are also less likely to believe in God. And they show less of an illusion of explanatory depth then less reflective people.
• Intuition gives us a simplified, coarse, and usually good enough analysis, and this gives us the illusion that we know a fair amount. But when we deliberate, we come to appreciate how complex things actually are, and this reveals to us how little we actually know.
• Unlike most people, highly reflective people crave detail. They like to explain things, so it is not far-fetched to assume that they explain things even before they are asked to. And anyone who does that will not suffer the illusion of explanatory depth.

5: Thinking with Our Bodies and the World

• The subsumption architecture conceives of intelligence as a grand hierarchy: High-level sophisticated tasks are performed by combing simpler skills that are in turn just organizations of even simpler skills. Sophisticated tasks get done not through exhaustive computation and planning but by engaging a hierarchy of actors in an organized way that, at the lowest level, are just responding directly to the environment.
• The assumption that the world is behaving normally gives people a giant crutch. It means that we don't have to remember everything because the information is stored in the world. If I need to know something, all I have to do is look at it. If I need to know what the sentence was at the top of the page, I don't have to remember it - I can just look at it again. The visual environment functions as a sort of outside.
• You're seeing only a tiny bit of the world at a time, but you know the rest is there, just not in your head.
• The world is acting as your memory.
• We feel like we have a model of our environment in our heads, a detailed representation of everything that is there. But we don't.
• Our cognition is unified with the objects that we're thinking about and with. When we make music, our thought about music and the music we make with our mouth or an instrument are part of the same process and highly interdependent.
• Mental activities do not simply occur in the brain. Rather, the brain is only one part of a processing system that also includes the body and other aspects of the world.
• The brain and the body and the external environment all work together to remember, reason, and make decisions. The knowledge is spread through the system, beyond just the brain. Thought does not take place on a stage inside the brain. Thought uses knowledge in the brain, the body, and the world more generally to support intelligent action. In other words, the mind is not in the brain. Rather, the brain is in the mind. The mind uses the brain and other things to process information.

6: Thinking with Other People

• -

7: Thinking with Technology

• -

8: Thinking About Science

• -

9: Thinking About Politics

• -

10: The New Definition of Smart

• -

11: Making People Smart

• -

12: Making Smarter Decisions

• -

Conclusion: Appraising Ignorance and Illusion

• -