Notes on Seeing What Others Don’t꞉ The Remarkable Ways We Gain Insights

book
notes
My notes from the book Seeing What Others Don’t꞉ The Remarkable Ways We Gain Insights by Gary Klein.
Author

Christian Mills

Published

November 10, 2024

Book LInks:

Part I: Entering Through the Gates of Insight

Chapter 1: Hunting for Insights

Introduction: The Genesis of an Investigation

  • Klein began collecting clippings about unusual discoveries.
  • These clippings included newspaper and magazine articles, as well as notes from interviews and conversations.

The Stolen BMW: An Illustrative Example

  • Two police officers were stuck in traffic during a routine patrol.
  • The younger officer observed the driver of a new BMW ashing a cigarette onto the car’s upholstery.
  • This sparked an insight: Who would ash a cigarette in a brand new car?
  • The officers pursued the car and discovered it was stolen.

The Up Arrow: A Question of Insights

  • Klein, a cognitive psychologist, gives presentations on decision-making.
  • In 2005, influenced by Positive Psychology (a movement started by Martin Seligman focusing on positive experiences), Klein began to incorporate the concept of insights into his presentations.
  • Seligman believed therapy focused too much on reducing misery and not enough on adding meaning and pleasure.
  • Klein felt similarly about decision research, emphasizing the need to improve insights alongside reducing errors.
  • A slide with two arrows illustrated this point:
    • Down arrow: Reducing errors.
    • Up arrow: Increasing insights.
  • Performance improvements = (Down arrow of errors) + (Up arrow of insights)
  • Klein argues that eliminating errors alone does not lead to insights and may even hinder them.
  • Audiences resonated with this idea but wanted to know how to boost the up arrow (increase insights).
  • This question prompted Klein’s investigation into insights, starting in September 2009.
  • The investigation aimed to explore how insights are formed in natural settings, rather than artificial laboratory environments.
  • This led to two mysteries:
    1. What sparks an insight?
    2. What prevents us from grasping an insight?
  • A third, related challenge also emerged: How can we practically increase the flow of insights?

Lighting Up Life: The Story of Martin Chalfie

  • Martin Chalfie, a professor at Columbia University, won the 2008 Nobel Prize in Chemistry.
  • His path to the prize began with a lunchtime seminar in 1989.
  • Chalfie studied the nervous system of transparent worms. This transparency was an incidental feature, unimportant until his insight.
  • Prior to the seminar, Chalfie’s research required killing the worms to examine their tissues.
  • At the seminar, the speaker discussed the green fluorescent protein (GFP) found in jellyfish, which emits green light when exposed to ultraviolet light.
  • This triggered Chalfie’s eureka moment: He realized he could insert GFP into his transparent worms and use ultraviolet light to track the protein and the cells it was placed in.
  • GFP became a vital tool in molecular biology. It is now available in multiple colors and used in diverse applications, including:
    • Tracking viruses in mice.
    • Visualizing prostate cancer cells.
    • Illuminating nerve fibers during surgery.
    • Detecting pollution.
    • Targeted pesticide application in agriculture.
    • Creating bioluminescent trees (a theoretical application).
    • Creating the world’s first transgenic dog, Ruppie.
  • Chalfie’s insight demonstrated key characteristics of insight formation:
    • Sudden and unexpected.
    • Emotionally charged (excitement).
    • A combination of existing ideas (transparent worms and GFP).
    • Transformative, shifting his research focus.
    • Unique to him due to his specific research subject.
    • Creative, resulting in something new.
  • Insight “Geiger counter” cues:
    • Sudden discovery.
    • Jolt of excitement.
    • Tightly fitting combination of ideas.
    • Confidence in the new direction.
    • Uniqueness of the insight despite shared information.

Unmasking Madoff: The Persistence of Harry Markopolos

  • Harry Markopolos, a financial analyst and certified fraud examiner, exposed Bernie Madoff’s Ponzi scheme.

  • In 1999, Markopolos was challenged to replicate Madoff’s impressive investment returns.

  • Upon analyzing Madoff’s publicly available data, Markopolos quickly realized something was amiss. The numbers didn’t add up.

  • Madoff claimed to use a conservative split-strike conversion strategy, yet reported consistent profits, only losing money in three months over seven years.

    A split-strike conversion strategy involves buying a basket of stocks as well as option contracts on them. For each stock, the investor buys a put option to protect against the price falling too low and, at the same time, sells a call option to let someone else buy it if it rises above a given strike price. If the price rises, the investor makes a profit, but only up to the strike price. The gain is capped. If the price falls far enough that it hits the put level, the investor has limited the loss.

  • Markopolos immediately concluded that Madoff’s results were fabricated.

  • Despite repeated warnings to the SEC from 2000 to 2008, Markopolos was dismissed as a crank. Reasons for this included:

    • Madoff’s high reputation and connections.
    • Markopolos’s relative obscurity and perceived eccentricity.
    • Markopolos’s status as a competitor and his mention of a potential reward.

  • Markopolos believed the SEC was ill-equipped to handle such complex fraud and more focused on protecting Wall Street.

  • Like Chalfie and the young cop, Markopolos’s insight was sudden.

  • His experience allowed him to recognize inconsistencies others missed.

  • The insight transformed his beliefs about Madoff, shifting from disbelief to suspicion.

  • Unlike Chalfie, who combined ideas, Markopolos identified inconsistencies in data.

  • Chalfie focused on building on ideas, while Markopolos and the cop focused on identifying unlikely or impossible beliefs.

Stumbling onto a Plague: Michael Gottlieb and the AIDS Epidemic

  • Dr. Michael Gottlieb published the first report on the AIDS epidemic.
  • In 1980, while at UCLA, he encountered a 31-year-old male patient with a severe throat yeast infection.
  • Blood tests revealed an abnormally low helper cell count and a high suppressor cell count, the opposite of the typical pattern.
  • The patient later developed pneumocystis pneumonia, a fungal infection typically affecting individuals with compromised immune systems, not healthy young adults.
  • Gottlieb noted the patient’s lifestyle (gay) and overheard him refer to himself as a “sick queen.”
  • Over the following months, Gottlieb encountered more patients with similar symptoms, all gay men.
  • By April 1981, Gottlieb had five cases, recognizing a pattern.
  • He published his findings in the Morbidity and Mortality Weekly Report, the first public announcement of the AIDS epidemic.
  • Gottlieb’s insight was the recognition of a frightening pattern, though he didn’t initially know the cause.
  • Unlike the sudden insights of the previous examples, Gottlieb’s developed gradually.
  • His insight focused on identifying a pattern, whereas Chalfie combined ideas, and Markopolos and the cop detected inconsistencies.

Getting My Car Fixed: An Everyday Insight

  • Klein experienced a minor insight regarding car repairs.
  • After rescheduling his car service, Klein devised a better solution involving dropping off his car key instead of the car itself.
  • This insight came after a period of incubation, without new information.
  • This example illustrates that insights can be commonplace and don’t always require great creativity.

Conclusion: The Nature of Insight

  • Humans are naturally inclined to gain insights.
  • We seek patterns, connections, inconsistencies, and better ways of doing things.
  • Klein’s initial project aimed to find commonalities among insights and offer advice on increasing them.
  • However, the diverse nature of the collected stories led to the central mystery: What sparks an insight?

Chapter 2: The Flash of Illumination

Graham Wallace’s Model of Insight

  • Graham Wallace, co-founder of the London School of Economics, published the first modern account of insight in his 1926 book, The Art of Thought.
  • Wallace’s model, still commonly used to explain insight, consists of four stages:

    1. Preparation: Conscious, systematic, but often fruitless investigation of a problem.

    2. Incubation: Shifting to unconscious thought by ceasing conscious deliberation.

      • Helmholtz (1891): “Happy ideas come unexpectedly without effort, like an inspiration… particularly readily during the slow ascent of wooded hills on a sunny day.”

      • Wallace advised mental relaxation and avoidance of mentally demanding activities during this stage.

      • John Drinkwater quote: “Haunting the lucidities of life that are my daily beauty move a theme beating along my undiscovered mind.”

    3. Illumination: Sudden and certain emergence of the insight (“happy idea”) from unconscious associations.

      • Fringe consciousness may provide an intimation of impending illumination.

      • Premature verbalization of the insight can disrupt its full formation.

    4. Verification: Consciously testing the validity of the insight. May involve working out details, particularly in fields like mathematics.

Critique of Wallace’s Model

  • Preparation Stage Critique:
    • Klein argues that the preparation stage isn’t necessary, citing examples from Chapter 1 (police officer, Chalfie, Markopolos, Gottlieb, and Klein’s own car key insight) where insights arose unexpectedly without deliberate preparation.
    • Distinction between Expertise and Preparation: While background knowledge and expertise play a role, they differ from deliberate preparation to solve a specific problem.
      • Generally Prepared Mind (Expertise): Possessing the necessary background knowledge to recognize the insight (Chalfie, Markopolos, Gottlieb).
      • Specifically Prepared Mind (Deliberate Preparation): Consciously preparing to solve a defined problem (Wallace’s view).
    • DNA Structure Discovery Example: While Watson and Crick’s work involved deliberate preparation, other researchers with similar preparation (Rosalind Franklin, Linus Pauling) did not achieve the same breakthrough.
    • Conclusion: Deliberate preparation is neither necessary nor sufficient for many insights. Wallace’s focus on success stories skewed his model.
  • Incubation Stage Critique:
    • The incubation stage is also deemed unnecessary in many Chapter 1 examples. Only Gottlieb and Klein had significant time between initial observations and the insight.
    • Incubation’s Appeal: While appealing to our “play ethic,” it’s questionable whether enforced relaxation would reliably boost insights.
    • Conclusion: Incubation isn’t necessary and is often impossible given the circumstances of many insights.
  • Illumination Stage Critique:
    • All examples involved a flash of illumination, although Gottlieb’s was more gradual.
    • Wallace’s Explanation: Klein finds Wallace’s explanation of unconscious associations unsatisfactory, describing it as “too magical.” This stage and its underlying mechanisms are the focus of Klein’s investigation.
  • Aha vs. Insight:
    • Klein proposes a distinction between the “aha” experience and the insight itself.
    • Analogy: “Aha is to insight as orgasm is to conception.” The experience is distinct from the achievement, and the latter can occur without the former.

Redefining Insight

  • Impasse vs. Better Story: Traditional cognitive science views insight as overcoming an impasse, but the Chapter 1 examples don’t involve impasses. Klein seeks a better definition.
  • Insight Definition: An unexpected shift to a better story about how things work.
    • These stories explain causes of past/present events (cop, Markopolos, Gottlieb) or ways to cause future outcomes (Chalfie, car keys example).
    • These shifts involve changes to core beliefs, not minor adjustments.
    • Discontinuous Discoveries: Unexpected transitions from a mediocre story to a better one. Sometimes immediate (cop, Chalfie, car keys), sometimes gradual (Markopolos, Gottlieb).

Transformative Effects of Insights

  • Insights transform us in five ways:
    • Understanding: Provide a new viewpoint, transforming our thinking.
    • Actions: Change our abilities and notions of what’s possible (Chalfie, car keys).
    • Perception: We look for different things, guided by the new story (Gottlieb).
    • Feelings: Change what excites us or makes us nervous (police officers, Markopolos).
    • Desires: Shift our goals and ambitions (Chalfie, Markopolos, Gottlieb).
  • Quotes:
    • Klein’s Friend: “Insight is when it happens. Everything that happens afterward is different.”
    • Hilary Mantel (Wolf Hall): “Insight cannot be taken back. You cannot return to the moment you were in before.”

Unique Characteristics of Insights

  • Non-conscious and Non-deliberate: Insights arise without warning, unlike routine problem-solving (Watson and Crick’s DNA discovery).
  • Coherence and Unambiguity: Insights provide a single, clear answer, not a set of options (“Oh, yes, that’s it”).
  • Confidence and Certainty: A sense of closure leads to confidence in the insight.
  • Aesthetic Reaction: Insights can evoke a sense of beauty (Watson and Crick, Chalfie, Markopolos, Gottlieb, police officer).

A Naturalistic Investigation

  • Naturalistic Approach: Klein adopts a naturalistic approach, studying insights in real-world settings rather than through laboratory puzzles.
  • Previous Research on Firefighters: Klein’s previous work on firefighter decision-making (recognition-primed decision strategy) informed this investigation, particularly the concept of shifting stories based on new information.
  • Stories and Anchors: Stories frame and organize details of a situation. Anchors are core beliefs that guide interpretation of details. Most stories have 3-4 anchors.
    • Cop: Expensive car, driver’s indifference.
    • Chalfie: Worm transparency, GFP glow, protein implantation.
    • Markopolos: Fraudulent funds, Madoff’s conservative method.
    • Gottlieb: Disease targeting gay men, immune system devastation, vulnerability to infections.

Research Methodology

  • Archaeological Trench Analogy: Klein’s research is like an archaeological trench, a focused exploration rather than a full excavation.
  • Data Collection: 120 insight incidents compiled from media (books, articles), interviews, and personal discussions. Focus on successful insights.
  • Control Sample: A smaller set of cases with built-in controls (comparing those who had the insight with those who didn’t) was added later to address the bias toward success stories.
  • Data Analysis:
    • Summarization: Each incident summarized in a short account (3 pages or less).
    • Coding: 14 coding categories were used to analyze incidents (e.g., impasse, incubation, suddenness).
    • Inter-rater Reliability: Initial agreement of 78% improved to 98% after discussion and refinement.
  • Five Insight Strategies:
    • Connections, Coincidences, Curiosities, Contradictions, Creative Desperation.
    • Most incidents involved multiple strategies.
    • Klein questions which strategy is primary or whether a combined approach is more appropriate. These strategies are explored in the following chapters.

Chapter 3: Connections

Introduction: Connecting the Dots and Solving Problems

  • Martin Chalfie’s experience exemplifies the process of connecting disparate information to solve a problem.
  • New information combined with existing knowledge leads to discoveries.
  • This chapter explores this process through the examples of:
    • Admirals who identified Pearl Harbor’s vulnerability.
    • A child psychologist who made discoveries about infant empathy.
    • Charles Darwin’s formulation of the theory of evolution.

The Battle of Taranto and Pearl Harbor

  • The Battle of Taranto (November 11-12, 1940): Demonstrated the vulnerability of battleships to carrier-launched aircraft.

    • Context: World War II, Britain fighting Germany, Italy, and Japan.
    • Italian fleet hindering British resupply efforts in Egypt.
    • Operation Judgment: British plan to attack the Italian fleet sheltered in the Bay of Taranto.
    • Secrecy maintained through lack of written records.
    • First all-carrier air attack in history: Launched from HMS Illustrious.
    • Attack launched on November 11th, 9 PM, reaching the target at 10:58 PM.
    • Within an hour, half the Italian fleet was disabled for six months.
    • Attack carried out with 24 British Swordfish biplane bombers.
    • Overcoming the depth challenge:
      • Previous belief: Torpedo attacks required a minimum depth of 30 meters (98 feet).
      • Taranto Bay was only 12 meters (40 feet) deep.
      • British innovation: Used wire to adjust torpedo trajectory for a “belly flop” instead of a nosedive.
      • Added wooden fins to prevent deep dives.

  • Admiral Isoroku Yamamoto’s Insight:

    • Recognized the implications of Taranto for Pearl Harbor.

    • Envisioned the American fleet at Pearl Harbor as similarly vulnerable.

    • Developed the idea in January 1941, refining it into the blueprint for the Pearl Harbor attack (December 7, 1941).

    • Yamamoto’s opposition to war:

      • Foresaw Japan’s defeat in a war with the United States.

      • Expressed concerns in a leaked letter:

        Should hostilities once break out between Japan and the United States, it would not be enough that we take Guam and the Philippines, nor even Hawaii and San Francisco. To make victory certain, we would have to march into Washington and dictate the terms of peace in the White House. I wonder if our politicians, who speak so lightly of a Japanese-American war, have confidence as to the final outcome and are prepared to make the necessary sacrifices.

      • Studied at Harvard, fluent in English.

      • Opposed Japanese invasions of Manchuria and China, the pact with Nazi Germany, and the war with the United States.

      • Appointed commander-in-chief to protect him from assassination attempts by Japanese militarists.

  • Japanese Army Leaders’ Perspective:

    • Confident of Japanese victory.
    • Eager to eliminate the United States as an obstacle to their Southeast Asian domination plans.

  • Yamamoto’s Rationale for Pearl Harbor:

    • Acknowledged Japan’s inability to outlast the United States in a prolonged war.
    • Aimed for a decisive opening blow to cripple American naval power.

  • The Pearl Harbor Attack (December 7, 1941):

    • 353 Japanese aircraft from six carriers.
    • All eight battleships at Pearl Harbor hit, four sunk, others damaged.
    • Other ships sunk or damaged: three cruisers, three destroyers.
    • 188 U.S. aircraft destroyed.
    • 2,402 American casualties.

  • Admiral Harold Stark’s Warning:

    • U.S. Chief of Naval Operations (CNO).

    • Recognized Taranto’s implications even before Yamamoto.

    • Memo (November 22, 1940):

      By far the most profitable object of a sudden attack in Hawaiian waters would be the fleet units based in that area.

    • Suggested placing torpedo nets within the harbor.

    • Letter to Secretary of the Navy Frank Knox (January 24, 1941):

      If war eventuates with Japan, it is believed easily possible that hostilities would be initiated by a surprise attack upon the fleet or the naval base at Pearl Harbor.

  • Failure to Heed Stark’s Warning:

    • Stark issued warnings and suggested torpedo nets.
    • Challenges to maintaining vigilance: Daily emergencies, distractions, interference with training.
    • Perceived implausibility of the threat: Belief that Pearl Harbor’s shallow depth prevented torpedo attacks.
    • Failure to communicate intelligence about Japanese shallow-water torpedoes.
    • Stark removed as CNO in March 1942 and reassigned to London.

  • Yamamoto and Stark’s Shared Insight:

    • Anticipated war between Japan and the United States.
    • Recognized U.S. naval superiority.
    • Understood Taranto as an analog for a potential attack on Pearl Harbor.

  • Connecting the Dots:

    • Yamamoto, Stark, and Chalfie demonstrate the process of receiving new information and grasping its implications.
    • Connecting new data to existing knowledge to gain insights and guide action.

Broccoli and Goldfish: Infant Empathy

  • Alison Gopnik’s Insight:
    • Developmental psychologist at UC Berkeley.
    • Focus: Infant cognition.
    • Belief: Babies possess greater cognitive abilities than commonly assumed.
    • Theory of Mind: Understanding that others have their own minds and preferences.
    • View: Children learn through theory formation, similar to scientists.
    • Rejection of the idea of babies as “defective grown-ups.”
    • Observation: Babies are more attuned to unexpected events than adults.
    • The Philosophical Baby (2009): Explores infant capabilities, including empathy.
  • Challenging Existing Beliefs about Empathy:
    • Previous belief: Children develop perspective-taking around age seven.
    • Gopnik’s hypothesis: De-centering (perspective-taking) occurs much earlier.
  • The Pineapple Incident:
    • Gopnik’s son’s reaction to pineapple in Kirsch inspired an experiment.
    • Son’s statement: “Mommy, pineapple is yummy for you, but it’s yucky for me.”
  • The Broccoli and Goldfish Paradigm:
    • Experiment with 14- and 18-month-old babies.
    • Presented with broccoli and goldfish crackers.
    • Researcher Betty Reticoli expressed preference (happy or disgusted face) for each food.
    • 18-month-olds offered the food Reticoli preferred, even if it was broccoli.
    • 14-month-olds offered goldfish regardless of Reticoli’s expressed preference.
  • Results and Implications:
    • 18-month-olds demonstrated empathy, while 14-month-olds did not.
    • 18-month-olds did not respond egocentrically or selfishly.
    • Opened new avenues for research on infant cognition.
  • Gopnik’s Connection Insight:
    • Similar to Yamamoto, Gopnik connected a new idea (her son’s statement) to her existing work.
    • Led to a new understanding and expanded research possibilities.

The Mother of All Scientific Insights: Darwin’s Theory of Evolution

  • Charles Darwin’s Journey:
    • Age 22: Embarked on the HMS Beagle voyage to chart the South American coast.
    • Background: Cambridge graduate, intended to become an Anglican parson.
    • Family background: Freethinkers, attended Unitarian chapel.
    • Interest in natural history.
    • Influenced by Anglican boarding school after mother’s death.
    • Indifferent to medical studies at the University of Edinburgh.
    • Switched to Cambridge, focusing on natural history.
    • Follower of botany professor John Henslow.
    • Joined HMS Beagle as an unpaid naturalist.
    • Five-year expedition (1831-1836).
    • Observations: Geography, animal species, various phenomena.
    • Key observations:
      • Seashells on a cliff in Cape Verde.
      • Fossil bones of extinct mammals in Patagonia.
      • Variations in mockingbirds across the Galapagos Islands.
  • The Question of Variation:
    • Darwin pondered the driving force behind variations in species.
    • Awareness of deliberate cultivation of variations by farmers and breeders.
  • Influence of Malthus:
    • September 1838: Darwin read An Essay on the Principle of Population by Thomas Robert Malthus.
    • Malthus’s argument: Population growth leads to competition for resources.
    • Darwin’s connection: This competition could explain variations in species.
  • Natural Selection:
    • Advantageous variations are selected in the competition for resources.
    • Individuals with advantageous traits are more likely to survive, reproduce, and pass on their traits.
    • Unfavorable variations are eliminated.
    • Nature automatically selects advantageous traits, similar to farmers’ deliberate selection.
  • Theory of Natural Selection:
    • Based on blind variation and selective retention.
    • Blind variation: Natural variability in physical features within a species.
    • Selective retention: Variations increasing survival value are passed on to offspring.
    • 1871: Darwin added sexual competition and selection to his theory.
  • Alfred Russel Wallace’s Independent Discovery:
    • British naturalist working in the Malay archipelago.
    • Developed the same theory of evolution after reading Malthus.
  • Darwin and Wallace’s Shared Insight:
    • Both wondered about species change, primed by Malthus’s ideas.
    • Connected the concept of competition for resources to evolution.
    • First-hand experience as naturalists, observing variations within and between species.
    • Recognized individual differences as the basis for blind variation and selective retention.

The Connection Strategy and Its Limitations

  • Connection Strategy: A dominant pattern in insights, involving connecting new information with existing knowledge.
    • Accounted for 98 out of 120 cases (82%).
    • Aligns with Graham Wallace’s concept of unconscious associations bursting into awareness.
    • Suggests insights can be increased by exposure to diverse ideas.
  • Reservations about the “Connecting the Dots” Metaphor:
    • Oversimplifies the process of making sense of information and gaining insights.
    • Ignores the role of “non-dots” (irrelevant information) and “anti-dots” (contradictory information).
  • Non-Dots and Anti-Dots in the Taranto-Pearl Harbor Connection:
    • Non-dots: Irrelevant messages.
    • Anti-dots: Differences between Taranto and Pearl Harbor that could have blocked the insight.
      • Distance and logistics: Japanese fleet needed to travel 4,000 miles, requiring refueling and a larger convoy.
      • Weather: Japanese fleet vulnerable to unpredictable weather during the long journey.
      • Defenses: Pearl Harbor had stronger defenses than Taranto, including U.S. aircraft carriers.
      • Time of attack: Japanese attack had to be during daytime, increasing the risk of detection.
      • Reconnaissance: U.S. ceased reconnaissance missions, unknowingly benefiting the Japanese.
      • Torpedo nets: U.S. chose not to deploy torpedo nets due to the perceived shallowness of Pearl Harbor.
      • Shallow water: Japanese torpedoes were initially unsuitable for shallow water.
  • Significance of Yamamoto and Stark’s Insight:
    • Recognized the fundamental lesson of Taranto: Naval fleets were vulnerable to air attacks.
    • Understood the vulnerability of concentrated naval forces.
  • Beyond Connecting the Dots:
    • The connection strategy involves changing the way we think.
    • Other insight strategies (contradictions, creative desperation) also play a role.

Multiple Insight Strategies at Play

  • Coding for Insight Strategies:
    • Each incident coded for five strategies: connections, coincidences, curiosities, contradictions, and creative desperation.
    • Most insights involve a blend of strategies.
  • Connection Strategy Statistics:
    • 98 cases involved connections.
    • Only 45 cases relied solely on the connection strategy.
    • 53 cases involved connections plus other strategies.

Conclusion: The Investigative Process

  • Insights are not simply about connecting the dots.
  • Like a mystery story, the investigation continues beyond the first likely suspect.

Chapter 4: Coincidences and Curiosities

Introduction to Coincidences

  • When we observe a coincidence, we question whether it’s random chance or a deeper pattern.

  • A coincidence is the observation of seemingly related events without an obvious causal link.

  • Coincidences are often chance occurrences that can be ignored, but sometimes they signal a new pattern.

  • Example: Michael Gottlieb and the emergence of AIDS

    • Gottlieb encountered multiple patients with compromised immune systems.
    • Instead of dismissing it as chance, he investigated further.
    • The coincidence of similar symptoms, combined with the patients all being gay, led to the identification of a new pattern: AIDS.

  • People who notice coincidences, trends, and irregularities are valuable, even if often wrong.

  • Their observations should be considered, as they might lead to discoveries.

  • Klein initially wasn’t sure how to interpret coincidences in his research.

  • Coincidences disrupted initial stories but eventually led to better ones.

  • These incidents were classified as insights because they initiated a path towards a better understanding.

Jocelyn Bell Burnell and the Discovery of Pulsars

  • Jocelyn Bell Burnell, an Irish astrophysicist, turned a coincidence into a major discovery.

  • Early exposure to astronomy through her father, the architect of Armagh Observatory.

  • Chose radio astronomy to avoid all-night data collection.

  • As a graduate student at Cambridge University, she built a radio telescope.

  • Data output was on pen recorders, creating squiggly lines on chart paper.

  • Her thesis project aimed to find quasars, distant objects emitting high energy.

  • The Coincidence:

    • Noticed a small, unusual squiggle (quarter-inch out of 400 feet of chart paper) on the data recorder.
    • Initially marked with a question mark, but her brain recognized a recurring pattern.
    • To analyze the signal, she had to run the paper at high speed for short periods.
    • After a month, she captured a detailed trace revealing regular pulses.

  • Investigating the Coincidence:

    • Concerned about equipment malfunction, but technicians found no issues.
    • The signal pulsed regularly every 1.339 seconds.
    • Later, she found a similar signal in a different part of the sky, pulsing every 1.25 seconds. This confirmed the pattern was not an anomaly.
    • Discovered more examples, confirming her discovery of pulsars, rapidly spinning neutron stars, remnants of supernova explosions.

  • Nobel Prize Controversy:

    • Her advisor, Antony Hewish, was listed as the first author on the publication and received the Nobel Prize in 1974.
    • Burnell’s contribution was acknowledged through other awards, but her omission from the Nobel Prize sparked criticism.
    • A prominent astrophysicist acknowledged her discovery as “the greatest astronomical discovery of the 20th century.”
    • Burnell remained diplomatic about the omission.

  • The Importance of Coincidence:

    • Burnell’s insight stemmed from noticing and investigating a coincidence.
    • Like Gottlieb, she recognized the potential significance of the coincidence.

  • Coincidences as Guides:

    • Coincidences can guide the search for evidence and change understanding, focus, and actions.
    • Burnell adjusted her schedule to capture the bursts in more detail.

Defending the Defender: Denver Broncos vs. Green Bay Packers

  • Super Bowl XXXII (1998): Denver Broncos vs. Green Bay Packers.

  • Denver coaches noticed a coincidence in game films: Green Bay defensive back Leroy Butler consistently disrupted plays, seemingly appearing out of nowhere.

  • The Coincidence: Butler’s frequent presence in crucial moments.

  • Denver’s Strategy: Prioritized blocking Butler using various players (tackle, fullback, wide receiver).

  • Outcome: Terrell Davis (Broncos running back) had a successful game, winning MVP. Butler still made tackles but was less disruptive.

  • The Broncos’ winning strategy was based on the coincidence of Butler’s disruptive presence.

Coincidence vs. Connection Insights

  • Connection Insights: New information provides crucial details (e.g., Chalfie, Yamamoto, Gopnik examples). The specific details are essential for the insight.
  • Coincidence Insights: Repetition is key. The specific details of individual instances are less important than the recurring pattern.
  • Example: If Gottlieb’s first patient hadn’t shown up, the subsequent patients would still have revealed the AIDS pattern.

Curiosities

  • Curiosities represent another insight strategy.

  • Insights sparked by single events or observations that provoke a “What’s going on here?” reaction.

  • Example: Alexander Fleming and Penicillin

    • Fleming observed a halo of destruction around a mold contaminating a Staphylococcus bacteria culture.
    • The bacteria near the mold were destroyed, while others grew normally.
    • This unexpected observation (“That’s funny”) led to the discovery of penicillin, the first antibiotic.

  • Curiosity vs. Coincidence: Curiosities are triggered by single events, while coincidences involve repeated patterns.

  • Example: Wilhelm Röntgen and X-rays

    • Noticed a barium platinocyanide screen glowing despite a cardboard covering blocking light from his cathode ray apparatus.
    • Investigated the unexpected glow and discovered X-rays, a new form of light.
    • Initially met with disbelief, but Röntgen eventually received the first Nobel Prize in Physics.

  • Example: Russell Ohl and Transistors

    • Observed a significant current jump when using a cracked piece of silicon exposed to light.
    • Investigated the anomaly and discovered the impact of impurities on silicon’s electrical conductivity.
    • This led to the development of transistors and diodes, and later, silicon solar cells.

The Danger of Coincidence

  • Humans are naturally attuned to coincidences and associations, sometimes overly so.
  • We can perceive connections that don’t exist, leading to skepticism about coincidences.
  • Insight requires discerning potentially meaningful coincidences from spurious ones.
  • Successful individuals like Burnell, Gottlieb, and the Broncos coaches identified coincidences with significant implications.
  • Their expertise allowed them to recognize atypical events with potential consequences.
  • We are prone to noticing coincidences, even spurious ones.
  • It is important to test coincidences before accepting them. Collect evidence to verify that they aren’t spurious.
  • If the test fails, discard the coincidence.
  • This cautious approach is characteristic of individuals with a “down-arrow” attitude, who have low tolerance for mistakes.
  • However, over-reliance on testing can be problematic, as evidence can be flawed.

Barry Marshall and the Helicobacter pylori Discovery

  • Barry Marshall discovered that H. pylori causes ulcers and stomach cancer, challenging the prevailing belief that stress was the cause.

  • Faced resistance from the medical community, which favored surgical treatments and antacids.

  • The Coincidence:

    • Marshall collaborated with Robin Warren, who had observed a corkscrew-shaped bacterium in patients’ stomachs.
    • Marshall found a former patient of his on Warren’s list, linking chronic stomach pain with the bacterium.
    • Another patient’s stomach pain disappeared after antibiotic treatment for a bacterial infection.

  • Initial Setback and Accidental Discovery:

    • Marshall’s initial study found no H. pylori in the first 30 patients due to a lab error (discarding cultures too early).
    • A lab disruption due to a superbug investigation led to a positive H. pylori culture, revealing the previous error.

  • Confirmation and Continued Resistance:

    • Subsequent tests confirmed the presence of H. pylori in all patients with duodenal ulcers.
    • Despite evidence, the medical community continued to reject Marshall’s findings.

  • Self-Experiment and Eventual Acceptance:

    • Marshall infected himself with H. pylori, developed gastritis, and then cured himself with antibiotics, proving his theory.
    • Media attention, especially through non-scientific publications, helped disseminate his findings.
    • The FDA and NIH eventually acknowledged his research, leading to widespread acceptance.

Walter Reed and the Yellow Fever Discovery

  • Walter Reed investigated yellow fever in Cuba, challenging the prevailing belief that it spread through unsanitary conditions or air.

  • Carlos Juan Finlay’s Mosquito Hypothesis (and its initial rejection):

    • Finlay observed a correlation between the presence of Culex mosquitoes and yellow fever outbreaks.
    • His hypothesis was ridiculed, and his experiments failed because he didn’t account for the incubation period.

  • Walter Reed’s Investigation:

    • Reed learned about a jail cell case where only one prisoner contracted yellow fever despite shared living conditions, suggesting a vector like a mosquito.
    • Henry Rose Carter’s observation of an incubation period in yellow fever outbreaks further supported the mosquito hypothesis.
    • Carter’s research on houses with yellow fever victims showed that visitors became ill only after a certain period, indicating mosquito incubation.

  • Confirmation of the Mosquito Hypothesis:

    • Reed’s assistants, Jesse Lazear and James Carroll, allowed infected mosquitoes to bite them after an incubation period, contracting yellow fever (Lazear fatally).
    • Controlled experiments confirmed that Culex mosquitoes transmitted the disease.

  • Overcoming Obstacles:

    • Reed’s team had to overcome flawed data (Finlay’s study) and a prevailing belief.
    • The mosquito theory lacked plausibility compared to the miasma theory (disease caused by bad air).

The Importance of Challenging Evidence

  • The ulcer and yellow fever examples show how flawed evidence can seemingly disprove a correct insight.
  • Evidence can be affected by unknown variables.
  • While evidence is important, it should not be blindly trusted.
  • Both Marshall and Reed had to challenge existing beliefs and evidence.
  • Coincidence insights often involve contradictions between existing beliefs and new evidence.

Conclusion

  • Coincidences, curiosities, and contradictions can all lead to valuable insights.
  • It’s crucial to be open to exploring these avenues, even when they challenge established knowledge.
  • Evidence is important, but it can be misleading; critical thinking and careful investigation are essential.

Chapter 5: Contradictions

Introduction to Contradictions

  • Contradiction insights trigger the emotional reaction of disbelief (“no way”).
  • This involuntary response occurs when encountering illogical ideas.
  • Analogous to a “tilt” message in a pinball machine.
  • Mental tilt reflexes are the opposite of connections and coincidences.
  • Focus on inconsistencies rather than how ideas fit together.
  • Contradiction insights signal flaws in current understanding, leading to a better story.
  • Examples:
    • Young cop and speeding driver (quick new story).
    • Harry Markopolos and Madoff’s Ponzi scheme (longer process).
  • Contradictions differ from curiosity insights:
    • Curiosities prompt wondering, while contradictions cause doubt (“That can’t be right”).
  • Only one of nine curiosity insights also coded for contradiction.
  • Author initially resisted the concept of contradiction insights, expecting insights to arise from connections or coincidences.
  • Contradictions appeared in over a third of the sample (45/120 cases), often dominating the insight process.
  • The tilt reflex is a powerful trigger.

Banking on Contradictions

  • Investment managers profited from contradiction insights during the 2003-2007 housing bubble.
  • While others invested in subprime mortgages, some responded with disbelief at the dramatic price increases.
  • These investors concluded a housing bubble was about to burst.
  • Other individuals also identified the bubble:
    • Sheila Bair (FDIC head) warned about subprime mortgages in 2006.
    • Joshua Rosner (financial analyst) predicted lower home prices and foreclosures in 2001.
    • Dean Baker (research co-director) described the bubble’s disastrous collapse in 2002.
  • Five cases illustrate different cues triggering the tilt reflex:
Steve Eisman
  • Insider (Wall Street analyst specializing in subprime mortgages).
  • Initially advocated for the subprime market but changed his mind due to shoddy loan practices in the late 1990s.
  • Vindicated by the late 1990s subprime collapse.
  • Expected lenders to become more cautious, but the opposite happened.
  • By 2005, subprime growth continued despite rising interest rates.
  • Discovered lenders had learned to sell risky loans to Wall Street banks, repackaged as bonds, masking the risk.
  • Eisman’s tilt reflex intensified.
  • Investigated ratings agencies, finding their models didn’t account for decreasing housing prices.
  • Started a hedge fund to short the subprime market.
John Paulson and Paolo Pellegrini
  • Outsiders (hedge fund managers).
  • Noticed unsustainable growth rates in the subprime market.
  • Recognized a bubble driven by increasing demand based on expected price increases.
  • Concluded the bubble would collapse even if prices leveled off.
  • Identified the contradiction between market belief in continued price increases and economic forces preventing this.
Michael Burry
  • Outsider (contrarian investor with Asperger’s).
  • Transitioned from medicine to finance.
  • Started a successful hedge fund.
  • Suspicions about the subprime market arose in 2003.
  • Found declining lending standards in 2004, contradicting the belief in ever-growing housing prices.
  • Noted analyst claims: “Home prices have never gone negative” and “Home prices won’t even go flat.”
  • Identified the contradiction between optimism and declining lending standards.
  • Predicted a market decline due to unsustainable subprime mortgages.
  • Estimated a two-year horizon (2007) for the bubble to burst due to increasing interest rates on variable-rate loans.
Greg Lippmann
  • Insider (trader at Deutsche Bank).
  • Instructed to bet against the housing market in 2005.
  • Discovered rising mortgage default rates (1% to almost 4%) despite rising housing prices.
  • Calculated that even slight increases in default rates would render mortgage bonds worthless.
  • Tilt reflex triggered by the contradiction.
  • Recognized the vulnerability of financial markets dependent on continually increasing housing prices.
Gene Park
  • Insider (worked for AIG).
  • Considered investing in New Century (mortgage lender) but found their success based on poor subprime loans.
  • Observed risky investments in mortgage insurance that didn’t account for a potential housing market crash.
  • Tilt reflex triggered by a jobless friend receiving offers for unaffordable houses.
  • Recognized an unsustainable bubble and AIG’s vulnerability.
  • Unable to profit from his insight or convince his bosses of the impending danger.

The Skeptical Mindset

  • The five investors and Harry Markopolos were skeptics.
  • While open-mindedness is valued for discoveries, a suspicious mind can also be beneficial.
  • Skepticism allows investigation of overlooked pathways.
  • Two-thirds of the contradiction insight cases involved individuals with suspicious mindsets.
  • They resisted popular enthusiasms.
Meredith Whitney
  • Wall Street analyst trained by Steve Eisman.
  • Warned about reduced credit standards in 2005, largely ignored.
  • Gained recognition for predicting Citigroup’s financial crisis in 2007.
  • Investigated Bear Stearns in early 2008 despite its seemingly strong asset sheet.
  • Initial optimism faded upon learning about delayed access to Federal Reserve loans for investment banks.
  • Suspected traders were lying about Bear Stearns’ stability.
  • Confirmed the possibility of Bear Stearns’ collapse through another source.
  • Adopted a deliberately skeptical mindset, interpreting data as evidence of Bear Stearns’ impending failure.
  • Identified factors contributing to the potential collapse: contracting businesses, loss of market share, high leverage.
  • Confidently predicted Bear Stearns’ collapse based on her gut feeling and skeptical analysis.
  • Compared this insight to her previous insights about subprime lending and Citigroup.
  • Whitney’s skeptical lens provided a different perspective, enabling her to gain insights missed by others.
  • Acted as a detective, challenging popular beliefs.
  • Similar to the five investors who foresaw the subprime bubble’s end.
  • A suspicious mind offers unique benefits, exposing alternative facts and trends.

The Broad Street Pump

  • John Snow’s cholera discovery illustrates the tilt reflex.
  • Cholera epidemics spread rapidly, causing dehydration and death.
  • Miasma theory (disease spread by bad air) prevailed at the time.
  • John Snow:
    • London physician known for anesthesia expertise.
    • Pioneered safe anesthesia dosage regulation.
    • Administered chloroform to Queen Victoria.
    • His prominence didn’t shield him from criticism for challenging miasma theory.
  • Snow’s interest in cholera sparked by a contradiction:
    • Two individuals contracting cholera in the same lodging room, challenging miasma theory.
    • Tilt reflex activated.
    • Noted other contradictions:
      • Inconsistent pattern of disease transmission.
      • Lack of lung damage in cholera victims, suggesting ingestion as the cause.
  • Insight: Cholera caused by ingestion, not inhalation.
  • Gathered data suggesting contaminated water as the source.
  • Evidence:
    • Cholera outbreak in a slum linked to a contaminated well.
    • Natural experiment with two water companies: “down river” company (contaminated water) had higher cholera rates.
  • Used contradictions and coincidences to support his theory.
  • 1854 cholera outbreak:
    • Cholera cases clustered around the Broad Street pump.
    • Anomalies: brewery workers (drinking beer) didn’t contract cholera.
    • Supporting evidence: woman contracting cholera after drinking water from the Broad Street pump brought by her son.
  • Broad Street pump handle removed, coinciding with the epidemic’s end (though it was already waning).
  • Snow’s theory faced ridicule, but gained acceptance after the Broad Street pump incident.

Contradictions and Insights

  • Contradictions fueled Snow’s and Walter Reed’s insights.
  • Humans are built to detect anomalies and inconsistencies.
  • Contradictions lead to insights (the better story) and can be considered insights themselves.
  • Noticing a contradiction shifts understanding.

Thomas Kuhn and Paradigm Shifts

  • Thomas Kuhn’s The Structure of Scientific Revolutions:
    • Anomalies lead to breakthroughs.
    • Normal science adds detail to existing theories, suppressing contradictions.
    • Discrepancies accumulate, leading to crises and paradigm shifts (new ways of understanding).
    • Paradigm shifts involve paradigm destruction.
    • Paradigm shifts are insights, moving from a mediocre frame to a better one.
  • Author’s critique: Kuhn overemphasizes paradigm shifts and destruction.
  • Kuhn highlights the role of anomalies and contradictions in generating new paradigms.

Seeing the Light

  • Einstein’s theory of special relativity illustrates contradictions leading to insights.
  • Einstein’s thought experiments about light revealed a paradox:
    • Light beam appearing frozen if traveling alongside it.
    • Maxwell’s equations predicting light’s constant speed.
  • Resolved the paradox by changing the constant:
    • Newtonian physics: space and time are constant.
    • Einstein: speed of light is constant, space-time is variable.
  • Implication: Time dilation, length contraction, and mass increase as speed approaches light speed.
  • Einstein, like Snow, Markopolos, and the financiers, used contradictions for discovery.

Conclusion

  • The strategy of using contradictions differs from noticing coincidences, curiosities, and making connections.
  • Klein questioned the ability to formulate a single answer to the mystery of insight.
  • One more strategy remained: the scientific method.

Chapter 6: Creative Desperation

Trapped by Assumptions

  • Some insights are accidental, arising from unplanned circumstances or being in the right place at the right time.
    • Examples: The young cop (mentioned earlier in the book, context not provided in this excerpt), Martin Chalfie’s experience in a lunch seminar (context not provided in this excerpt).
  • Other insights are deliberate, occurring when individuals are stuck and require a breakthrough.
  • Creative desperation: Term coined by Dutch psychologist Adriaan de Groot to describe strategies chess grandmasters employ when facing difficult situations.
    • Occurs when plausible moves fail, leading to unorthodox, potentially risky plays.
  • Scientific research on insights often involves puzzles designed to stump participants.
    • Some participants fail, while others struggle and then experience unexpected insights.
  • Creative desperation is considered the epitome of insightful problem-solving by research psychologists.
    • It is the paradigm used in experiments because it involves finding a solution through a flash of illumination rather than steady analysis.
  • Graham Wallace’s four-stage theory of insight aligns with the concept of creative desperation (details of the four-stage theory not provided in this excerpt).
  • Creative desperation differs from other insight strategies (connecting ideas, noticing coincidences, curiosities, or contradictions) because it requires escaping a seemingly inescapable trap.

Fighting Fire with Fire

  • Creative desperation can occur in life-or-death situations.

  • Mann Gulch Fire (August 5, 1949): Example of creative desperation used by Wagner Dodge.

    • Fifteen smokejumpers parachuted into western Montana to control a forest fire.

    • Twelve died after being trapped by a blow-up: a sudden, unexpected firestorm caused by the collision of fire and wind.

      • A blow-up is to a typical forest fire what a hurricane is to an ocean storm.

    • The smokejumpers planned to descend the north slope of Mann Gulch to the Missouri River and cross to the south slope to attack the fire from below, minimizing risk.

    • R. Wagner Dodge: 33-year-old team leader, described as taciturn.

      • Had nine seasons of experience with the Forest Service, four as a smokejumper foreman.
      • Reassigned from training his crew at the last minute.

    • Dodge noticed smoke at the bottom of the north slope, indicating a spot fire: a new fire started by wind-carried sparks and embers from the main fire.

      • Spot fires can quickly lead to a blow-up.

    • Dodge ordered the team to retreat uphill, but the fire gained speed.

    • Norman MacLean’s book Young Men and Fire: Contains a graph illustrating the smokejumpers’ plight.

      • Graph depicts the race between the fire (dotted line) and the smokejumpers (solid line).
      • Horizontal axis: Time (30 minutes).
      • Vertical axis: Distance above and below point six (where Dodge first saw smoke).
      • Smokejumpers descended to point six, reversed course at 5:45 PM.
      • Dropped tools at 5:53 PM.
      • Fire reached point six at 5:49 PM (19 minutes after spotting).
      • Fire’s speed increased with the valley’s steepness, reaching 660 feet per minute.
      • Final slope: 76%.

    • MacLean’s reflection on being caught in a forest fire:

      Burning to death on a mountainside is dying at least three times, not two times, as has been said before. First, considerably ahead of the fire, you reach the verge of death in your boots and your legs. Next, as you fail, you sink back in the region of strange gases and red and blue darts where there is no oxygen, and here you die in your lungs. Then you sink in prayer into the main fire that consumes. And if you are a Catholic, about all that remains of you is your cross.

    • Author’s visit to Mann Gulch:

      • Part of a team investigating the 1994 South Canyon fire in Colorado.
      • Attempted running up the 76% slope and found it exhausting.

    • Slowest smokejumper caught at 5:56 PM (11 minutes after reversing course).

    • Dodge’s escape: Lit an escape fire in front of him and took refuge in its ashes.

      • Protected his mouth and nose with a wet handkerchief.
      • Others didn’t understand his actions.
      • Survivor’s quote: “We thought he must have gone nuts.”

    • Author’s hypothetical reconstruction of Dodge’s thinking:

      • Four Anchors (beliefs):
        • Uphill slope favoring the fire.
        • Fire gaining speed behind him.
        • Need for an island of safety (ridge top or rocky patch).
          • Rocky area 200 yards uphill, unreachable in time.
        • Dry grass (fuel) as the fire’s sustenance.
      • Dodge focused on neutralizing the fuel by burning it with the escape fire.

Disarming a Flawed Assumption

  • Dodge’s testimony didn’t explain his insight.
  • Aron Ralston (May 2003): Analogous case of creative desperation.
    • Trapped by a boulder pinning his right arm in a Utah canyon.
    • Trapped for over five days, recounted in his book Between a Rock and a Hard Place.
    • Movie adaptation: 127 Hours, starring James Franco.
    • Initial attempts to chip away the rock failed, dulling his knife.
    • Discovered trapped arm had died due to lack of circulation.
    • Attempted amputation but knife was too dull.
    • During an emotional outburst, he felt his arm bend unnaturally.
      • Realized he could break the bones rather than cut them.
      • Snapped the bones, cut through the tissue and nerve, and freed himself.
  • Both Dodge and Ralston shifted their focus:
    • Dodge: From the fire to the fuel.
    • Ralston: From freeing his arm to freeing himself from the dead arm.
  • Both used the source of their entrapment (fire, boulder) as a tool for escape.

Cheryl’s Kisses

  • Cheryl Kane: Financial manager at Klein’s research company.
  • Problem: Employees not filling out time cards.
    • Fear of government audit due to government funding.
    • Directives ineffective.
  • Cheryl’s mother’s suggestion: Reward instead of harass.
  • Solution: Hershey’s Kisses as rewards for timely time card submission.
    • Resulted in increased compliance and proactive communication.
  • VIP Treatment: Visible, Immediate, and Personal incentive.
  • Cheryl’s initial trapping assumption: Necessity of cajoling and threats.

Other Examples of Creative Desperation

  • David Charlton (Corning Incorporated):
    • Faced criticism from chemical engineers about a new reactor vessel design due to heat transfer concerns.
    • Noticed the thin walls and small internal volumes of the reactor.
    • Hypothesized that historical data on heat transfer in larger reactors wouldn’t apply.
    • Tests proved him correct.
    • Overturned the assumption that the new material was unsuitable.
  • Napoleon Bonaparte (1793, Siege of Toulon):
    • Faced a stronger Anglo-Spanish force occupying Toulon, a crucial French port and naval arsenal.
    • The French Republic needed to repel the invaders to maintain control of the seas and discourage further revolts.
    • Napoleon, an artillery captain, noticed two lightly guarded forts overlooking the harbor (l’Éguillette and Balaguier).
    • Realized he could use these forts and light artillery to disrupt the invaders’ resupply route.
      • Inspired by American tactics during the Revolutionary War (Dorchester Heights, Yorktown).
    • Initial resistance from French commander Jean-François Carteaux, who was fixated on capturing Toulon by force.
    • Carteaux’s dismissal allowed Napoleon to implement his plan, leading to the British withdrawal.
    • Napoleon’s insight: Didn’t need to defeat the invaders, just force them to leave by cutting off supplies.

Conclusion

  • Creative desperation involves actively searching for insights by re-examining assumptions.
  • 29 out of Klein’s 120 cases (nearly a quarter) fit this category.
  • Klein questions how to reconcile these cases with other types of insights.

Chapter 7: Different Ways to Look at Insight

Investigating the Origin of Insights

  • Klein explores five potential sources of insight: connection, coincidence, curiosity, contradictions, and creative desperation.
  • Connection is the most frequently observed theme, while creative desperation aligns with how scientists view insight.
  • Several investigation methods are used:
    • Analyzing coded data
    • Reviewing scientific literature on insight
    • Examining the stories themselves

Looking at the Data

  • Klein and Andrea Gerosa coded data from collected cases.
  • All five insight explanations emerged in the data:
    • Connection: 82%
    • Contradictions: 38%
    • Coincidences: 10%
    • Curiosities: 7.5%
    • Impasses/Creative Desperation: 25%
  • The percentages exceed 100% because some cases involved multiple themes.
  • The data challenged some existing beliefs about insight:
    • Open-mindedness: Two-thirds of contradiction insights arose from suspicion, not open-mindedness.
    • Preparation: While most (98/120) insights were deliberate, 18% were accidental, contradicting Graham Wallace’s emphasis on preparation.
      • Examples: Jocelyn Bell Burnell’s discovery of pulsars while looking for quasars, and Martin Chalfie learning about green fluorescent protein.
    • Incubation: Only 5/120 cases clearly involved incubation. In 39%, it was impossible (e.g., Wagner Dodge escaping a wildfire). Over half the remaining cases lacked sufficient detail to determine incubation’s role. Klein concludes incubation is not essential for insight.
  • Sudden vs. Gradual Insights:
    • 56% of cases were sudden “aha” insights.
    • 44% were gradual, challenging the view that suddenness defines insight.
      • Klein argues insight is not synonymous with “aha,” just as conception is not the same as orgasm.
    • Gradual Insight Examples:
      • Coincidences: Michael Gottlieb’s identification of AIDS through a gradual accumulation of similar patient cases. Denver Broncos coaches realizing they needed to stop Green Bay Packers linebacker Leroy Butler after observing him disrupt play repeatedly. Jocelyn Bell Burnell’s recognition of pulsars after multiple encounters with the same signal.
      • Incremental Progress: Revolutionary technologies like Gutenberg’s printing press and Ford’s mass production involved solving multiple problems gradually, not a single inspiration. Stephen Johnson’s concept of “slow hunches” is cited.
      • Deliberate Searches: Meredith Whitney’s investigation of Bear Stearns’ financial troubles.

The Lucky Punch: A Case Study in Gradual Insight

  • Klein analyzes Floyd Mayweather Jr.’s knockout punch against Ricky Hatton in a 2007 boxing match.
  • Initial impression: a lucky punch.
  • Slow-motion replay reveals Mayweather deliberately timed his punch, exploiting Hatton’s tendency to drop his right hand when throwing a left hook.
  • Mayweather likely studied Hatton’s fight tapes and then refined his timing in the ring.
  • Hatton threw his signature left hook 18 times, 10 of which were in Mayweather’s corner, where the knockout occurred. Mayweather only countered with his own left hook in that corner.
  • Mayweather tested Hatton’s tendencies throughout the fight, observing his reactions to jabs and hooks.
  • The knockout was the culmination of a gradual process of observation, analysis, and timing. Klein’s own understanding of the fight unfolded gradually, with no single “aha” moment.

Looking at the Scientific Literature

  • Klein intentionally delayed reviewing existing research to maintain fresh perspective.
  • After data collection and categorization, the literature review began.
  • “The Nature of Insight” (1995) is highlighted as a valuable resource, though it revealed disagreements among researchers.
  • Debates included the role of the “aha” experience, cognitive restructuring, and the usefulness of the insight concept itself.

Insights and Decision Biases

  • Klein explores the relationship between insights and decision biases, systematic deviations from rational judgment.
  • Example: Overestimating car crash deaths compared to stroke deaths due to media exposure.
  • Insights are viewed as a counterpoint to biases.
  • Heuristics and Biases Movement: Founded by Danny Kahneman and Amos Tversky, focusing on cataloging and explaining decision biases.
    • Kahneman won a Nobel Prize for this work.
  • System 1 and System 2 Thinking:
    • System 1: Fast, intuitive thinking (source of biases).
    • System 2: Slower, critical, analytical thinking (monitors and corrects System 1).
  • The heuristics and biases research emphasizes strengthening System 2 (downward arrow in performance diagram - reducing errors).
  • Klein argues for balancing this focus with an appreciation for insights (upward arrow - gaining new understanding).

Looking at the Stories

  • Klein draws parallels to previous research on firefighter decision-making, where analyzing stories revealed a recognitional model of decision-making, challenging the assumption of option comparison.
  • Klein has used story-based methods successfully in other research, leading to the development of naturalistic decision research.
  • Naturalistic Methods: Involve searching for meaningful patterns in stories, without predetermined hypotheses or analysis methods.
  • This approach is suited to early stages of scientific exploration.
  • Klein aims to find answers within the 120 collected insight stories.

Synthesizing the Five Themes

  • The data coding revealed five insight themes: connections, coincidences, curiosities, contradictions, and creative desperation.
  • Klein seeks to understand how these themes relate and potentially combine.
  • Key examples for further exploration include:
    • Financial experts predicting the 2007-2008 financial crisis (contradiction).
    • John Snow’s discovery of cholera transmission.
    • Einstein’s development of special relativity.
    • Martin Chalfie’s work with green fluorescent protein.
  • Klein’s working definition of insight is “an unexpected shift to a better frame.”
  • The concept of a frame with data-fitting slots is introduced as a potential avenue for understanding insight.
  • Klein acknowledges the vagueness of these ideas but hopes they will lead to a new model of insight.

Chapter 8: The Logic of Discovery

Introduction

  • Klein noticed opposing insight strategies while reviewing stories: creative desperation and contradiction.
  • Difficulty synthesizing these seemingly reverse activities.

Creative Desperation

  • Triggered by creative blocks, seeking to escape fixation.
  • Involves identifying and discarding a weak belief trapping the individual.
  • Examples:
    • Wagner Dodge: Setting backfire to eliminate fuel for the forest fire.
    • Aron Ralston: Using boulder leverage to break arm bones instead of cutting.
    • Cheryl Kane: Using kisses instead of threats for time card compliance.
    • Napoleon at Toulon: Threatening supply lines instead of direct assault.

Contradiction

  • Focuses on embracing the weak belief or anomaly.
  • Using the contradiction to reconstruct the narrative.
  • Examples:
    • John Snow: Cholera spread by water, not air (miasma theory). Contrary evidence:
      • Uneven cholera incidence despite shared air.
      • Healthy lungs but damaged digestive tracts in autopsies.
    • Albert Einstein: Special relativity. Contradiction:
      • Two light beams and the speed of light limit.
      • Resolution: Revising Newtonian concepts of constant space and time.

The Contradiction Path Script

  • Encounter an anomaly.
  • Resist discarding the anomaly.
  • Accept the anomaly as valid.
  • Revise existing beliefs to accommodate the anomaly.
  • Discard previous anchoring beliefs if necessary.
  • Examples: Abandoning fixed space-time or miasma theory.
  • Achieve a new frame incorporating the anomaly.
  • Data: 45 contradiction insights triggered by anomalies/inconsistencies. In 42 cases, the individual explored the anomaly.

The Investors and the Financial Bubble

  • Investors (Eisman, Paulson, Burry, Lipman, Park) experienced contradiction:
    • Conflicting views on housing market future.
    • Their data contradicted prevailing optimism.
  • Anchored on their anomalous data, despite clashing with Wall Street consensus.
  • Recognized the contradiction as an opportunity.

Contradiction vs. Creative Desperation

  • Both focus on outliers, weakly supported beliefs.
  • Creative desperation: Attacking and discarding the weak anchor.
  • Contradiction: Embracing and building upon the weak anchor.

Two Distinct Insight Paths

  • Motivations: Escaping a bad situation (desperation) vs. rethinking conventional wisdom (contradiction).
  • Triggers: Searching for a flawed assumption (desperation) vs. encountering an inconsistency (contradiction).
  • Activities: Replacing the flawed assumption (desperation) vs. building on the weak assumption (contradiction).
  • Similarities:
    • Disruptive to existing beliefs.
    • Modification of core beliefs (anchors).
    • Outcome: Changes in understanding, actions, perspectives, feelings, and desires.

Triple-Path Model of Insight

  • Three paths: Contradiction, Connection, and Creative Desperation.
The Contradiction Path
  • Trigger: Contradiction/Inconsistency.
  • Activity: Rebuild story using the weak anchor.
The Connection Path
  • Trigger: Coincidence, Curiosity, Spotting an implication.
  • Activity: Add a new anchor.
The Creative Desperation Path

  • Trigger: Impasse, need to escape.

  • Activity: Discard a weak anchor.

  • Outcome (for all paths): Changes in understanding, actions, perspectives, feelings, and desires.

The Connection Path

  • Differs from desperation and contradiction paths.
  • Activity: Adding a new anchor (not attacking or building on existing weak anchors).
  • New anchor often comes from new information.
  • Examples:
    • Martin Chalfie: Green fluorescent protein + translucent worms = new research direction.
    • Charles Darwin: Malthus’s competition theory = new anchor for evolution theory.
    • Admiral Stark & Yamamoto: Battle of Taranto = new anchor for naval strategy (aircraft carrier attacks).
    • Alison Gopnik: Son’s “yucky-slash-yummy” comment = new anchor for infant intelligence research.
  • Coincidences and curiosities as triggers.
    • Carlos Finlay: Yellow fever and Culex mosquito coincidence.
  • Connection path combines connections, coincidences, and curiosities.
  • Builds a new story without discarding existing anchors.
  • Different motivation, trigger, and activity, but the outcome is still a shift in understanding.
  • This shift involves changing core components (anchors).

Blending Paths

  • Multiple paths are common in insights.
  • Example: Yellow fever discovery:
    • Carlos Finlay: Coincidence (mosquitoes and yellow fever).
    • Walter Reed: Contradiction (miasma theory vs. uneven infection rates).
    • Walter Reed’s team: Connection (mosquito incubation period).
  • Connection path often involved in blends.

Conclusion

  • Triple-path model: A plausible explanation for insights.
  • More comprehensive and useful than previous accounts.
  • Future work may uncover additional paths.
  • Earlier accounts are not wrong, but incomplete, focusing on single paths:
    • Wallace: Creative desperation.
    • Associations/combinations: Connection.
    • Problem reformulation/restructuring: Contradiction.
  • Triple-path model clarifies why researchers seem to talk past each other (different paths).

Part II: Shutting the Gates

Chapter 9: Stupidity

Introduction: Fantasy Baseball and an Unexpected Insight

  • Klein recounts an incident during a fantasy baseball pennant race in September 2003.
  • Fantasy baseball context:
    • Managers select real baseball players for their virtual teams.
    • Teams score points based on players’ real-game performance (wins, strikeouts, etc.).
    • Season mirrors the regular baseball season (April to September).
  • Klein’s daughter, Devorah, was playing in her first season.
  • She was concerned about exceeding the league’s 1,250-inning limit for pitchers.
  • With 1,249 innings pitched, she had one pitcher scheduled for the final day (Sunday).
  • Question: Would her team get credit for all his strikeouts or only those in the first inning before reaching the limit?
  • Devorah located the rules online.
  • The rules stated that all pitcher statistics counted on the day the limit was exceeded, but none thereafter.
  • Devorah had an insight:
    • Understanding: She didn’t need to worry about exceeding the limit on Sunday.
    • Action: She could add as many pitchers as possible who were playing on Sunday before midnight.
  • Devorah dropped inactive pitchers and replaced them with free agents scheduled to pitch on Sunday.
  • Example: Dropping Roger Clemens for Matt Clement.
  • Devorah’s strategy worked; she gained enough points to finish fifth.
  • The tactic was novel in their league but known elsewhere.
  • Klein, despite having the same information, missed the insight.
  • This raises questions about what causes “stupidity” and blocks insights.

Stupidity in Action: Everyday Examples

  • Example 1: Misplaced Car Keys
    • Author put car keys in briefcase before a flight.
    • Daughter’s ear infection led to a train journey home (New York to Toledo).
    • Mother-in-law picked them up in Toledo and drove them to the Dayton airport.
    • Author left briefcase in mother-in-law’s car, forgetting the keys were inside.
    • Author knew the location of the keys but failed to connect the information in time.
    • Result: Rented a car, extra expense, inconvenience.
  • Example 2: The Staircase Fall
    • A colleague left mail (including glossy materials) on the bottom stair.
    • After several hours upstairs, she descended in the dark.
    • She knew the stairs well but forgot about the mail.
    • Slipped on the glossy mail and suffered a compound leg fracture, requiring 16 weeks in bed.
  • Example 3: Missed Opportunity
    • Author wanted to show his wife a storage locker.
    • An errand took them past the facility.
    • Only later did he realize he should have brought the locker keys.
  • Analysis of Stupidity Examples:
    • These are failures to make obvious connections or spot contradictions.
    • They are not memory failures, as the information was accessible.
    • A more active process of “spotlighting” relevant information is missing.

Stupidity vs. Insight: A Continuum

  • Stupidity seems the opposite of insight on the contradiction and connection pathways.
  • Insight: Spotting subtle inconsistencies and contradictions.
  • Stupidity: Missing obvious connections and contradictions.
  • Examples of stupidity highlight the everyday use of insight strategies.
    • We constantly make connections, seek implications, and identify inconsistencies.
  • We don’t credit ourselves for noticing the obvious but criticize ourselves for missing them.
  • Stupidity: Failing to make obvious connections, missing anomalies and inconsistencies, clinging to wrong assumptions.
  • Insight and stupidity can be viewed as opposite poles of a continuum, with normal alertness in between.

The Stupidity File: False Insights

  • Klein collected over 20 examples of stupidity alongside the 120 insight cases.
  • False insights: Mistakenly believing one has had an insight, with full confidence in the erroneous conclusion.
  • False insights in financial bubbles:
    • Investors aren’t just driven by greed; they often have false insights about transformative changes.
    • Examples:
      • Japanese stock and real estate bubble (late 1980s): Japan’s supposed economic dominance.
      • U.S. dot-com bubble (late 1990s): The “new paradigm” of information technology.
      • U.S. housing bubble (early 2000s): Advanced risk management tools.
  • Klein acknowledges the need for further study of stupidity but chooses to focus on obstacles to insight.

Obstacles to Insight: The Huxley Question

  • T.H. Huxley’s reaction to Darwin’s theory: “How extremely stupid not to have thought of that.”
  • This isn’t about stupidity but about missing obvious insights.
  • The question: What blocks these insights?
  • Laboratory research using puzzles (water jars, nine dots, pendulum) allows for controlled comparisons of success and failure.
  • Klein’s diverse examples lacked this control.
  • Could a “natural experiment” be designed?

Chapter 10: The Study of Contrasting Twins

Introduction

  • Klein investigates why people fail to have insights even with all necessary information.
  • Analysis of 120 cases, including 30 “contrasting twins”: pairs of individuals with the same information where one gains insight and the other does not.
  • Selection criteria: the twin without insight must be a specific, identifiable person in the story (not an anonymous bystander).
  • Examples of contrasting twins:
    • Wagner Dodge (escape fire insight) vs. Robert Sally and Walter Rumsey.
    • Napoleon (Toulon strategy) vs. General Jean-Francois Carteaux.
    • John Snow (cholera/water) vs. Edwin Chadwick and William Farr (miasma theory).
    • David Charlton (heat transfer) vs. his materials engineering team.
    • Devorah (fantasy baseball) vs. Klein.

Four Reasons for Missed Insights

  • Flawed Beliefs: Fixation on erroneous ideas.
  • Lack of Experience: Insufficient experience to recognize implications.
  • Passive Stance: Lack of active scanning for opportunities and new developments.
  • Concrete Reasoning Style: Difficulty tolerating contradictions and ambiguity, hindering playful exploration of ideas.

Flawed Beliefs

  • Most common reason (21/30 cases): failure twins fixated on incorrect ideas.
  • Example: Medical establishment’s acceptance of miasma theory for cholera, rejecting John Snow’s contaminated water theory.
  • Successful twins: “speculate and test.”
  • Failure twins: cling to flawed beliefs.
Cuban Missile Crisis Example
  • John McCone (CIA head): Insightful twin.
    • Warned of potential Soviet ballistic missiles in Cuba.
    • Suspicious of Soviet activity after Bay of Pigs incident.
    • U-2 photos showed Soviet SAMs (surface-to-air missiles).
    • McCone’s reasoning: SAMs suggest something valuable to hide (ballistic missiles).
    • Intensified surveillance, even during honeymoon.
    • Proven correct by history.
  • Sherman Kent (chief CIA analyst): Failure twin.
    • Believed Soviet Union wouldn’t provoke nuclear conflict due to U.S. military superiority.
    • Disagreed with McCone.
    • Later justified himself by claiming Khrushchev was irrational.
  • Author’s assessment: Kent was gripped by a flawed theory, while McCone was more skeptical and data-driven.
Yom Kippur War Example
  • Major General Eli Zeira (Israeli Director of Military Intelligence): Failure twin.
    • Fixated on the belief that Egypt wouldn’t attack without air superiority.
    • Dismissed evidence of Egyptian military buildup as a training exercise.
    • Ignored warnings from subordinates, reserve generals, King Hussein of Jordan, and an Israeli spy within Egyptian ranks.
    • Stilled warnings to Israeli leaders even two hours before the attack.
  • Junior Israeli intelligence analysts: Insightful twins.
    • Recognized the threat posed by the Egyptian buildup and lack of actual training activities.
    • Not fixated on predicting Sadat’s motivations; focused on objective data.
Flawed Data
  • Clinging to flawed data can also hinder insight.
  • Example: David Charlton’s skepticism of data saved Corning’s composite project, while engineers trusting the flawed data raised a false alarm.
  • Other examples:
    • Walter Reed initially told to ignore the (correct) mosquito theory.
    • Barry Marshall’s initial data showed no link between gastritis and Helicobacter pylori due to premature sample discarding.
The Importance of Core Beliefs
  • Core beliefs anchor understanding and are resistant to change.
  • Anomalies are explained away rather than used to revise beliefs.
  • Scientists explain away anomalies by:
    • Questioning data collection methods.
    • Deeming data irrelevant.
    • Identifying other possible causes.
    • Making cosmetic changes to their theory.
  • Physicians also exhibit “knowledge shields” to protect initial (sometimes incorrect) diagnoses (Paul Feltovich).

Lack of Experience

  • Second reason for missed insights.
  • Examples:
    • Napoleon’s artillery experience allowed him to see opportunities others missed.
    • John Snow’s medical experience provided insight into cholera’s effects on the digestive system, contrasting with Chadwick and Farr’s lack of medical expertise.
  • Experience: not just knowledge, but how it tunes attention and sensitizes to cues.
  • Generally prepared mind: Individuals prepared by their interests and efforts to notice what others miss (e.g., Martin Chalfie and the green fluorescent protein).
  • John McCone’s prior concern about Soviet buildup in Cuba gave him a skeptical mindset.
  • Junior Israeli analysts’ “tilt reflexes” triggered by the Egyptian buildup.
  • Flawed beliefs can override experience (e.g., Sherman Kent and Eli Zeira had extensive experience but were hindered by flawed beliefs).

A Passive Stance

  • Third reason for missed insights.
  • Failure twins often passive, not actively seeking new information or questioning assumptions.
  • Active stance present in 21/30 insightful twins.
  • Example: Devorah’s active thinking about the fantasy baseball problem vs. Klein’s passive approach.
Ginger’s Non-Compete Agreement
  • Ginger: Insightful twin.
    • Frustrated by a non-compete agreement preventing her from working with former clients, even though she didn’t know who they all were.
    • Realized the impossibility of complying without a client list, which her previous employer wouldn’t provide.
    • Used the agreement’s internal inconsistency to her advantage.
    • Obtained release from the non-compete clause by requesting the impossible client list.
  • Legal officer at Ginger’s new company: Failure twin.
    • Only interpreted the agreement, didn’t consider its practical implications.

A Concrete Reasoning Style

  • Fourth reason for missed insights.
  • Concrete thinkers: Focus on facts, dislike speculation and ambiguity.
  • Playful thinkers: Enjoy exploring hypothetical scenarios and juggling ideas.
  • Example: Meredith Whitney’s speculation about Bear Stearns’ financial situation.
  • Playful/concrete reasoning style is a personality trait, while active/passive stance can vary situationally.
  • 14/30 failure twins exhibited concrete thinking.
  • Examples: General Carteaux, Chadwick, and Farr.

The Limits of Positive Behaviors

  • Eliminating flawed beliefs, gaining experience, adopting an active stance, and engaging in playful reasoning do not guarantee insight.
  • Many instances of failure despite these positive behaviors.

Intelligence as a Factor

  • Not a significant factor in the contrasting twins study.
  • Most cases required a baseline level of intelligence for involvement.

The Double Helix: Triumphs and Failures

  • Most contrasting twins (24/30) differed on multiple factors.
  • James Watson and Francis Crick: Discovery of DNA’s double helix structure.
  • Multiple contrasting twins within this story.
  • Watson and Crick’s advantages:
    • Shared belief that DNA carried genetic information.
    • Hunch about DNA’s repeating structure (Watson from a lecture).
    • Speculation that DNA was a double or triple helix, inspired by Linus Pauling’s work on protein structure.
    • Model-building approach.
    • Watson’s observation of identical shapes of adenine-thymine and guanine-cytosine pairs.
  • Watson’s eight simultaneous insights:
    • DNA structure (helix).
    • Number of strands (two - double helix).
    • Information carrier (nucleic acids).
    • Attractive force (hydrogen bonds).
    • Arrangement (sugar-phosphate backbone outside, nucleic acids inside).
    • Base pairings.
    • Opposite chain directions (mirror images).
    • Replication mechanism (zipper-like).
The Scientific Community vs. Watson and Crick
  • Scientific Community:
    • Flawed belief: genetic information carried by proteins, not DNA.
    • Dismissed Oswald Avery’s 1944 study showing bacterial DNA carrying genes.
    • Inexperience: leading geneticists lacked biochemistry knowledge, organic chemists studying DNA weren’t interested in genetics.
  • Watson and Crick:
    • Not burdened by the flawed protein belief.
    • Right blend of experience: Crick (physics, X-ray diffraction, proteins, gene function), Watson (biology, phages, bacterial genetics).
    • Benefited from Alfred Hershey’s 1952 discovery that only phage DNA enters the host cell, contradicting the protein belief.
Chargaff vs. Watson and Crick
  • Erwin Chargaff:
    • Discovered equal proportions of adenine/thymine and guanine/cytosine in DNA.
    • Didn’t see the implications of his findings.
    • Passive attitude: content with his research, lacked Watson and Crick’s active inquiry.
  • Watson and Crick:
    • Crick immediately saw the implication of complementary pairing upon hearing Chargaff’s results.
    • Active mindset, especially Crick’s focus on DNA function and replication.
Franklin vs. Watson and Crick
  • Rosalind Franklin:
    • Flawed belief: DNA not a helix (due to flawed data from earlier research).
    • Fixated on disproving the helix idea.
    • Overlooked evidence of helical structure in her own Photo 51 for ten months, while Watson immediately recognized it.
    • Lacked model-building experience, hindering understanding of distances and angles.
    • Passive stance: disliked speculation, preferred data gathering.
    • Concrete reasoning style.
  • Watson and Crick:
    • Open to the helix idea.
    • Model-building approach.
    • Active and playful reasoning styles.
Watson vs. Crick
  • Watson:
    • Initially didn’t grasp the significance of Chargaff’s findings or Crick’s insight about complementary pairing.
    • Fixated on like-with-like pairing.
    • Passive thinking on this specific issue.
    • Focused on DNA structure rather than function.
  • Crick:
    • Immediately saw the implication of complementary pairing.
    • Active and functional perspective.
Luck and the Double Helix
  • Luck played a role in the discovery:
    • DNA bases are flat, enabling Watson’s 2D modeling and discovery of complementary pairing.
    • Franklin’s reliance on flawed data about dry and wet DNA forms.
    • Watson and Crick’s serendipitous encounter with a colleague who corrected their understanding of hydrogen bonds, confirming complementary pairing.

Conclusion

  • Four factors hinder insights: flawed beliefs, limited experience, passive stance, concrete reasoning style.
  • Playful/concrete reasoning is a personality trait.
  • Additional barriers can be inadvertently created (e.g., computer systems).

Chapter 11: Dumb by Design

This chapter explores how common design guidelines for computer-based decision support systems can hinder insight. Klein uses the example of Daniel Boone rescuing his daughter to illustrate this point.

Introduction: Design Guidelines for Computer Systems

  • Information technology professionals have developed guidelines to make computers more helpful.
  • These guidelines apply to decision support and information management systems.
  • Four common guidelines are presented:
    1. The system should help people do their jobs better.
    2. The system should clearly display critical cues.
    3. The system should filter out irrelevant data.
    4. The system should help people monitor progress toward their goals.
  • The chapter questions the effect of these guidelines on insights.

The Daniel Boone Example: Rescuing Jemima

  • Daniel Boone, primarily a hunter, needed to rescue his daughter Jemima and her two friends after they were kidnapped by a raiding party.

The Kidnapping

  • Setting: Kentucky, 1776. Boonesboro, a settlement on the Kentucky River.
  • The raiding party: Two Cherokees and three Shawnees.
  • The victims: Jemima Boone (13), Betsy Calloway (16), and Fanny Calloway (14).
  • The event: The girls took a canoe out on the river. The raiding party seized the opportunity and abducted them.
  • The alarm: The girls’ screams alerted the settlers.

The Pursuit and Boone’s Insights

  • Insight Number One: Redirecting the Horsemen
    • Boone formed a rescue party.
    • Richard Calloway, the father of two of the girls, led a group of horsemen.
    • Boone initially searched upstream and downstream.
    • Boone realized searching was inefficient. He anticipated the Shawnees would head north to their villages in Ohio, likely crossing the Licking River at the Upper Blue Licks.
    • Boone redirected the horsemen to ambush the raiding party at the Upper Blue Licks.
    • This insight utilized the “creative desperation path” of the triple path model. It abandoned the assumption that searching was necessary and replaced it with the strategy of ambush.
    • Calloway’s story, actions, attention, feelings, and goal were all transformed by this insight.
  • Insight Number Two: Abandoning the Chase
    • The upstream search party located the kidnappers’ trail.
    • Boone joined the pursuit, aided by the girls leaving trail markers.
    • Boone realized they were falling behind.
    • He anticipated the raiding party’s continued northward movement toward the Upper Blue Licks.
    • He decided to abandon the chase and lead his party directly to the Upper Blue Licks.
    • This insight was another example of creative desperation. It involved discarding the assumption of catching the kidnappers through tracking and adopting the strategy of intersection.
    • This insight shifted Boone’s story, actions, attention, feelings, and goal.
  • Insight Number Three: Cooking the Meal
    • Boone’s party spotted signs of the kidnappers: muddied water, a dead snake, and a freshly butchered buffalo calf.
    • Boone anticipated the kidnappers would cook the buffalo at the next water source.
    • Upon reaching Bald Eagle Creek, Boone divided his party to search upstream and downstream.
    • The upstream group located the raiding party making camp and cooking.
    • This insight used the buffalo carcass as a cue to predict the kidnappers’ next actions. It led to a change in Boone’s understanding, actions, attention, emotions, and goal.

The Rescue

  • Boone’s party surrounded the kidnappers’ camp and launched a surprise attack.
  • The girls were rescued, unharmed except for minor injuries.

Applying Design Guidelines to Boone’s Situation

  • Klein analyzes how the four design guidelines would have affected Boone’s ability to generate insights.

  • Guideline 1: The system should help people do their jobs better.

    • This would have focused on Calloway’s job of catching the Indians and Boone’s job of tracking them.
    • The system would become useless when they shifted tactics.
    • It would have been detrimental, requesting irrelevant information.
    • Boone would have discarded the system.
    • This guideline assumes stable job roles, not applicable in situations requiring insights and changing tactics.

  • Guideline 2: The system should clearly display critical cues.

    • Predefined critical cues would likely miss those becoming relevant after an insight.
    • The cues for searching were irrelevant to the ambush.
    • Boone’s tracking cues became irrelevant after he abandoned the chase.
    • Database structures designed for initial tasks become obsolete after insights change the nature of the work.

  • Guideline 3: The system should filter out irrelevant data.

    • This is considered harmful.
    • The system would have filtered out navigational information needed for the ambush.
    • The buffalo carcass cue might have been downplayed.
    • Filtering data presupposes knowing what’s relevant, hindering the “happy accidents” that spark insights.
    • Examples of how filtering would have hampered historical insights:
      • Battle of Taranto: Air attacks from aircraft carriers wouldn’t have been considered relevant.
      • Discovery of Pulsars: Unusual squiggles would have been filtered out.
      • Battle of Toulon: Small forts like L’Aiguillette and Balaguerre would have been deemed irrelevant.
    • Filtering leads to “filter bubbles,” limiting exposure to diverse perspectives.
    • Example of filter bubbles: Eli Pariser’s experience with Facebook filtering his conservative friends.
    • Search engines like Google personalize results, potentially creating echo chambers.

  • Guideline 4: The system should help people monitor progress toward their goals.

    • Progress monitoring becomes problematic when insights lead to restructured tasks or revised goals.
    • The system would have criticized the riders’ inefficient search, Boone’s abandonment of the trail, and his pause at Bald Eagle Creek.
    • This feature discourages insights that lead to changing goals.
    • Examples of how progress monitoring would have hindered historical insights:
      • Jocelyn Bell Burnell: Wasting time on squiggles instead of quasars.
      • Walter Reed: Chasing mosquitoes instead of improving sanitation.
      • Wagner Dodge: Stopping to light an escape fire.

Stronger Design Equals Weaker Insights

  • The four design principles rely on order and structure, while insights are disorderly.
  • To foster insight, systems should:
    • Focus less on past practices.
    • Give decision-makers room to discover and modify tasks.
    • Facilitate easy shifts in goals and plans.

Conclusion

  • Klein acknowledges the usefulness of the four guidelines but emphasizes their potential to stifle insight.
  • Organizations often claim to value innovation but their practices often hinder it.
  • Klein is pessimistic about designing systems that foster insight because the traditional guidelines are too compelling and organizations might resist more flexible approaches.

Chapter 12: How Organizations Obstruct Insights

This chapter explores how organizations inadvertently suppress employee insights due to ingrained and invisible practices. It examines the motivations and methods behind this suppression.

The Motivations

Organizations stifle insights due to core values embedded in their operational structure:

  • Predictability: Organizations prioritize predictable workflows, resource allocation, and schedules.
  • Aversion to Surprises: Surprises disrupt plans and smooth operations, which organizations strive to avoid.
  • Craving for Perfection: Defined as the absence of errors, perfection is highly valued. Errors are seen as disruptive to plans and operations.
  • Predictability Trap: The overemphasis on predictability hinders insights, which are inherently disruptive and unpredictable.
  • Perfection Trap: The relentless pursuit of perfection, or the absence of errors, overshadows the potential benefits of insights that could lead to improvements beyond the initial objectives.

The Predictability Trap

This section uses a project management scenario to illustrate the predictability trap.

  • Project Setup: A manager, despite valuing innovation and creativity, creates a detailed project plan with timelines, resources, and assigned responsibilities to ensure on-time and within-budget completion. This plan is necessary for approval and tracking.
  • Resistance to Change: The manager receives suggestions from team members:
    • Suggestion 1: Reassigning team members based on skill needs. The manager dismisses this due to potential disruptions and management overhead.
    • Suggestion 2: Modifying the project plan to combine tasks and improve customer value. This is rejected by the director due to contractual obligations and potential payment delays.
  • Consequences: The manager, initially supportive of insights, becomes wary of them due to the negative reactions from superiors. Predictability becomes prioritized over potential gains from disruptive insights.
  • Insight as Disruption: Insights, being unpredictable and disruptive, clash with the desire for smooth progress reviews and adherence to established goals and tasks. They introduce unforeseen risks and complications.
  • Managerial Burden: Managers, already burdened with project oversight, lack the capacity to fully explore the implications of team members’ insights.
  • Aversion to Creativity: A 2012 study by Mueller, Melwani, and Goncalo (using an implicit association test) found that people associate novel ideas with impracticality, unreliability, and errors. Novelty is linked to uncertainty, and the desire to reduce uncertainty leads to lower evaluations of creative ideas. This aversion to uncertainty explains managerial skepticism towards novel ideas.
  • Historical Examples: The initial resistance to innovations like the telephone, Google search, VisiCalc, the Xerox 914 copier, and Xerox’s rejection of personal computers demonstrate a historical pattern of corporate suspicion towards disruptive technologies.
  • Managerial Focus: Managers prioritize defining tasks, setting schedules, and establishing completion criteria, thereby minimizing reliance on unscheduled and unpredictable insights.

The Perfection Trap

This section defines and explores the concept of the perfection trap.

  • Perfection as Absence of Errors: Organizations prioritize reducing errors due to their measurability and manageability. This pursuit of perfection aligns with the desire for predictability.
  • Context-Dependent Value: While perfection is valuable in well-ordered situations with clear goals and stable conditions, it becomes detrimental in complex and chaotic environments with evolving standards.
  • Example: Olympics Gymnastics Scoring: The focus on error detection in gymnastics judging, exemplified by the 2012 London Olympics, illustrates how performance evaluation can prioritize minimizing mistakes over artistry and other less quantifiable qualities.
  • Reasons for Disliking Errors: Organizations dislike errors due to safety risks, coordination disruptions, waste, reduced project success rates, cultural erosion, lawsuits, and negative publicity.
  • Managerial Focus on Error Correction: Managers dedicate significant time to identifying and rectifying errors, prioritizing error reduction over fostering insights. Spotting and correcting errors is perceived as easier than encouraging insights.
  • Perfection vs. Improvement: While achieving perfection in executing the initial plan might lead to promotions, insights can offer improvements beyond the original design. However, the perceived risks of deviating from the plan often outweigh the potential benefits of exceeding initial objectives.
  • Beyond Perfection: In complex situations with “wicked problems” lacking clear solutions, the concept of perfection becomes irrelevant. The goal shifts towards discovering better solutions through insights, rather than adhering to an initial, potentially flawed vision.
  • Performance Equation: Performance improvement is achieved by reducing errors and uncertainty (down arrow) and increasing insights (up arrow). However, these two objectives often conflict, leading organizations to overemphasize error reduction and fall into the predictability and perfection traps.

The Methods

This section describes how organizations obstruct insights through management controls and organizational structure.

Enforcing the Down Arrow

  • CIA Post-Iraq War: After the flawed intelligence leading to the 2003 Iraq invasion, the CIA intensified its focus on error reduction through stricter documentation, source verification, assumption identification, and uncertainty estimation. “Tradecraft” became synonymous with these precautions. An experienced analyst revealed that all his successes involved violating tradecraft. There is no parallel effort to encourage insights.
  • BBC Post-Iraq War: Following criticism for broadcasting inaccurate information about Iraq, the BBC implemented stringent procedures, including extensive editorial guidelines and pre-broadcast reviews. This resulted in over-reliance on procedures and a lack of experienced judgment when needed. Two subsequent incidents highlight the flaws:
    • Error of Omission: Failure to report child molestation allegations against Jimmy Savile due to procedural obstacles.
    • Error of Commission: Falsely accusing a Thatcher government member of pedophilia due to inadequate fact-checking. Experienced staff were unavailable, and remaining staff prioritized procedural adherence over critical thinking.
  • Legal Profession: Lawyers focus on risk avoidance and identifying potential problems, prioritizing error avoidance over achieving client goals.
Methods for Reducing Errors and Uncertainty:
  • Tighter standards
  • Increased controls
  • Documenting all sources
  • Identifying assumptions
  • Estimating uncertainty values for assumptions
  • Increased reviews
  • More rigorous justification of conclusions
  • Reliance on checklists and procedures
  • Increased schedule precision
How Down-Arrow Methods Interfere with Insights:

  • Distraction: Error-avoidance tasks consume time and mental energy, leaving less for thoughtful speculation and exploration of new ideas.

  • Reluctance to Speculate: The pressure to avoid errors discourages unvalidated intuitions and insights.

  • Negative View of Insights: Disruptive insights are perceived as threats to established schedules and decisions. Organizations prioritize closure and discourage revisiting settled matters.

  • Repression of Anomalies: Anomalies, being disruptive, are often ignored or dismissed.

  • Passivity: Checklists and procedures encourage mindless adherence to steps, discouraging active thinking and inquiry.

  • Benefits and Drawbacks of Checklists: While checklists are crucial for safety and efficiency in fields like aviation, nuclear power, and healthcare, their reliance on mindlessness can stifle insight generation. Checklists are effective in well-ordered situations but can be detrimental in complex, ambiguous environments.

  • Balancing Arrows: Management controls are necessary, but excessive controls hinder insights due to the costs associated with change. Performance relies on balancing error reduction and insight generation.

Organizational Repression

  • Hierarchical Filtering: Organizational hierarchies filter insights as each level must approve for ideas to reach decision-makers. This can lead to suppression of insights from junior levels.
  • CIA Coup Example: A junior analyst’s accurate prediction of a coup was suppressed by a supervisor who didn’t want to be wrong, leading to the White House being caught off guard.
  • FBI Phoenix Memo: FBI Special Agent Kenneth Williams’s warning about suspicious flight lessons was ignored due to its unusual nature, highlighting the organizational tendency to dismiss anomalies.
  • CIA Berlin Wall Example: A CIA analyst’s insight about the weakening impediments to German reunification was diluted through peer review, resulting in the agency being caught unaware of the impending fall of the Berlin Wall.
  • Japanese Midway War Game: The Japanese Navy’s dismissal of a war game scenario mirroring the actual American ambush at Midway illustrates organizational repression of dissenting views and overconfidence.
  • Self-Censorship: Insight suppression occurs at all levels, including self-censorship by junior analysts who fear negative consequences for expressing unpopular views.
  • Snowden Research Project: A study using garden path scenarios revealed that individual team members often noticed weak signals indicating a different story than the apparent one, but these insights were rarely voiced or were ignored by team leaders.
  • Intelligence Agency Presentation: Junior analysts confirmed insight suppression and self-censorship due to fear of jeopardizing their careers. Senior analysts expressed surprise, highlighting a disconnect between official policy and organizational reality.
  • Black Swans: While organizations seek to predict “black swan” events, their inherent unpredictability makes this impossible. Organizations could benefit from heeding early warnings from internal alarmists, but their tendency to discount anomalies hinders this.
  • Errors of Omission: The fear of making errors of commission (false claims) can lead to errors of omission (missing crucial warnings).
  • Risk Aversion: Organizations stifle disruptive insights due to risk aversion, prioritizing the avoidance of errors over the potential gains from unexpected discoveries.
  • Inherent Organizational Nature: The need for predictability and perfection is inherent in organizational structure, making it difficult to address even when recognized as a problem.
  • Password Metaphor: A senior intelligence official compared excessive reviews to overly long passwords, where the marginal gain in security is outweighed by the increased workload.
  • Scientific Community: Even scientists, who should prioritize insights, often fall into the trap of overemphasizing error avoidance. The pursuit of absolute truth can lead to vacuous statements, hindering scientific progress. A more effective strategy would be to make the most extreme defensible statements, pushing the boundaries of knowledge and accepting the risk of some errors.
  • Thomas Kuhn: Kuhn’s concept of “normal science” describes how scientists often suppress novelties that challenge established paradigms, highlighting the prevalence of error aversion even in scientific research.
  • Examples of Scientific Suppression: The resistance faced by Carlos Finlay (mosquitoes and yellow fever), John Snow (contaminated water and cholera), and Barry Marshall (H. pylori and ulcers) demonstrates the scientific community’s susceptibility to the down arrow.
  • Puzzle-Solving Focus: Kuhn’s description of normal science as puzzle-solving parallels the focus of insight researchers on puzzle problems, which, as explored in the next chapter, has contributed to limiting the understanding of insights.

Chapter 13: How Not to Hunt for Insights

Introduction: A Thought Experiment on Ineffective Insight Research

  • Klein reflects on their research process for collecting 120 insight stories, which led to the development of the triple-path model.
  • A thought experiment is proposed: if the goal was to fail at understanding insights, what flawed methods would be employed?

Counterproductive Strategies for Insight Research

  • Timing and Structure:
    • Scheduled Sessions: Imposing specific start and end times for capturing insights, despite their spontaneous nature.
    • Assigned Tasks: Assigning unrelated insight tasks instead of letting subjects explore their own interests.
    • Meaningless Tasks: Using tasks that lack meaning or relevance to the subjects, hindering genuine engagement.
    • Short Timeframes: Restricting available time to an hour, despite the fact that many insights develop over longer periods.
  • Pressure and Evaluation:
    • Performance Appraisal: Making subjects aware of being timed and evaluated, creating pressure that inhibits insight (supported by Teresa Amabile’s research on creativity at Harvard Business School).
  • Verbalization Interference:
    • Think-Aloud Reports: Requiring constant verbalization of thought processes, which can disrupt insight formation (as noted by Graham Wallace and other researchers).
  • Limiting Exploration:
    • Impasse Focus: Exclusively using tasks that create impasses, neglecting other paths to insight such as connections, coincidences, curiosities, and contradictions.
    • Ignoring Experience: Disregarding the role of experience, despite its prevalence in two-thirds of Klein’s research cases. Specifically, overlooking the use of experience for making connections and seeing contradictions.
    • Novel Tasks: Studying how people handle entirely unfamiliar tasks, preventing the application of prior knowledge.
  • Diabolical Methods:
    • Intentionally designing experiments that actively hinder the use of experience, creating artificial roadblocks to insight.

The Prevalence of Flawed Insight Experiments

  • Klein criticizes the common practice in insight research of using puzzles with college undergraduates under highly controlled conditions.
  • This approach is described as “diabolical” and is argued to limit the understanding of insights.

The Legacy of Impasse Problems: From Chimpanzees to College Students

  • Wolfgang Kohler’s Chimpanzee Study (1917):
    • Kohler observed chimpanzee “Sultan” using insight to solve a problem involving retrieving a banana beyond reach by combining two sticks.
    • This study challenged the notion of trial-and-error learning and sparked interest in insight research.
  • The Shift to Human Subjects:
    • Researchers began using college undergraduates due to their availability and ease of management.
  • The Nine-Dot Puzzle:
    • This classic puzzle requires connecting nine dots with four straight lines without lifting the pencil.
    • The solution involves extending the lines beyond the perceived boundaries of the dots.
  • Assumptions and Impasses:
    • The nine-dot puzzle exemplifies how unnecessary assumptions create impasses.
    • Assumptions like staying within the boundaries of the dots or pivoting lines only on dots hinder the solution.
    • Trina Kershaw (University of Massachusetts, Dartmouth) and Stellan Ohlsson (University of Illinois at Chicago) identified these assumptions.

The Creative Desperation Path

  • Impasse tasks, like the nine-dot puzzle, exemplify the creative desperation path of the triple-path model.
  • This path involves consciously challenging beliefs, assumptions, and anchors to overcome a blockage.
  • Fixation on traditional object uses or inappropriate goals can create impasses.

Laboratory Experiments and the Creative Desperation Path

  • Laboratory experiments on insight often rely on this path.
  • Subjects are presented with problems designed to create impasses through unnecessary assumptions.
  • Success hinges on identifying and discarding these assumptions.

The Checker Game Example

  • “Two men play five checker games, and each wins an even number of games with no ties. How is that possible?”
  • The impasse arises from the assumption that the men played against each other. The solution lies in realizing they could have played against different opponents.

More Impasse Examples

  • Box-Candle Puzzle: Attaching three candles to a door using boxes, tacks, and matches. The solution is to use the boxes as candle stands.
  • Pendulum Puzzle: Grabbing two strings beyond arm’s reach. The solution involves creating a pendulum with one string using a heavy object.

Real-World Impasses

  • Aron Ralston’s escape from being trapped under a boulder is cited as a real-world example of overcoming a fixation.

Neuropsychological Research on Impasses

  • Neuropsychologists use brain scans during impasse problem-solving to identify brain regions activated before the “aha” moment (e.g., 2010 article by Arna Dietrich and Riam Konso, American University of Beirut).

Critique of Domesticated Insight Tasks

  • Klein critiques the overuse of “domesticated” insight tasks like the nine-dot, box-candle, and pendulum puzzles.
  • These tasks are considered predictable and lack the exploratory aspect of real-world insight.
  • While acknowledging the ingenuity of researchers studying impasses, Klein questions the applicability of their findings outside the laboratory.

The Need for Broader Research

  • A call for researchers to explore impasses in natural settings and investigate the other paths of the triple-path model (connection and contradiction insights).

The Misrepresentation of Experience

  • Klein criticizes the use of the impasse paradigm to devalue experience.
  • The majority of Klein’s research highlighted the crucial role of experience in gaining insights.
  • Impasse puzzles exploit experience by eliciting routine assumptions that are incorrect in the specific context.

The Water Jar Puzzle and the Einstellung Effect

  • The water jar puzzle example illustrates how inducing automaticity through repetitive tasks can lead to an Einstellung effect.
  • Subjects trained on a specific solution method (B - A - 2C) overlook a more efficient solution (A + C) when it becomes available.
  • This is presented as a misleading demonstration of how experience hinders insight.

Challenging the Critique of Experience

  • Klein challenges the notion that experience hinders insight, arguing that the water jar study creates a false representation of experience by inducing mindless routine.
  • Real-world examples, such as Harry Markopolos uncovering Bernie Madoff’s fraud and Charles Darwin’s development of natural selection theory, demonstrate the importance of experience.
  • Admirals Stark and Yamamoto’s understanding of the Battle of Taranto’s significance is also cited.

Automaticity vs. Real-World Experience

  • Klein distinguishes between automaticity, induced in laboratory settings, and real-world experience.
  • Automaticity requires repetitive, identical conditions, unlike the varied nature of real-world experiences.

Functional Fixedness

  • The concept of functional fixedness, the tendency to rely on conventional object uses, is discussed.
  • While puzzles exploit this tendency, Klein argues it has limited relevance to real-world insights.

Expertise and Overconfidence

  • Klein acknowledges that experts can be trapped by their assumptions, as seen with Sherman Kent and Eliezer’s misjudgments.
  • However, this is attributed to overconfidence in flawed beliefs, not automaticity or functional fixedness.

Stellan Ohlsson’s Concerns and the Limitations of the Impasse Paradigm

  • Stellan Ohlsson, a prominent impasse paradigm researcher, has expressed concerns about the limitations of this approach.
  • He has suggested that there may be “nothing left to explain” within the impasse paradigm and acknowledged its inability to explain insights like Darwin’s.

Klein’s Departure from Conventional Research

  • Klein contrasts their naturalistic approach with the controlled experiments preferred by conventional researchers.
  • They argue that the focus on the “aha” experience for defining insights limits the scope of research.

The Purpose of Science

  • Klein emphasizes that the purpose of science is to learn about the world, not just to adhere to rigid methods.
  • They advocate for using methods that effectively capture the phenomenon under investigation, even if it means departing from conventional approaches.

Conclusion: Retiring the Impasse Paradigm

  • Klein concludes that the impasse paradigm may have become “played out” and proposes moving beyond it.
  • While valuable insights have been gained, it’s time to explore new approaches that capture the full complexity of insights.

Part III: Opening the Gates

Chapter 14: Helping Ourselves

How Can We Gain More Insights?

  • Examining insight paths individually, as suggested by the triple-path model of insight, makes the challenge of gaining more insights more manageable.

The Triple-Path Model of Insight

The Tilt Reflex (Contradiction Path)
  • Leveraging Contradictions: We can utilize contradictions, confusions, and conflicts as catalysts for insights. These disruptions, often sources of frustration, represent opportunities for discovery. Replacing consternation with curiosity is key.
  • The Tilt Reflex: This reflex helps us recognize and process contradictions.
  • Example 1: Ginger and the Non-Compete Agreement:
    • Ginger, bound by a non-compete agreement, felt frustrated by the contradiction between the legal obligation and her inability to fulfill it due to lacking her previous company’s client list.
    • Her tilt reflex helped her understand this contradiction, leading her to confront her former employer and gain release from the agreement.
  • Example 2: Dennis and the ABC Page Job:
    • Dennis experienced a disconnect between his childhood routines of assigning blame and the realities of starting a Hollywood writing career.
    • After complaining to his potential manager at ABC about their delayed hiring process, the manager’s blunt response triggered Dennis’s tilt reflex.
    • Dennis realized the irrelevance of blame games outside his family and the importance of deliverability and trustworthiness in a professional setting.
      • This epiphany marked a transition to adulthood and informed his understanding of effective management.
      • His insight was about abandoning unproductive behaviors and recognizing what truly matters to managers.
      • This realization also inspired his later work on “The Larry Sanders Show,” a comedic exploration of the blame syndrome.
  • Example 3: The Fire Captain and the “Attitude Problem”:
    • A fire captain, initially believing a young firefighter had an attitude problem, used an OJT (on-the-job training) technique learned in a workshop.
    • The technique involved asking the firefighter to explain his rationale instead of immediately reprimanding him.
    • This led the captain to realize the firefighter’s actions were well-intentioned, leading to the self-insight: “I was the attitude problem.”
    • The captain’s tilt reflex was triggered by the unexpected explanation, revealing his own role in creating conflict through rigid adherence to procedures. He gained insight into his firefighter and himself by actively seeking another perspective and being open to contradictory views.
Swirl (Connection Path)

  • Accidental Linkages: The connection path relies on the collision of diverse ideas. Increased “swirl” and turbulence create more opportunities for serendipitous discoveries.

  • Examples of Accidental Linkages Leading to Insight:

    • Martin Chalfie’s lunchtime seminar on jellyfish.
    • Michael Gottlieb’s referral of a patient with unusual symptoms.
    • Jocelyn Bell Burnell’s observation of squiggles in radio telescope data.
    • Barry Marshall’s study of a corkscrew-shaped bacterium.
    • The fire captain’s experience in the OJT seminar.

  • Increasing Accidental Linkages: Expanding exposure to unfamiliar activities and diverse individuals can seed new combinations of concepts.

  • Stephen Johnson’s Recommendations (from Where Good Ideas Come From, 2010):

    • Increase exposure to diverse ideas.
    • Interact with creative individuals.
    • Foster serendipity and random collisions of ideas.
    • Engage with different communities (both physical and virtual).
    • Encourage collaborative work.
    • Cultivate multiple hobbies.

  • Organizational Strategies for Increasing Swirl:

    • Random office assignments to prevent specialization silos.
    • Designing spaces that encourage interaction (e.g., Pixar’s atrium).

  • Critique of Swirl as an Insight Strategy:

    • While acknowledging the value of swirl in accidental discoveries, Klein expresses skepticism about deliberately increasing it as a reliable method for generating insights.

    • Fallacy of Backward Thinking: Replicating past conditions that led to insights doesn’t guarantee future success and can generate numerous unproductive ideas.

    • Examples of Spurious Correlation: Increasing lunchtime seminars won’t necessarily replicate Chalfie’s experience; making more desserts won’t replicate Gopnik’s discovery.

    • Henri Poincaré’s Perspective: Creativity is about selecting useful combinations from an infinite number of possibilities, not simply generating more combinations.

      Creation does not consist in making new combinations. The combinations so made would be infinite in number and most of them absolutely without interest. To create consists precisely in not making useless combinations.

Critical Thinking (Creative Desperation Path)
  • Escaping Mental Traps: This path involves recognizing and correcting flawed assumptions when we’re stuck.
  • Critique of Assumption Listing:
    • Klein dismisses the practice of listing all assumptions as ineffective and likely stemming from the fallacy of backward thinking.
    • Listing all assumptions is distracting and requires expertise to evaluate their validity.
  • Napoleon’s Insight at the Battle of Toulon: Napoleon’s background and prepared mind allowed him to re-examine the tactical situation, not a systematic listing of assumptions.
  • Identifying Flawed Assumptions: Look for conflicts, confusion, surprises (unexpected events), and contradictions within a team or project.
  • Critical Thinking:
    • Defined as “a systematic analysis of evidence, a process of thinking that questions assumptions…by logically reviewing the evidence in order to decide whether or not to believe specific claims.”
    • Critical thinking is crucial for sound judgment and decision-making.
  • Caution Against Overreliance on Critical Thinking: Excessive critical thinking can stifle creativity and exploration of new ideas. It can dampen the “up arrow” and interfere with non-critical, playful thinking.
  • Role of Critical Thinking in Insight: Most useful in times of desperation when flawed assumptions hinder progress.

The Pause That Refreshes (Incubation)

  • Incubation in Wallace’s Four-Stage Model: Wallace emphasized incubation as a stage in the creative process.
  • Incubation in the Research Sample:
    • Only five cases demonstrated incubation.
    • Forty-seven incidents occurred too rapidly for incubation.
    • Sixty-eight cases were difficult to classify.
    • Examples of insights without incubation: Wagner Dodge, Chalfie, Markopoulos, Gopnik, and Aron Ralston.
  • David Perkins’s Perspective (from Archimedes’ Bathtub):
    • Research has not definitively proven the incubation effect.
    • Controlled experiments may not accurately reflect real-world conditions.
  • Author’s Perspective on Incubation: Despite skepticism based on critical thinking, Klein personally values and practices incubation.
  • Rebecca Dodds et al.’s Research:
    • Review of 39 experiments since 1938 shows that two-thirds found an incubation effect.
    • Incubation is more effective with longer periods and thorough preparation.
  • Possible Explanations for Incubation’s Effect (Eli and Sun, 2010):
    • Unconscious Processing: The unconscious mind continues working on the problem.
    • Mental Recovery: Incubation allows recovery from mental fatigue.
    • Remote Associations: Relaxation facilitates access to remote associations otherwise blocked by critical analysis.
    • Heightened Sensitivity: Priming from prior work increases sensitivity to relevant information in unexpected places (e.g., the Gopnik case).
  • Incubation and Insight Paths:
    • Most useful on the creative desperation path.
    • Potentially relevant to the contradiction path.
    • Different incubation mechanisms might apply to different insight paths (e.g., mental recovery for desperation, remote associations for connection).

Summary of Insight Paths

  • Contradiction Path: Focuses on inconsistencies and anomalies.
  • Connection Path: Emphasizes exposure to new ideas.
  • Creative Desperation Path: Utilizes critical thinking to address flawed assumptions.

Clarifying Insight Methods

  • Exposure to diverse ideas is primarily relevant for the connection path.
  • Breaking through fixation and impasses applies mainly to the desperation path.

Chapter 15: Helping Others

Introduction

  • Shifting from self-help to helping others presents a different challenge.
  • It often involves correcting others’ flawed beliefs.
  • This requires understanding the nature of these flawed beliefs.
  • The most fundamental, and often toughest, part is diagnosing the confusion and identifying the errors in their thinking.

Diagnosis

  • Klein’s daughter, Devorah, a cognitive psychologist, works on designing new products and understanding user interaction.
  • Example: Devorah’s project involving an adapted Kindle-type reader for people with impaired vision.
    • Case study of a 79-year-old retired English professor frustrated by his poor vision.
    • Devorah demonstrated the device, which the professor followed despite being computer illiterate.
    • He remained dubious, and Devorah couldn’t discern the source of his hesitation.
    • Through patience and persistence, she sparked his intrigue about the device’s potential for reading.
    • He expressed concern about needing to clear a shelf for the “devices,” revealing his misconception.
    • Devorah diagnosed his flawed belief: he thought the e-readers were like books on tape, each holding a single book.
    • She informed him that the device could hold dozens, perhaps hundreds of books.
    • His reaction was astonishment and excitement, demonstrating a shift in his understanding.
  • Key takeaway from Devorah’s approach:
    • She waited for the professor to articulate his thoughts, recognizing his need to process.
    • She didn’t offer unsolicited advice but provoked his curiosity, leading him to ask the crucial question.
  • General principle for helping others:
    • Avoid unsolicited advice and attempts to convince others of what they “need” to know.
    • Instead, connect with their desire for insights and facilitate their own discoveries.

The Role of Insight in Psychotherapy

  • Psychotherapists work with clients seeking insights into their poor decisions and destructive patterns.
  • Psychotherapy focuses on insights gained by both the therapist and the client.
  • Example: Klein’s brother, Mitchell, a psychotherapist, shared a case study about a woman named Barbara.
    • Barbara ran a business with her cousin and experienced anxiety and depression due to the cousin’s irresponsible behavior.
    • Mitchell diagnosed the cousin with extreme narcissistic personality disorder.
    • He helped Barbara understand her cousin’s narcissism and correct her belief that the cousin’s actions were deliberate attempts to hurt her.
    • Barbara gained insight, felt liberated from anger and confusion, and ultimately took over the business, leading to its success.
  • Both Devorah’s and Mitchell’s cases highlight the importance of diagnosis as the first step towards insight.
  • Providing information becomes straightforward once the flaw in thinking is identified.

Diagnosis Plus Action

  • Insight alone doesn’t guarantee behavioral change or wiser choices.
  • Therapists value insights that translate into action.
  • Example: Bob Barkas, a psychotherapist, shared a case study of a mother and her depressed teenage daughter.
    • The mother was consistently critical of her daughter.
    • Bob asked the mother to tally positive and negative comments made to her daughter for a week.
    • The mother realized her overwhelmingly negative communication style, leading to an emotional breakthrough.
    • Bob then guided her to reverse the negativity by praising positive behaviors or the absence of negative ones.
    • The relationship improved significantly, and the daughter’s depression subsided.
  • This case demonstrates that helping others may require guiding them towards new behaviors in addition to correcting beliefs.

Facilitating Discovery

  • Example: Klein’s experience helping his friend Jimmy improve his racquetball skills.
    • Jimmy struggled with his backhand return of serve.
    • Instead of giving direct instructions, Klein had Jimmy observe the ball’s trajectory without swinging.
    • He then had Jimmy point to ideal hitting spots before finally allowing him to swing.
    • This process facilitated Jimmy’s own discovery of how to hit a backhand effectively.
  • Klein emphasizes the value of helping others make their own discoveries rather than simply telling them what to do.

Using Contradictions to Repair Flawed Beliefs

  • Creating constructive dilemmas through contradictions can be a powerful tool.
  • Example: Doug Harrington, a Navy pilot transitioning to a new aircraft (A-6), experienced sudden difficulty landing.
    • The Landing Signal Officer (LSO) repeatedly told him to “come right,” which contradicted Harrington’s perception of being aligned.
    • The senior LSO diagnosed the issue: Harrington’s seating position in the A-6 was offset from the center line, affecting his alignment.
    • The LSO devised a thumb-and-doorframe exercise to demonstrate the effect of the offset.
    • This exercise created a contradiction that forced Harrington to abandon his flawed belief about landing alignment.
    • He immediately grasped the problem and performed flawlessly the next day.
  • The LSO’s approach exemplifies effective teaching/coaching: diagnosing the problem and facilitating self-discovery through a contradicting experience.

Example: Deborah Ball’s Teaching Method

  • Deborah Ball, a master teacher, taught her third-grade class about odd and even numbers.
  • A student, Sean, claimed that six could be both odd and even because 3 x 2 = 6.
  • Instead of correcting him directly, Ball facilitated a class discussion.
  • The class formulated a rule: an odd number has one left over when grouped by twos.
  • This contradicted Sean’s claim, leading him to self-correct his understanding.
  • The class also identified “Sean numbers”—even numbers composed of an odd number of groups of two.
  • Ball’s approach, similar to the LSO’s, highlights the power of contradiction in repairing flawed beliefs and fostering self-discovery.

Fumbling: A Missed Opportunity

  • Klein recounts a training session with Marines in 1997.
  • A sergeant, presented with an ambush scenario, ordered his fire team to charge directly at the enemy.
  • Other NCOs suggested flanking the enemy.
  • Reflecting on this incident, Klein realized he missed an opportunity to help the sergeant.
  • The sergeant misapplied the ambush doctrine, which dictates attacking the source of fire directly.
  • The sergeant’s unit wasn’t being ambushed directly, but rather witnessing other units under attack.
  • Klein should have explored the sergeant’s reasoning and helped him differentiate between being ambushed and observing an ambush.
  • This missed opportunity highlights Klein’s own learning process in helping others gain insights.

Chapter 16: Helping Our Organizations

The Tyranny of the Down Arrow

  • Performance Equation: Organizations need to break the “tyranny of the down arrow” in the performance equation. This means finding a balance between reducing errors (down arrow) and increasing insights (up arrow).
  • War on Error: Organizations overemphasize error reduction at the expense of gaining insights.
  • Pressure for Predictability: Managers prioritize specifying tasks and timetables, viewing insights as disruptive.
  • Visibility of Errors: Errors are public, easily tracked, and measurable, providing readily available performance metrics for managers.
  • The Brake Pedal Analogy: The down arrow is like a brake pedal; organizations are pressing too hard, hindering progress.
  • Forces of Predictability: These forces are constant and make relaxing the emphasis on error reduction difficult to sustain.
  • Difficulty Visualizing Insights: Individuals can easily visualize errors but struggle to imagine potential insights.
  • Past Failures: Advocating for insights has historically been unsuccessful, discouraging future attempts.
  • Controls and Organizing: Controls are central to organizational structure and are driven by the need for predictability and perfection.

Strengthening the Up Arrow

  • Promoting Insights: Countervailing forces are needed to create pressure for insights and discoveries.

  • Insight Advocates: Establish a team of insight advocates to promote practices that encourage discoveries.

    • Model this on existing teams like quality control or analytical integrity offices.
    • Responsibilities:
      • Collect examples of insights and discoveries.
      • Disseminate these examples monthly to encourage others and demonstrate organizational value for insights.

  • Undocumented Insights: Many valuable insights go undocumented.

  • David Klinger Example: Author’s experience on a US Navy project illustrates how an accidental conversation led to a valuable insight about staffing levels.

    • The specific question about reducing staffing, not a general inquiry about insights, catalyzed the discussion.

  • Kurt Swogger Example: Swogger acted as a one-man insight advocate at Dow Chemical by actively listening to employees and connecting disparate pieces of information.

    • “Your people actually have the answers. They just don’t know it. And there’s a piece here and a piece there that you have got to put together.”

    • Swogger’s insight stemmed from recognizing the potential of a new catalyst developed by chemists.

    • The new product, “Six Days,” significantly reduced delivery times for customized polymer solutions.

  • Storytelling: Insight advocates can use stories to share what they learn.

  • Singapore Workshop Example: Workshop with Patrick Lambe (Straits Knowledge) and Sean Callahan (Anecdote) highlighted the connection between stories and insights.

  • Anecdote’s Tagline: “Putting Stories to Work.”

  • Leadership Story Example: Sean Callahan’s story about a simple leadership observation that led to widespread organizational change demonstrated the power of storytelling.

    • The story focused on a manager’s practice of giving employees undivided attention.
    • Four reasons for the story’s impact:
      • Self-discovery by participants.
      • Concrete instructions within the narrative.
      • Social proof through peer agreement.
      • Emotional resonance.

  • Emotion and Action: The emotional content of stories can translate insights into lasting change.

  • Author’s Pessimism: Despite these examples, Klein remains pessimistic about an organization’s ability to sustain such initiatives.

  • Sean Callahan’s Optimism: Callahan believes storytelling programs are sustainable, citing an example of a skeptical leader who became a strong advocate after gaining a personal insight.

  • Chief Innovation Officers: Haydn Shaughnessy’s 2012 Forbes article discusses the role of chief innovation officers in supporting insight-driven initiatives.

  • Downward Arrow Persistence: Klein’s pessimism stems from observing how “flavor of the month” initiatives often give way to the persistent forces of predictability and perfection.

Loosen the Filters

  • Alternate Reporting: Create alternate reporting routes to bypass routine editing and filtering.
  • Practical Limitations: Workers often prefer corporate endorsement, meaning going through the established review process.
  • “Jumping Levels” Strategy: A three-star general’s strategy involves direct dialogue with lower-ranked individuals he trusts to bypass filtering.
  • “Court of Appeals”: A former intelligence analyst suggests an oversight group to review rejected, bold position pieces.
  • Balancing the Arrows: Loosening filters shouldn’t eliminate them entirely. The goal is balance, not dominance of either arrow.
  • FBI Example: Overloading with unfiltered information can lead to dysfunction, as illustrated by the potential for an overwhelming number of warnings in the FBI.
  • Oversight Group as Middle Ground: An oversight group can provide an escape hatch for blocked insights without dismantling the filtering process.

Increase Organizational Willpower

  • Acting on Insights: The core problem often lies not in lacking insights, but in failing to act on them.

  • Lack of Willpower: Organizations may lack the willpower to change, even when they recognize the need.

  • Admiral Stark Example: Admiral Stark’s insight about the vulnerability of Pearl Harbor was not acted upon due to organizational resistance.

  • Rosabeth Moss Kanter’s Observation:

    • “I still observe executives exhibiting the same lack of courage or knowledge that undercut previous waves of innovation. They declare that they want more innovation, but then ask, ‘Who else is doing it?’ They claim to seek new ideas, but shoot down every one brought to them.”

  • Eastman Kodak Example: Kodak’s bankruptcy, despite inventing the digital camera, illustrates a failure of willpower, not insight. They prioritized short-term high profit margins from film over long-term adaptation.

  • Encyclopedia Britannica (EB) Example: EB’s decline, despite creating early CD-ROM encyclopedias, also demonstrates a lack of willpower. Their established sales force resisted change to protect their existing business model.

  • Kodak and EB’s Lack of Willpower: Both companies were trapped by successful business models and failed to adapt quickly enough.

  • Continual Transformation vs. Insights: Klein questions the concept of “continual transformation,” arguing it may hinder the emergence of accidental insights.

  • Goal Insights: Organizational willpower is demonstrated by acting on goal insights, which involve changing the goals themselves, not just adapting plans.

  • Sean MacFarland and Operation Iraqi Freedom Example: Colonel MacFarland’s reversal of goals in Anbar province, Iraq, by aligning with Sunni tribes against al-Qaeda, showcases the power of goal insights. This reversed a losing situation and transformed Anbar into a secure zone.

  • Goal Fixation: Organizations often reward tenacity in pursuing established goals, which can lead to goal fixation and hinder adaptation.

  • Kishore Sengupta Study: Sengupta’s 2008 study demonstrated managers’ tendency toward goal fixation in a computer simulation, even when original goals became obsolete.

  • Diagnosing Organizational Problems: Organizations need self-insight to determine whether their problems stem from preventing, filtering, or failing to act on insights.

Appeal to Authority

  • Changing the Culture: Leaders can be persuaded to change the organizational culture by demonstrating the negative consequences of overemphasizing the down arrow.
  • The Six Sigma Experiment: The widespread adoption and subsequent failure of Six Sigma in many Fortune 500 companies provides evidence of the downsides of overemphasizing error reduction. Six Sigma hindered innovation by focusing excessively on defect reduction.
  • Six Sigma’s Impact: Despite initial success, Six Sigma ultimately hampered innovation and led to underperformance compared to the S&P 500.
  • 3M Example: 3M experienced decreased creativity under Six Sigma, with researchers facing excessive documentation requirements.
  • Balancing the Arrows with Six Sigma: Six Sigma shouldn’t be abandoned but integrated into a balanced approach.
  • Ambidextrous Organizations: Charles O’Reilly III and Michael Tushman’s concept of ambidextrous organizations advocates for separate groups focused on efficiency (down arrow) and innovation (up arrow) reporting to the same manager. This helps prevent the error-reduction culture from stifling creativity.
  • Pragmatism and Survival: Appealing to the desire for organizational success and survival in a competitive environment can be a powerful motivator for change.

Chapter 17: Tips for Becoming an Insight Hunter

Opening the Gates to Insight

  • Tracking and unpacking insights are crucial for understanding them.
  • Increased sensitivity to insights leads to observing more of them (tracking).
  • Unpacking involves dissecting the process of how the insight was gained.

Challenges of Secondary Sources

  • Unpacking insights from articles is difficult due to the inability to ask questions.
  • Klein’s research project faced this challenge due to reliance on secondary sources.
    • Example: Wagner Dodge (died 1955) and the Mann Gulch Fire (Norman Maclean’s account).
    • Example: Investors who profited from the 2007 subprime housing market collapse (Michael Lewis and Gregory Zuckerman’s accounts).
  • Secondary sources were sufficient for Klein’s initial research goals (making discoveries).
  • Direct interaction with the individual who had the insight provides deeper understanding.
  • Ideal scenario: Observing the insight unfold and probing for details.
    • Example: Klein’s daughter, Devorah, and her fantasy baseball team discovery.

The Power of Interviews

  • Interviews bridge the gap between secondary sources and direct observation.
  • Many of Klein’s research cases (almost a third) came from in-depth interviews.
    • Example: U.S. Army General Martin Dempsey and urban warfare tactics in Iraq.
    • Example: Dave Charlton and heat conduction in thin glass coatings.
    • Example: Ginger and non-compete clause struggles.
  • Be prepared to hunt for insights, as opportunities can arise unexpectedly.

Case Study: Lizard Zero

  • Setting: Alaska vacation in June 2010.
  • Encounter: Jay Cole (retired curator, Department of Herpetology and Ichthyology, American Museum of Natural History) and his wife, Carol Townsend.
  • Initial conversation: Jay mentioned a lizard species that seemed to have regressed to cloning.
  • Delayed follow-up: Vacation distractions postponed further discussion.
  • Dinner interview: Klein arranged a dinner with Jay and Carol to learn more.
  • The Lizard Story:
    • The Anomaly: In the late 1950s, naturalists found all-female whiptail lizard colonies in the southwestern U.S. and northern Mexico.
    • Prior Research: A 1958 publication by Ilya Derevsky reported similar all-female colonies in Armenia (3,000 observed lizards, no males).
    • Skepticism and Alternative Explanations:
      • Rejection of findings (some due to Cold War tensions).
      • Biased samples.
      • Prior copulation with males and sperm storage.
      • Hermaphroditism and self-fertilization.
    • Disproving Alternative Explanations:
      • Females reproduced after isolated hibernation.
      • Histology showed no hermaphroditism or stored sperm.
    • The Cloning Hypothesis: Lizards reproduced by cloning themselves (genetically identical offspring). Embryos developed from unfertilized eggs.
    • Testing the Hypothesis (pre-DNA analysis): Skin transplants (rejected in sexual species, accepted in these unisexual lizards).
    • Confirmation through DNA Studies: J. Cole and others conducted DNA studies after Carol Townsend developed methods to keep the lizards alive.
    • Origin of Cloning: Hybrid origin (mating of two related lizard species).
    • Lizard Zero: A specific female hybrid individual initiated the cloning lineage.
    • Reverse Darwinism: Sexual reproduction promotes variation, while these lizards regressed to cloning (no genetic variation).
  • Lost History: The originator of the Lizard Zero story could not be definitively identified.
  • Cole’s Involvement:
    • Graduate student (early 1960s) in the lab that discovered the all-female colonies.
    • Histology and genetics work in Arizona (1965).
    • Research program on the genetics of cloned lizards at the American Museum of Natural History (1969).
  • Museum Exhibit: Three whiptail lizard specimens on display (one from the all-female species, two from the parent species).
  • Insight Pathway: Contradiction (existing ideas contradicted by observations).
  • Key takeaway: Unusual observation resisted by the research field ultimately leads to insights.

Case Study: The Canny Consumers

  • Context: Market research for Procter & Gamble (P&G) to introduce a new detergent.
  • Target Consumers: “Economical homemakers” (female heads of household) who preferred less expensive detergents.
  • P&G’s Existing Model: These consumers buy the cheapest detergent.
  • Author’s Team’s Involvement: Hired to conduct interviews about tacit knowledge.
  • Initial Meeting: P&G team expressed low expectations, believing they already knew everything.
  • Author’s Response: Determined to achieve a breakthrough.
  • Interview Methodology:
    • Two teams, 12 homemakers interviewed over two days (two-hour interviews).
    • Mock supermarket aisle photograph (brands shown, prices withheld).
    • Interviewees asked for price information as they “shopped.”
  • Findings:
    • Pickier than expected: Homemakers considered quality alongside price.
    • Quality and Cost Consciousness: Concerned about both quality and value.
    • Bargain Hunting: Took pride in finding good quality at low prices.
    • Sophisticated Strategy:
      • Target Set: 3-4 acceptable brands.
      • Couponing and Sales: Used coupons and sought in-store bargains.
      • Purchase Decision: Cheapest brand from the Target set on the shopping day.
      • Rationale for Target Set Size: Balance between bargain opportunities and calculation effort.
  • P&G’s Previous Misconception: Homemakers were perceived as solely price-driven and unconcerned with quality.
  • Impact: The insights helped P&G achieve a successful product rollout.
  • Appreciative Inquiry: Respecting the intelligence of the subjects is crucial for gaining insights.

Appreciative Inquiry and Kobe

  • Example: Klein’s grandson, Kobe, and his parents’ underestimation of his abilities.
  • The Questionnaire: Rebecca (Kobe’s mother) found a question about understanding “Where’s Daddy?”
  • The Experiment: Rebecca taught Kobe about noses (Mama’s, Kobe’s, and then asking for Dada’s).
  • Kobe’s Response: Kobe located his father and grabbed his nose.
  • Lesson: Low expectations can hinder the discovery of insights. Appreciative inquiry is essential.

Appreciative Listening and the HMS Gloucester

  • Context: Operation Desert Storm (1990). Michael Riley (British naval officer) on HMS Gloucester protecting USS Missouri.
  • The Incident: Riley identified and ordered the shooting down of an Iraqi Silkworm missile.
  • Post-War Analysis: No evidence of the missile’s characteristics on radar recordings.
  • Riley’s Claim: Extrasensory perception (ESP).
  • Author’s Team’s Investigation: Two-hour interview and radar video review.
  • Initial Dead End: No evidence of acceleration.
  • The Breakthrough: The missile’s lower altitude caused it to break free from ground clutter farther out than usual.
  • Riley’s Misperception: Mistook the unusual appearance for acceleration.
  • Importance of Appreciative Listening: Taking Riley’s perception seriously, even though incorrect, led to the solution.
  • Previous Analysts’ Oversight: Focused on the images, neglecting Riley’s experience.
  • Riley’s Acceptance: Reluctantly agreed with the team’s explanation.
  • Key takeaway: Listen sympathetically and appreciatively, even when faced with incorrect information, to understand the reasoning.

Probing Checklist for Understanding Behavior

  • Knowledge: What did the person know or not know?
    • Examples: Devorah and the elderly gentleman, Mitchell and his client, Walter Reed and his colleagues.
  • Beliefs and Experience: Beliefs, perceptual skills, patterns, judgments of typicality (tacit knowledge).
    • Examples: Mike Riley and radar blips, Harry Markopolos and Bernie Madoff’s trades.
  • Motivation and Competing Priorities: What were the person’s goals and constraints?
    • Examples: Cheryl Kane and chocolate kisses, Dennis and the page job, P&G and the homemakers.
  • Constraints: What limitations influenced their actions?
    • Examples: Daniel Boone and the kidnappers, Ginger and the compliance officer.

Sticky Keys: Unpacking a Simple Insight

  • Initial chronology (not a story): Helen’s key doesn’t work, Gary assumes it’s the key, makes new keys, they don’t work, Gary solves the problem.
  • Detailed story (with insights):
    • Helen suggests lock problem. Gary dismisses it.
    • Gary makes new keys (multiple attempts).
    • Gary realizes his own key sticks.
    • Gary lubricates the lock, solving the problem.
  • Insight Story: Bad key frame replaced by bad lock frame (garden path story).
  • Garden Path Story: Tenaciously holding onto an erroneous frame despite contrary evidence.
  • Routinization of Deviance: Anomalous event repeated until it becomes the new normal.
    • Example: NASA Challenger disaster and O-ring scorching.
    • Gary’s increasing jiggling of the key.
  • Helen’s Interpretation: Gary doesn’t listen to her.
  • Alternative Interpretation: Gary honorably admitted his foolishness (but he had forgotten Helen’s initial suggestion).
  • Conclusion: Even simple incidents can reveal multiple layers of insight.

Chapter 18: The Magic of Insights

The Illusion of Magical Insights

  • Insights often appear magical because we only see the end result, not the process leading to it.
  • Graham Wallace’s model focused on the sudden “illumination” aspect of insight, contributing to the perception of magic.
  • This chapter argues that the process is less mysterious than it seems.

The Triple-Path Model of Insight

  • Klein proposes a triple-path model, offering a more nuanced understanding of insight:
    • Contradiction Path: Triggered by encountering conflicting information.
    • Creative Desperation Path: Arises from reaching an impasse and needing to abandon flawed beliefs.
    • Connection Path: Occurs when connecting seemingly disparate ideas.
  • Each path alters existing beliefs and understanding, leading to the “flash of illumination.”
  • Insights replace an old story with a new, more accurate and useful one.

Addressing Myths Surrounding Insight

  • The triple-path model helps clarify common misconceptions about insight. These myths often apply to one path but not the others.
  • Is insight a matter of breaking out of an impasse?
    • Yes, for the Creative Desperation path, but not necessarily for the Contradiction or Connection paths.
  • Does incubation help?
    • Evidence is mixed; if it helps, it may work differently for each path.
  • Does experience hinder insights?
    • Sometimes, on the Creative Desperation path, as ingrained beliefs can become obstacles. Not necessarily on the other paths.
  • Should we keep an open mind?
    • Yes, on the Connection path, to embrace new possibilities. However, the Contradiction path often benefits from a skeptical mind to evaluate conflicting information.
  • Should we expose ourselves to a “swirl” of different ideas?
    • Potentially beneficial for the Connection path, but not necessarily for the others.
  • Is insight a matter of connecting the dots?
    • Only applicable to the Connection path. Even then, it’s an illusion of hindsight, as we forget the irrelevant information.
  • It is crucial to specify which path is being discussed to avoid miscommunication.

Clear-Cut vs. Soft Insights

  • The triple-path model originated from 120 cases studied by Klein.
  • Many insights are “clear-cut,” having a definitive solution:
    • Examples include scientific discoveries (DNA structure, disease investigations, financial fraud investigations, astronomical discoveries like pulsars).
  • “Soft” insights often involve people and lack a single, verifiable solution:
    • Examples include interpretations of historical events (Battle of Taranto, Napoleon’s tactics), sports strategies, financial market predictions, and psychotherapy diagnoses.
  • While soft insights cannot be definitively validated, they are still valuable.
  • The triple-path model applies to both clear-cut and soft insights.

Major vs. Minor Insights

  • Insights vary in their impact:
    • High-impact insights: Darwin’s theory of evolution, Yamamoto’s Pearl Harbor attack strategy, Einstein’s theory of special relativity.
    • Everyday/Minor insights: Personal problem-solving (car keys, sticky house key), workplace solutions (chocolate kisses for time cards), fantasy baseball strategies, coaching advice.
  • The triple-path model applies to both major and minor insights.

A Richer Understanding of Insight

  • Klein’s research has led to a deeper understanding of insight, beyond the Graham-Wallace stages.
  • This includes:
    • A more comprehensive description of insight.
    • Understanding obstacles to insight.
    • How the focus on reducing errors (“down arrow”) can hinder insights.
    • Strategies for improving the chances of gaining insights.

Fostering Insights

  • For Individuals:
    • Contradiction Path: Be open to surprises and willing to challenge existing beliefs.
    • Connection Path: Be open to new experiences and speculate about unfamiliar possibilities.
    • Creative Desperation Path: Critically examine assumptions to identify flawed beliefs.
  • Helping Others Gain Insights:
    • Listen empathetically to understand flawed beliefs.
    • Avoid labeling people as “stupid.”
    • Become a detective to uncover the flawed belief.
    • Help the individual replace the flawed belief.
  • For Organizations:
    • Diagnose what hinders insights.
    • Reduce excessive controls and procedures that stifle insights in the name of error reduction.
    • Recognize that organizations often prioritize predictability and abhor mistakes, thus stifling innovation.
    • Cut back on practices that interfere with insights.
    • Address organizational fear of the unpredictable nature of insights.
    • Establish oversight groups to act as a “court of appeals” for potentially valuable but disruptive insights.
    • Be willing to act on insights once they are identified.

The Nature of Insights

  • Popular usage of the term “insight” often refers to shallow bits of information, not true shifts in understanding.
  • True insights involve a discontinuous shift in our understanding, a change in the story we tell ourselves about how the world works.
  • This shift requires discarding or modifying core beliefs.
  • Insights are unpredictable leaps to new understanding, often surprising and not derived from conscious, deliberate reasoning.
  • Insights are usually unique to an individual, even if others have the same information.
  • Insights often arrive as a coherent new story, sometimes complete (Watson and Crick’s DNA structure) and sometimes as a starting point for further investigation (Markopolos’ Madoff investigation, Gottlieb’s identification of a new disease).

The Transformative Power of Insights

  • Insights can lead to significant changes:
    • Career changes: Many examples throughout the book demonstrate how insights have led individuals down new career paths.
    • Personal changes: Insights can lead to profound shifts in personal beliefs and behaviors (e.g., the firefighter who realized he was the “attitude problem”).
  • Insights not only provide new understanding but also change how we view the world, our mental equipment, capabilities, priorities, and goals.

Habits of Mind and the Up Arrow

  • Three habits of mind drive insights:
    • Noticing connections, coincidences, and curiosities.
    • Going “tilt” at inconsistencies.
    • Weeding out flawed beliefs.
  • These habits of mind can be considered forces that drive the “up arrow” of discovery and innovation.
  • The heuristics and biases community has focused on the downsides of these habits (e.g., seeing false connections), but Klein emphasizes their crucial role in making discoveries.
  • These habits of mind combat mental rigidity that can arise from experience and successful beliefs.
  • They disrupt routine thinking and foster discovery.

The Irrepressible Nature of Insights

  • The “up arrow” of insight has its own inherent power and cannot be eliminated.
  • We are intrinsically driven to seek insights, evident in our fascination with projects, hobbies, and social relationships.
  • Insights are acts of creation, bringing forth new ideas that didn’t exist before.

Reverence for Insights

  • The ancient Greeks revered the capacity for insight, honoring the Muses as goddesses of creative inspiration.
  • While demystifying insights, Klein emphasizes the importance of appreciating the awe and wonder they inspire.
About Me:

I’m Christian Mills, a deep learning consultant specializing in practical AI implementations. I help clients leverage cutting-edge AI technologies to solve real-world problems.

Interested in working together? Fill out my Quick AI Project Assessment form or learn more about me.