Here’s the secret — It doesn’t just happen, it takes a lot of effort. There is a way to do it. When people do it successfully, we call them genius. This is a deep dive into what an insight is, and how geniuses end up finding them.


It’s July 2003, the iPod’s been in the market for a couple of years. Their “Rip, Mix, Burn” campaign has given birth to the ethos of the download.

An entire generation of kids (including me) believe that they don’t ever have to pay for music.

And Steve Jobs announces the iTunes Music Store — the first legal music service to offer Internet downloads from the catalogs of all five major music companies.

Here’s what he has to say about that:

Interviewer: There’s a whole generation that thinks music is free.

JOBS: I don’t think that’s true. I talk to a lot of these kids. They don’t think it’s free. They know they’re stealing music.

Interviewer: But is karma enough of a threat?

JOBS: No. Let me try explaining it another way. These kids are using the best product. Until yesterday, Kazaa was the best product. Why is that? Because the minute you get your music over the Internet and experience that instant gratification, there’s no going back. … I don’t blame these fifteen-year-old kids. I blame us — for not coming up with a better product that was legal. And so they’ve been using the best product out there, and what we have to do to compete is make a better product. And I believe people will gravitate to a better product. …

An excerpt from his Interview with Esquire, the day after he announced the iTunes Store.

The Insight: “People pirate music because it’s not easy for them to access it legally.”

iTunes, as we all know, goes on to become the foundation of Apple’s digital hub strategy. This insight holds true almost two decades ago later foreshadowing what we see today with Netflix, Disney+, and HBO’s streaming war.


What is an insight?

An insight is an explanation — Think of it as a statement with a “because” in the middle. If you want a more formal definition, Wikipedia says: “An insight is the understanding of a specific cause and effect within a specific context.”

Examples:

  • You clicked on this article because it talks about Steve Jobs.
  • You are reading this because you want to take a break from work.

Insights help make predictions, like “If we explain how to find insights like Steve Jobs, you’ll want to sign up for our beta.”

The problem with insights, however, is that they are not facts. This means they may false, and lead to inaccurate predictions — you don’t care about Steve Jobs, and don’t end up clicking on our call to action.

They can be only be measured in terms of the sets of observations and the relationships between them that they can meaningfully explain, and the accuracy of the predictions made based on them.

In other words, an insight is good if it explains all relevant observations and helps make accurate predictions.

Here are a few examples of good insights:


Example #1 — The earth is round.

The idea of a flat earth was a common belief across most ancient cultures. It made sense — everything looks flat; ships would sail off toward the horizon and often never return[6].

Observations:

However, Greek scientists around the 4th century BC observed[5]:

  1. That ships disappeared over the horizon while their masts were still visible.
  2. The earth had a curved shadow on the Moon during lunar eclipses.
  3. The position of the stars changed in the sky.

Explanation:

The idea that the earth was flat couldn’t explain these observations, but imagining it as a sphere did. So they concluded that the Earth was, in fact, a sphere.

Prediction:

A diagram showing how Eratosthenes measured the Earth, accessed from Simon Fraser University Online

Based on this insight, the Greek scientist Eratosthenes was able to predict that earth’s curvature would cause the angle of the reflection of Sun’s rays to be different between Syene and Alexandria (two cities around 800km apart).

They used the difference to calculate the circumference of the earth.


Example #2 — Species evolve.

Charles Darwin suddenly turned all of biology upside down in 1859 with the publication of the Origin of Species.

Before him several scientists had made a compelling case that life had been on earth for a long time, that it had changed over that time, and that many species had become extinct. But no one had a good explanation for why[12].

It was Darwin’s genius both to show how all this evidence favored the evolution of species from a common ancestor and to offer a plausible mechanism by which life might evolve.

Observations:

He did so based on the following observations[14]:

  1. There is variation in traits — For example, some beetles are green and some are brown.
  2. There is differential reproduction. — Green beetles tend to get eaten by birds and survive to reproduce less often than brown beetles do.
  3. There is heredity. — The surviving brown beetles have brown baby beetles because this trait has a genetic basis.

Explanation:

Since the environment can’t support unlimited population growth, not all individuals get to reproduce to their full potential. The more advantageous trait, brown coloration, which allows the beetle to have more offspring, becomes more common in the population.

Prediction:

With the insight above, he could predict — If this process continues, eventually, all individuals in the population will be brown.

The theory we commonly refer to as “Natural Selection” and it is the foundation of our understanding of evolution.


Example #3 — People pirate for instant gratification.

Looking back at Steve Jobs’ interview above, you can see he offers an explanation for why kids pirate based on the following observations he made by “talking to kids”:

Observations:

  1. They don’t think it’s free.
  2. They know they’re stealing music.
  3. They get instant gratification by downloading music over the internet.

Explanation:

There is no legal product that offers them the instant gratification they have come to expect.

Prediction:

If people get instant gratification by downloading music over the internet legally, they would stop pirating. They launched iTunes, and we all know how successfully Apple’s digital hub strategy worked for them over the last 15 years.


Where do good insights come from?

Good insights come from looking for people who behave like the ancient greeks

The world’s biggest record companies in the early 2000s were behaving like the Ancient Greeks who believed that the earth is flat in 762BC.

The Greeks speculated that the Earth was a large disc surrounded by a gigantic body of water. Their speculation had its roots in mythology. That Oceanus limited the habitable world from the underworld[8].

With this speculation, Homer decided to send Odysseus to traverse it in order to arrive in the realm of the dead.

Meanwhile, the world’s biggest record companies believed that punishing people was the best way to stop piracy. They won the court battle to shut down Napster, we’re unable to fight the spread of piracy. There was no evidence to suggest that legal measures could reduce piracy[3].

They started investing millions in building software programs that would sabotage the computers and Internet connections of people who download pirated music![2] 😲

Jobs recognized this.

And then doing science like Darwin.

Many biologists had promoted evolutionary theories, but in order to explain just how life changed, they depended on speculation. It was a question people in the scientific community were actively trying to answer at the time.

German biologists believed life evolved according to predetermined rules, in the same way, an embryo develops in the womb. Another scientist Lamarck, for example, thought that life strove over time to rise from simple single-celled forms to complex ones[11, 12].

They had observed evidence but were unable to soundly describe the relationships between them. By doing science, both Darwin and the British biologist Alfred Russel Wallace independently conceived of the same explanation, the theory we call natural selection[12].

It’s not easy to find good insights, even if you know where they come from.


Why is it hard to find good insights?

Because you have to be a genius.

Aristotle, Darwin, and Jobs were all geniuses.

In his book, “On the Heavens” he covers the nature of the universe — introduces dimensionality of objects, states of matter, a rudimentary form of Newton’s first law of motion and the Geocentric Model. His ideas lay the fundamental framework for people like Galileo, Copernicus, and Newton[10].

Darwin’s ideas were largely based on direct observations. He traveled around the globe from 1831 to 1836, studying and cataloging the local plants and animals. Over the course of his travels, he began to see intriguing patterns in the distribution and features of organisms[14].

And of-course, Jobs led the creation of some of the most revolutionary products in technology by identifying key market trends at each point in time. Be it, how the computer could become the bicycle for the mind, or how animation technology could change the movie industry forever.

All their contributions were absolutely insane.

And being a “Genius is one percent inspiration and 99 percent perspiration.”

Inspiration is arbitrary, it can come from anywhere and comes only after you have the intent to do something.

Aristotle inspired by Plato and Pythagoras, who proposed the idea based on aesthetic grounds — Sphere’s look good[9].Darwin found his in economics — from an Essay on the Principle of Population[13]. Jobs allegedly got his from Richard Branson’s Prank in 1986[4].

And the 99 percent perspiration is really hard.

What makes people geniuses though is their uncanny ability to find patterns in a vast amount of information. They can organize, make sense of, recall and make connections between ideas and concepts us ordinary folk would really struggle with[16,18].

And this is 99% of the work that sets them apart.

Here’s an example from Darwin’s work — from the voyage of the Beagle[14]:

Darwin’s finches by Gould

Darwin’s seminal book, On the Origin of Species, was largely based on direct observations from his travels around the globe.

From 1831 to 1836, he was part of a survey expedition carried out by the ship HMS Beagle, which included stops in South America, Australia, and the southern tip of Africa. At each of the expedition’s stops, Darwin had the opportunity to study and catalog the local plants and animals.

Over the course of his travels, Darwin began to see intriguing patterns in the distribution and features of organisms. We can see some of the most important patterns Darwin noticed in distribution of organisms by looking at his observations of the Galápagos Islands off the coast of Ecuador.

Darwin found that nearby islands in the Galápagos had similar but nonidentical species of finches living on them. Moreover, he noted that each finch species was well-suited for its environment and role. For instance, species that ate large seeds tended to have large, tough beaks, while those that ate insects had thin, sharp beaks. Finally, he observed that the finches (and other animals) found on the Galápagos Islands were similar to species on the nearby mainland of Ecuador, but different from those found elsewhere in the world.

Darwin didn’t figure all of this out on his trip. In fact, he didn’t even realize all the finches were related but distinct species until he showed his specimens to a skilled ornithologist (bird biologist) years later! Gradually, however, he came up with an idea that could explain the pattern of related but different finches.

According to Darwin’s idea, this pattern would make sense if the Galápagos Islands had long ago been populated by birds from the neighboring mainland. On each island, the finches might have gradually adapted to local conditions (over many generations and long periods of time). This process could have led to the formation of one or more distinct species on each island.

If this idea was correct, though, why was it correct? What mechanism could explain how each finch population had acquired adaptations, or features that made it well-suited to its immediate environment?

During his voyage, and in the years after, Darwin developed and refined a set of ideas that could explain the patterns he had observed during his voyage. In his book, On the Origin of Species, Darwin outlined his two key ideas: evolution and natural selection.

– Excerpt from Darwin, evolution, & natural selection

Their body of work may be different, the fact that each one of them could repeatability find good insights means a set of steps they follow works.


How to find good insights?

Now that we have an understanding of what good insights are, know what goes into finding them, let’s dive into the steps that can lead us to them.

Like Darwin, we need to systematically synthesize our observations

Most of our observations come from reading, listening, taking notes or looking at things. All of these are qualitative. To become geniuses, we have to find patterns in this data. But being a genius is a lot of hard work.

A fifteen-minute conversation with someone transcribes to about 5 pages of raw data. And finding patterns in qualitative data is time-consuming and difficult.

When we draw things on the whiteboard, create mind-maps, make lists or put up sticky notes on the wall — we’re trying to organize all the observations we’ve gathered into something that makes sense, something we can act upon.

Therefore to find good insights, all we need to do is be able to analyze all the qualitative observations we make and find those patterns in a codifiable and repeatable way quickly.

We can do that by applying Thematic Analysis to our observations

In the late 1960s, brilliant scientists figured out that there was a gap between theory and empirical research[19]. That in the scientific method, there was no answer to what are the steps between making observations and defining a hypothesis other than “be a genius”.

Qualitative research methods were not seen as adequate methods of verification — that were open to subjective interpretation and bias. To solve this, they came up with a methodology called the Grounded Theory.

It’s a systematic way of gathering and analyzing qualitative data by deconstructing, categorizing, finding patterns and reorganizing it into theories that can be used to make predictions.

The steps involved in applying thematic analysis are:

  1. Deconstruct qualitative data into smaller units,
  2. Classify and cluster the similar units together to compare them more closely,
  3. Think about why the units belong together,
  4. And how clusters connect with each other.
  5. Evaluate if the explanation makes sense for all relevant observations.
  6. If they don’t reclassifying and recluster them

Once all the relevant observations are classified and clustered in a way where the explanation makes sense. You’ve got yourself an insight.

If you’re struggling to find patterns in your observations, you’ve not gathered enough of them so identify the units which are isolated and gather more observations that connect with them.

These steps outline how you can apply Thematic Analysis on any qualitative data set. But if you need a more context — I’ve written about how to apply it to qualitative data from user interviews.


Key Takeaways

Insights are sensible explanations of all relevant observations within a specific context. They help us make accurate predictions.

They can be found by identifying observations, trends or behaviors that don’t have a good explanation, and then systematically gathering and analyzing those observations.

One way to find them analyze and organize observations is by applying thematic analysis. Which helps identify patterns in the data by classifying and clustering the observations in different ways.

The process of finding insights from observations can be very time-consuming. So we decided to work on Epiphany to help you guys do it faster.

For those of you who are actively doing discovery — let us know by requesting access to our beta, and we’ll help you organize your findings, so you can find good insights more easily.

And do Subscribe to our blog for more on user research, user experience, design thinking, lean startup, dual-track agile, product management, and innovation.


References:

  1. Is Steve Jobs the God of Music?
  2. Software Bullet Is Sought to Kill Musical Piracy
  3. The Entertainment Industry v. Music, Movie Piracy
  4. Richard Branson Thinks His 1986 Prank Inspired Steve Jobs To Create iTunes
  5. Eratosthenes of Cyrene
  6. Flat wrong: the misunderstood history of flat Earth theories
  7. WHY DO SOME PEOPLE BELIEVE THE EARTH IS FLAT?
  8. The Cosmological Ideas of the Greek
  9. How the Ancient Greeks Knew the Earth Was Round
  10. Aristotle’s On the Heavens
  11. On the Origin of Species
  12. Natural Selection: Charles Darwin & Alfred Russel Wallace
  13. The Ecology of Human Populations: Thomas Malthus
  14. Darwin, evolution, & natural selection
  15. What makes a genuis
  16. See the World Through Patterns
  17. Ray Kurzweil’s How to Create a Mind
  18. Expertise in Problem Solving
  19. Grounded theory
  20. Hawkins, Jeff (2004). On Intelligence (1st ed.). Times Books. p. 272. ISBN 978–0805074567.