In late May, I went on a podcast to discuss late Harvard Professor Clayton M. Christensen's groundbreaking management book The Innovator's Dilemma. The book is considered one of the most influential management books that has shaped businesses and industries across sectors since it was first published in 1997. It explores why successful, well-managed companies often fail when faced with disruptive technological changes.
In this article, I'm sharing my notes from the book that I took while preparing for the podcast in the hope that it will encourage more of you to read the book, if you have not already. I will share more about the podcast when it comes out.
The book is divided into two parts.
Part one explains and establishes the theory of why well-run companies fail in the face of disruptive technologies.
- These companies often fail because the very management that has allowed them to become industry leaders also makes it extremely difficult for them to develop the disruptive technologies that eventually steal away their markets.
- Well managed companies are very good at developing the sustaining technologies that improve the performance of their existing products. This is because it aligns with their incentives and strengths:
- Listen to customers and give them more than they ask for
- Earn higher margin
- Serve existing customers better
- Target larger market to grow faster
Part two provides potential remedy to that challenge.
- Build autonomous organization to explore disruptive technologies
- Find ways to align incentives
Interesting observation: Christensen suggests in the book that dealing with disruptive technologies is not a technology/innovation problem, rather a marketing problem. Disruptive technologies often start small and scrappy and serve a different market segment who find the product useful. In order to make these products work, a company has to seek out and find these new customers, which most successful companies usually don’t want to do. More so, when this new market is undefined and appears small and less profitable and their existing market is more lucrative.
I think the challenge is also psychological. It is hard for the management of a successful company to approve investment in an unknown technology or business that may or may not produce results instead of investing in their current business which is producing immediate good results. While everyone talks about how important long term thinking is, we always prefer immediate results over long-term uncertain success. For a CEO of a successful public company, it makes more sense to do things that would propel the share price of his company up than doing something that may produce huge upside for the company in the long run. The world is designed in a manner where it is hard to do this.
Code Ideas
What is technology: The process by which an organization transforms labor, capital, materials, information and other resources into products and services of higher value.
The Innovator’s Dilemma: Successful companies listen to their best customers and invest in high-margin products, leaving them vulnerable to disruptors that target overlooked or low-end markets. By the time the disruptive technology improves enough to threaten the mainstream, it’s often too late for incumbents to respond effectively.
The Paradox of Good Management: Companies fail not despite good management, but because of it. The very practices that make companies successful in sustaining innovation contexts become liabilities during disruption.
Two types of innovation
Disruptive Innovation vs. Sustaining Innovation
- Sustaining Innovations improve existing products for current customers (e.g., faster processors in computers).
- Disruptive Innovations are often cheaper, simpler, and initially inferior but eventually improve to displace established products (e.g., digital cameras replacing film cameras).
- Two Types of Disruptive Innovation:
- Low-End Disruption: Targets over-served customers at the low end of an existing market with a "good enough" product at a lower price. (e.g., discount airlines)
- New-Market Disruption: Targets customers who were previously non-consumers, making a product or service accessible and affordable for a new market. (e.g., personal computers for home users)
The Disruption Process:
A predictable pattern of how disruption unfolds:
- Disruptive technology emerges with inferior performance on traditional metrics
- Established companies ignore it because their customers don't want it
- New entrants serve overlooked segments or create new markets
- Technology improves rapidly and eventually meets mainstream needs
- Established players lose market share as customers migrate
Failure Framework: Components
- The difference between disruptive innovation and sustaining innovation has strategic implication
- Technology improves faster than market need
- Rational decision making and resource dependence influence the kind of opportunities make attractive to successful companies
Principles of Disruptive Innovation/The Problem for Incumbents:
- Companies depend on customers and investors for resources/The theory of resource dependence: Successful companies are heavily reliant on their existing customers and investors for resources. These stakeholders demand continued improvement in existing products and short-term financial returns, making it difficult for companies to justify investing in disruptive technologies that initially have lower profit margins and smaller, unproven markets.
- Small markets don't meet growth needs of large companies: Disruptive innovations often start in niche markets that are too small to be attractive to large, established companies with high revenue growth targets. By the time these new markets grow large enough to be interesting, the disruptive technology has already matured, and new entrants have gained a significant lead.
- Markets that don't exist cannot be analyzed: It's impossible to conduct traditional market research for disruptive technologies because there are no existing customers or established markets to survey. This uncertainty makes it difficult for incumbents to make data-driven investment decisions.
- An organization’s capabilities define Its disabilities: A company's processes and values, which are optimized for their current business, can become "disabilities" when faced with disruptive innovation. These ingrained ways of working make it difficult to allocate resources or create new processes for something that doesn't fit the existing model.
- Technology improves faster than market need/Technology supply may not equal market demand: This has two aspects to it. On the one hand, companies often develop technologies faster than what the customers need. For instance, a bank offering video streaming in its app, which its customers might not be looking for but the bank still offers it in the hope that it will improve retention. This is a clear overshoot. This is a common outcome when a successful and well-managed company continues to invest in sustaining technology to make its customers happy. Technology can overshoot the market need. This is also true for disruptive technologies. Since technology matures faster, disruptive technologies often mature faster than the market expectation, becoming ready to compete with incumbents within a relatively short time.
The Innovation Stack
Three levels where innovation can occur:
- Product/service level: Core functionality
- Business model level: How value is created and captured
- Value network level: The broader ecosystem of suppliers, partners, and distribution
The Failure Process
Rational resource allocation
- Companies invest where returns are highest and most certain
- Disruptive technologies initially offer lower margins and smaller markets
- Resource allocation processes systematically reject disruptive projects
- This is economically rational behavior
Customer and investor captivity
- Successful companies become dependent on their best customers and investors
- These customers don't initially want disruptive technologies
- Companies rationally focus on what customers are asking for
- They miss opportunities their customers can't yet articulate
Organizational capabilities mismatch
- Companies develop capabilities optimized for their current value network
- Disruptive innovations require different capabilities:
- Different cost structures
- Different performance metrics
- Different market approaches
- Existing organizational processes and values reject projects that don't fit
Market size misjudgment
- Large companies need large opportunities to move the growth needle
- Disruptive markets start small and appear unattractive
- By the time markets become large enough to matter, it's often too late
- New entrants have established positions and learned faster
S-Curve of technology development
- Technologies improve faster than customer needs evolve
- This creates "performance surplus" - products become "good enough"
- Opens door for simpler, cheaper alternatives that were previously inadequate
Solutions and strategies for incumbents to deal with disruptive technologies
Christensen suggests that companies can mitigate the innovator's dilemma by:
- Create autonomous organizations: The most important recommendation is to spin off new, independent organizations or teams specifically tasked with nurturing disruptive innovations. These entities should have:
- Different cost structures and revenue expectations.
- A focus on new customers and markets.
- The freedom to experiment and learn from failure.
- Protection from the dominant culture and financial pressures of the parent company.
- Match the size of the organization to the size of the market: Small, nascent markets for disruptive technologies should be pursued by small, agile teams that can be profitable at lower revenue levels.
- Fail early and inexpensively. Embrace failure and learning: Recognize that disruptive innovation involves a high degree of uncertainty and requires experimentation, iterative learning, and a willingness to fail fast and cheaply.
- Targeting new customers: Focus on finding new markets for disruptive technologies rather than trying to force them into existing markets where they may not initially meet performance requirements.
- Strategic acquisition: Acquire smaller companies that are developing disruptive technologies and allow them to operate independently.
- Leverage existing resources without importing disabling processes/values: The new autonomous unit can still draw upon the parent company's resources (e.g., funding, technological expertise) but must be shielded from its ingrained processes and values that would stifle the disruptive innovation.
The "S-Curve" of Technological Progress and Market Demand:
- This framework illustrates that technologies progress through an S-shaped curve, with initial slow improvement, followed by rapid acceleration, and then a plateauing as the technology matures.
- Christensen highlights that the rate of technological progress often outstrips the rate at which market demand increases. This means that existing products eventually overserve the market. When a sustaining technology reaches a point where it offers more performance than mainstream customers can utilize, it creates an opening for a disruptive technology to enter at a lower performance point with a different set of attributes (e.g., lower cost, greater convenience) and gain a foothold.
The Value Network: This refers to the context within which a company operates, including its customers, suppliers, distributors, competitors, and the financial structure that dictates acceptable gross margins. Companies tend to optimize their processes and cost structures to thrive within their existing value network. Disruptive innovations often emerge from or create new value networks, making it difficult for incumbents to operate effectively within them without changing their fundamental structure.
Coda
One of my realizations from reading and rereading The Innovator’s Dilemma is that great books are simple and great ideas are often so obvious that when someone presents them to us we feel that “ah! that makes complete sense; how could I miss it”. This has been the case every time I read a great book; be it The Innovator’s Dilemma or Good to Great or Zero to One and many other similarly influential books. The hard part, however, is that executing a simple idea is not simple. It is often complex and takes more than good management and mere abilities.
