Over the last two editions, we looked at two of the most common experimentation mistakes.

First, the 'what vs why trap': how teams conflate “what happened” with “why it happened,” producing false learnings that compound in the wrong direction.

Then we examined why most growth experiments flop; and why the answer isn't bigger, bolder, or more.

Both pieces diagnosed the problem. This one’s about what to do instead.

At the highest level, growth work has two layers: Problem Discovery and Solution Discovery. Problem Discovery determines whether you're working on the right thing. What problems exist, why they exist, which levers influence them, and which one will most efficiently get you to your goal. Solution Discovery is everything after: ideating, prioritizing, testing, and learning.

Most teams spend nearly all of their time on Solution Discovery. The features and tests being shipped. Choosing the perfect prioritization framework, testing stack, etc.

That’s the gap this series has been about.

The bad doctor problem

Here’s a scenario I see constantly. A team looks at their funnel data, sees a big drop-off at, say, checkout, and concludes: “Our checkout page is the problem. Let’s optimize it.”

They Google the latest CRO tactics. They redesign the page. They run a dozen A/B tests. Nothing moves.

Why? Because they acted like the bad doctor. They saw a symptom (low checkout conversion) and jumped straight to a prescription (optimize the page). They never stopped to diagnose the actual problem.

Maybe the checkout page is fine. Maybe the drop-off exists because a page upstream didn’t adequately communicate value. Maybe 40% of the traffic came from dogs who hijacked their owners' laptops.

They’ll never know, because they skipped the diagnosis.

If you caught last week's newsletter, you'll recall how I made some of the very same mistakes early in my career at Grammarly.

What finally worked was stopping and doing the diagnostic work. I talked to customer support. I studied what visitors were seeing in search results before they reached our page. I looked at how our product, pricing model, and competitive landscape all fit together.

Every signal pointed to the same thing: visitors needed to know the product was free, and they needed to see it exactly where they were making the decision. Two words on the CTA button. 8x the lift of any other test I’d ever run.

The diagnosis is what made the solution obvious.

What happens when you get problem discovery right

Over at Lenny's Newsletter, Duolingo’s former CPO, Jorge Mazal, shared the story of how they reignited growth — taking daily active users from single-digit growth to a 4.5x increase over four years (as a mature product, at that!). It’s a great case study, and it happens to perfectly illustrate the power of Problem Discovery.

Their first two attempts skipped it entirely. They saw a problem (growth is stalling) and jumped straight to solutions. They borrowed a game mechanic from Gardenscapes that flopped. They copied Uber’s referral program, which excluded the exact users it depended on. Months of effort, minimal impact. Classic Solution Discovery without Problem Discovery.

The turning point: they stopped shipping and started diagnosing. First, they built a detailed growth model, segmenting their user base by engagement level and mapping how users flowed between segments daily. Then they ran a sensitivity analysis: if we improved each lever by the same amount, which one moves our North Star the most?

The answer was clear: current user retention rate (CURR) was 5x more impactful than the second-best lever. That single finding reoriented their entire strategy. They created a dedicated retention team, deprioritized new-user retention efforts, and focused everything on keeping current users coming back.

That's Problem Discovery. Or at least half of it. They nailed the first part: using a growth model to identify the highest-impact lever. Or, per last week's newsletter, to identify the 'what.' But, like so many teams, they stopped short of pursuing the 'why.'

Where even good teams fall short

Now, it's worth noting: from Jorge's account, they went from identifying the lever to jumping back into solutions fairly quickly. There's almost certainly more to the story than what he covered. We know as well as anyone that it's hard to capture all the nuance in a single essay. But what he chose to share is instructive, because it mirrors what I see most teams do.

Even after doing solid Problem Discovery work to find the right lever, they appear to have skipped the second half: diagnosing why that lever was stuck. Their solution process relied heavily on a single source of evidence — what had worked at other companies (Zynga, in particular). Jorge hypothesized, based on his experience at Zynga, that leaderboards would also work at Duolingo.

It happened to work. But relying on a single evidence source is risky. What if leaderboards hadn’t addressed the underlying issue? They might’ve tried a dozen more features borrowed from gaming companies and gotten nowhere — because they never fully investigated what was actually causing their best users to disengage.

Side note: I'm a firm believer that intelligently borrowing from other companies is one of the most impactful growth tactics out there. But it's much harder done than said. Jorge actually speaks to this in his essay: "I realized that I had been so focused on the similarities between Gardenscapes and Duolingo that I had failed to account for the importance of the underlying differences." That's the whole challenge in one sentence. On the surface, two companies can look extremely similar, yet the tactics that worked for one still completely fail for the other. The question is: what are the underlying differences, and how do you systematically identify them? I've built a framework for exactly this that we use with our agency clients, and it's one of the biggest edges we give them. I've never published it, but I'm planning to in a future edition.

The diagnostic process

The growth community does a great job teaching Solution Discovery: form a hypothesis, design a test, analyze results. What it largely skips is the Problem Discovery process that should come before any of that.

Here’s how it works:

1. Define the goal and constraints.

Where are you trying to go, how much can you invest, and by when? “Grow revenue” isn’t a goal — it’s a direction. “Add $400k in MRR within 90 days without increasing costs more than $50k/month” is actionable. Without constraints, you can’t evaluate tradeoffs between problems worth solving.

2. Map the levers.

Build a growth model. Break your North Star into its inputs. Revenue = customers × revenue per customer. Customers = leads × conversion rate. Leads = visits × sign-up rate. Keep breaking it down until you reach levers you can actually influence.

3. Identify high-impact levers.

You need to understand which levers have the greatest influence on your North Star. How you do this depends on your stage and data. A mature product like Duolingo can run a formal sensitivity analysis by applying a uniform increase to each lever and modeling the output. An earlier-stage company might work backwards from the goal: if we need $50k in new monthly revenue, what are the possible levers, and which ones does our model suggest are most efficient?

Either way, the point is the same: let the math narrow the field instead of guessing which lever "feels" most important.

4. Select the problem worth solving.

Knowing which levers have the most mathematical impact doesn't tell you where to focus. Not in isolation.

You also need to consider: can we actually move this thing? How much would it cost? Is there evidence it's genuinely underperforming, or does the math just make it look attractive?

The right problem to solve isn't always the highest-impact lever. It's the one that best balances impact, feasibility, and evidence — all while staying within the constraints.

5. Diagnose before you prescribe.

This is the step everyone skips. You’ve selected the lever. Don’t brainstorm solutions yet. Ask why that lever is stuck.

Talk to users. Talk to customer-facing teams — support, sales, anyone who hears directly from customers. Study historical data: has this metric always been this way, or did something change? Look at how other companies have solved similar problems. Watch session recordings. Examine what’s happening upstream in the funnel.

You’re building a case, not forming an opinion. The more evidence you layer, the closer you get to the root cause. And the closer you get to the root cause, the more obvious the right solution becomes.

6. Now enter Solution Discovery.

This is where the conventional growth playbook kicks in — and where it actually works, because it's grounded in real evidence. Now we can form testable hypotheses, run them through prioritization frameworks, design experiments, and start generating learnings that help us hone our hypotheses and get closer and closer to the truth.

Steps 1–5 are Problem Discovery. Step 6 is Solution Discovery. Most teams start at step 6. Some start at step 3. Almost nobody does steps 4 and 5. And that's where the real leverage lives.

Why this changes everything

Getting Problem Discovery right doesn’t just improve your hit rate on experiments. It changes the magnitude of outcomes.

Two words at Grammarly — 8x lift compared to far more complicated tests that preceded it. From stalled to 4.5x DAU growth at Duolingo. These aren’t normal results. They happened because the teams did the diagnostic work and landed on the right lever(s) before building anything.

When you understand the problem deeply, even small solutions can produce massive results. When you don’t, even massive solutions produce nothing.

Wrapping up the series

I hope you've enjoyed the three-part series on experimentation. It's a big topic. I did my best to make these editions as simple and actionable as possible, but if you have any questions, just shoot us a reply.

If there's one thing to take away from all of this:

The teams that consistently produce outsized results don’t have a better Solution Discovery process. They have a better Problem Discovery process. (Honestly, having one at all puts you far ahead of most organizations.)

When in doubt, slow down. Focus less on testing and more on understanding what problems exist, why they exist, which ones actually matter, and the levers that influence them.

​Justin Setzer​
Demand Curve Co-Founder & CEO