Product Validation. We all agree, it’s important, right? Right?
Right.
But we don’t seem to give it enough credit in practice? Or, in other words, we don’t validate our product enough.
Let me explain.
Liar, Liar, Phone on Fire.
Yeah, the Fire Phone.
Bezos wanted it. Amazon launched it. Consumers said: Nah.
As a result, Amazon recorded a $170 million charge related to Fire Phone inventory and supplier commitments, with $83 million in remaining inventory-related amounts at the end of the third quarter of 2014.
The Fire Phone dream died before it had a chance.
So, why the heck didn’t consumers appreciate a device with 3D-like visual effects and Firefly available to their amusement? Maybe they did. But perhaps not enough to pay a premium for them?
By 2014, smartphones were already an established market. Price, functionality, and the strength of the app ecosystem mattered. Amazon’s distinctive features may have been interesting, but they weren’t compelling enough.
Amazon thought they were what people would love. People spoke their part and rejected the product.
The Choice of a New Generation
When Pepsi started stepping on Coca-Cola’s toes, The Coca-Cola Company decided to get innovative. After extensive taste testing, it put its best minds together and came up with the new taste: a sweeter recipe. The relaunch was big. And less than three months later… Coca-Cola brought back the original formula as Coca-Cola Classic. The new generation wanted the old Coke.
So, what went wrong? Didn’t they test the new product extensively? Sure. But just the fact that taste tests involving nearly 200,000 consumers favored the new formula didn’t mean that validation was successful. Coca-Cola validated a product attribute, but not the product.
People get used to things. The original Cola was almost iconic. The replacement was not really about the taste as much as legacy and tradition. When timeless things go away, people who lean on them may lose the feeling of being grounded.
I can go on with examples. We don’t need to go too far back in time. Let’s just recall the Cracker Barrel logo.
The Most Expensive Mistake
…in business is building something nobody needs.
Even when the team is small, digital products are expensive to build. Development hours accumulate. Requirements expand. Integrations introduce complexity. And once engineering is in flight, changing direction becomes increasingly costly.
But how many times have I seen businesses take the idea straight to development, treating product requirements as though they were evidence that the product will succeed?
All the initial business and technical requirements are only a bunch of assumptions about what can make the product functional. But business requirements are not the market requirements.
In fact, any digital product is where business goals meet the market (customer) needs. The initial product idea may transform dramatically after you run the initial concept validation.
Even Google can learn it the hard way. Yes, Google.
Google Wave was an ambitious web application combining real-time communication and collaboration with live character-by-character editing. Did you hear about Google Wave? In 2010, Google announced that it would stop developing it as a standalone product. Users were not all that excited about it.
The Proof of Life. Or, Rather, Viability.
Not every uncertainty deserves a research project. First, figure out what you need to prove. Or the assumptions that could make the product fail.
- Do people want it?
- Can people use it?
- Can you build and operate it?
- Will people pay if it has a premium price?
- Can you actually deliver the value before the market shifts to something else or your value stops being an innovation?
Did you hear about Burbn? This was an app idea that included location check-ins and several social features. Tests showed that, most of all, people loved sharing photos. The team pivoted from the original concept to give consumers what they wanted. You know the app. Just under a different name.
It’s Instagram.
We can only speculate about what would have happened if Burbn’s owners had been stubborn and done what they wanted, hoping that the market would adapt.
Validate Cheaper
Many companies skip validating their product assumptions because the research may get costly. But it doesn’t have to.
It all depends on what you are trying to confirm. Here are a few insights:
Is there interest in the proposed solution? Do a concept demo, create a landing page, or launch a waitlist.
Is the problem real in a specific market? Conduct customer interviews and direct observations. Research competitors’ lowest-rated reviews.
Will people pay? Launch a paid pilot, preorder, or a real purchase test.
Can users complete the main workflow? Create a clickable prototype and perform observed user-task testing.
Can the team build the critical functionality? Run a technical spike or proof of concept.
And start with the riskiest component before committing to the full system.
Before Dropbox became a widely used file-syncing service, founder Drew Houston used a short demonstration video to show how the proposed product would work. The video generated massive interest, and the beta waiting list grew from roughly 5,000 people to 75,000 overnight.
You will still need to invest in this research, but the amount of the investment will represent a fraction of the blind development cost. If it’s unsuccessful, all you lose is that, not the cost of the full product and marketing. And storage and supply chain if your product is tangible.
The principle is simple: Don’t build more than you need to learn what you need to know.
Build This, Not That.
Before you commit to full-blown product development, build a prototype. A prototype lets you test an idea without paying the full cost of engineering.
It could be as simple as a paper sketch, a wireframe, a clickable Figma flow, a simulated interface, or a small technical proof of concept. Let your prototype represent your MVP. The medium matters depending on the service your product provides.
When Daily Harvest changed its online shopping model to allow one-time purchases and subscriptions, usability testing showed that the proposed cart was confusing. Customers struggled to understand how subscriptions worked, what discount applied, and where to select their options. The team used that feedback to revise the cart before the rebuild went live.
The lesson here is to learn what needs to be adjusted before the launch rather than face a bunch of frustrated customers and lost conversions.
Decision, Not Desire
Validation is great, but only if you have a point of reference. You need to evaluate your feedback against metrics that are tied to profits. You may get qualitatively positive results during validation, but if you translate them into quantitative metrics, they may not be good enough to project profits.
Before each experiment, define the hypothesis, the evidence that would support or challenge it, and the decision you’ll make based on the result.
You don’t always need numerical thresholds. Early discovery is often qualitative. But you should know what you’re trying to learn and what decisions the findings could change.
During Barack Obama’s 2008 campaign, the digital team tested different versions of its website signup page. One experiment compared button labels, including “Sign Up” and “Learn More.” The latter generated 18.6% more signups per visitor than the original wording. The team also tested images and video. Contrary to expectations, the video options performed worse than the still images in that test.
This is useful because the team wasn’t just gathering opinions. It compared alternatives against a specific outcome: signup conversion.
The broader lesson is that a test should be designed around a decision, not around the desire to collect positive feedback. Decide what you will measure before the results arrive, and be prepared for the evidence to contradict your preferred solution.
Validation Loop
Unlike many other loops, it’s a good thing. Validation is and should be an ongoing feedback loop.
Hypothesize → execute → validate → measure → and… repeat.
Move into production when most assumptions have been clarified and development cost justifies the projected revenue and profits. We can’t, of course, eliminate every single unknown. But we can gather enough evidence to make an informed decision about whether the product is worth building.
During development, continue validating new assumptions. After launch, validate the existing product and test new functional hypotheses. Now, you will have actual users, support requests, retention, conversion, and other relevant metrics to test.
In Airbnb’s early days, the founders discovered that poor listing photographs left prospective guests unsure about what they were booking. Rather than waiting for a large-scale technical solution, they took better photographs themselves. Improved photos helped increase bookings two to three times, and revenue in that segment doubled in less than a month.
The goal is not endless research. It is to make sure that verified assumptions are still in the clear and the new assumptions are getting validated.
You Can Do It. AI Can Help.
AI can be very helpful in product validation. It can organize interview notes, identify themes for human review, generate alternative user flows, draft usability-testing scenarios, create test data, explore edge cases, and assist with test automation.
But be careful not to assume that even the “Deep Research” feature of a paid AI model can replace real validation research. AI-generated personas are not substitutes for real customers, synthetic feedback is not evidence of market demand, and automated tests cannot tell you by themselves whether you’re solving the right problem.
AI can help you explore hypothetical scenarios and identify assumptions worth testing. But those outputs are still hypotheses, just generated with AI. You still need to validate them with real users and real-world evidence.
AI can make it easier to figure out what to focus on, but it cannot do the validation for you. Don’t trust anyone who says otherwise.
The Conclusion
The cost of validation is not just the price of interviews, prototypes, or testing tools. It is an investment in avoiding the much larger cost of building features that need to be redesigned, rewriting specifications, reworking code, or launching a product that fails to gain traction.
You don’t need a large research budget to validate a concept. You need to identify your most consequential assumptions, find inexpensive ways to test them, and let the evidence influence your decisions.
Just remember that market validation tells you whether someone wants the product. Technical validation tells you whether you can build it. Economic validation tells you whether you can afford to deliver it.
You need all three.
