TEST & LEARNS REVISITED
Unlocking the Value of Learning Beyond Launches
Michelle Hayward and Jake Taylor, Authors
On the hunt for growth and eager to tap all the potential insights that help point toward tomorrow’s best product launches (and most profitable business models), many leaders now turn to Test & Learns. But as design thinkers and innovation specialists, we’ve witnessed a widespread problem: a failure to commit to big ideas and dedicate the necessary resources to develop learnings into transformational growth.
Missed Opportunities in Today’s Testing & Learning
Our practice focuses on diagnosing problems and driving new actions. So when we started to see the same Test & Learn tensions arising across business and brand ecosystems, we took notice. From our viewpoint, many misapplications of the Test & Learn seemed to spring from risk aversion. Some risk avoidance is always healthy. But a Test & Learn can’t drive growth without meaningful investment in both learning and acting on insights.
We reached out to Anthony Geonnotti, Head of Future of Self Care R&D at Johnson & Johnson Consumer Health (and a Test & Learn veteran), for his perspective. He sees efforts to innovate blocked by longstanding financial practices. In an age of rapid innovation, they can’t keep pace.
“There should be a commitment to funding and action coming out of a Test & Learn,” Geonnotti explained. “Too often, teams are funded just enough to perform one experiment and then have to go back and fight to get funding for the next step. At the very least, agile project development practices should dictate that the end of each experiment includes an opportunity to reaffirm priorities and move to the next sprint.”
Learning to view (and fund) testing as an ongoing function vs. a series of self-limiting projects requires revisiting the approach and how it works best. Given our experience with diverse business ecosystems and success drivers, we identified these four core truths about when to test and why-including essential mindsets for acting against opportunity in today’s contexts.
Four Insights for Increasing Test & Learn Impacts
1) Growth Requires Commitment to Maximizing Success, Not Minimizing Investment
Impossible Foods founder Pat Brown bit off a big challenge: taking the meatless burger mainstream. But he invested in the value of that extraordinary opportunity-spending significant money to pursue learnings, overcome potentially show-stopping barriers, and develop better and better solutions over time. His company continues to improve on their success with the launch of Impossible Burger 2.0.
“A Test & Learn approach doesn’t necessarily save budget,” Geonnotti stressed. “It allows you to try more ideas, get closer to consumer needs, and more quickly prioritize better activations.”
To build on this insight, spare no expense to ensure the quality/consistency of what you test and commit to following through. It sounds simple, but invest like you believe in your own idea.
2) Well-designed Test & Learns Inform Smart Pivots, Not Go/No-go Decisions
“Pivot” has been in every leader’s vocabulary since Eric Reis, author of the New York Times best-seller The Lean Startup, coined it to describe changing direction vs. halting progress. Many leaders expect Test & Learns to tell them whether to make a move, while the approach can only really tell them where to go once already in motion.
By the time a new product is ready to go to market, most companies have already invested heavily to determine that consumers will want and need it. Instead of limiting their testing to labels or product attributes, far more brands could test levers and drivers for goals like velocity-learning how to accelerate their penetration vs. second-guessing their idea.
To build on this insight, start each Test & Learn planning process by defining a vision for success beyond the product.
3) Pivots Require A/B Testing, Fueling Innovation Via Data
A/B tests-comparing the performance of two versions of an idea-point toward more impactful strategies. In the Netflix docu-drama “The Social Dilemma,” tech creators explain how they used A/B tests to optimize behavioral influence (sometimes too effectively), exploring the impact of different reminders, prompts, and nudges on user engagement. The approach helped them shape the new experiences that now define a pervasive online industry.
Unfortunately, many companies underinvest by only testing a single advertising message or product placement, which can only succeed or fail. But what if the strongest path forward is a yet-to-be-realized option C: a new possibility that combines the best of A and B or builds on what each failed to deliver? What if simply changing a sell-in line could change everything?
To leverage this insight, avoid single-option tests. They can’t yield the insights to design new solutions that may be similar or related, but infinitely better. Anticipate and plan for contingencies, even if it means revising your entire marketing mix mid-experiment.
As Geonnotti urges, “Even if your primary hypothesis fails, leave smarter.”
4) When Planning, Taking a Holistic View of All Potential Success Drivers is Critical
Expansive A/B testing opens the door to countless new insights. Consumer data from connected packs can inform new messaging, identify target audiences, and illuminate new paths to purchase. But to make the most of insight-validated opportunities, leaders must keep their businesses’ bigger picture in view. They often don’t.
“The more cross-functional a Test & Learn is, the richer its potential to make meaningful advancements within the organization,” Geonnotti counseled. Again, total focus on the product limits what can be learned. Why exclude supply chains and distribution?
By definition, innovation requires changing the rules. That often means discovering and maximizing new success levers, never assuming factors that ensured growth in the past will do so today, much less tomorrow. So to build on this insight, think inclusively when it comes to defining and incorporating factors that might contribute to success-and reveal future paths.
This fourth, final insight may be the most important. Companies gain the most from Test & Learns when they empower learnings to shape growth strategies they haven’t yet imagined, not just answer one-off questions. They initiate a virtuous cycle: learning (and re-learning) the factors that can drive creativity, innovation, and impact across their business.
In the end, the Test & Learn is a highly effective and productive approach to innovation development. It always was. But it can only tell leaders how to get where they’re already committed to going.
“It’s so important to know your greater goal prior to building any Test & Learn,” Geonnotti said in closing. “Otherwise, you are rewarding a process, not outcomes.”
About IN THE ROOM WHERE IT HAPPENS IN THE ROOM WHERE IT HAPPENS is an article series from Bluedog Design’s most creatively strategic minds, launched to engage the many challenges and emerging opportunities facing the business leaders of today and tomorrow.
About Bluedog Design Bluedog partners with clients to apply principles of Design Thinking that strengthen their businesses-developing inventive growth strategies across brand identity and engagement, marketing, product lifecycles (prototyping through in-store activation), portfolio management, organizational structure and business operations.
Originally published at https://www.linkedin.com.
