Validating Design Decisions (Growth Design Monthly)

September 2019Dear Readers,Have you ever felt like you weren’t able to ship an experience you were proud of? Or maybe you’ve had a design cut back until it barely resembles the intent?Part of this comes from projected risk. The bigger the design proposal, the bigger the risk. The bigger the risk, the more important to de-risk. Otherwise, you might over-invest in the wrong direction. (Big bang redesigns, anyone?)Luckily, it’s 2019 and you don’t have to just release software and hope for the best. We have many tools at our disposal to validate that we’re making sound design investments. When designers are well-versed in a range of validation methods, they are better able to shape the experience⁠—from a humble first test to a holistic vision.Validation, All I Ever Wanted- Lex

What is Validation?Validation is a shorthand for "validating your assumptions." I find people conflate AB testing and experimentation so here, I am using the term validation to emphasize that there are more ways to derisk a decision. The goal is to gather evidence that you’re on the right track.There is an infinite number of ways to validate and derisk. AB testing is one way. Below, I share three more methods.

Validation Method: Feature FakesIf you have an existing product, you can start building a feature without building the entire thing. You can use feature fakes to see if customers will even try to access what you’re designing. A feature fake (also called a fake door) indicates that a feature exists but doesn’t actually include the full functionality. If customers won’t tap or click on something, it may not be worth building the actual feature.Good for: Testing discoverability of a feature Gauging interest in a featureHow to do it: Define your feature thoughtfully Design the indicator that allows people to access your feature (for example, the button or toggle or message) Build the indicator When people hit the indicator, show them a message that explains why the feature isn’t functioning. You could let them know it’s not available yet and save emails for notification later or you could show an error message and give them a way out.What it validates: That your feature is discoverable That people are interested in trying itResources: “Fast, Frugal Learning with a Feature Fake” by Joshua Kerievsky “Feature Fake” by Kevin Ryan UX for Lean Startups book by Laura Klein

Validation Method: Announcement TestsTeasing your product concept is a beloved Lean Startup tactic, famously working for Dropbox, Tesla and any project on Kickstarter. You can put your value proposition out there using a landing page, a video, or an email campaign. In exchange, you ask for personal contact info or better yet—pre-orders.Good for: Validating marketing channels Testing messaging of value propositions Building an interest or pre-order listHow to do it: Create a brief marketing plan around how and where you plan to reach your customers Design your messaging and capture mechanism (a landing page, an email campaign) Define what kind of interest you want to see (sign-ups, emails, pre-orders, etc) Launch it and start marketingWhat it validates: Your marketing channels Demonstrated interest in your value proposition  Your messagingResources: “How Dropbox Started As A Minimal Viable Product” by Eric Ries “How to successfully validate your idea with a Landing Page MVP” by Buffer CEO Joel Gascoigne “From Minimum Viable Product to Landing Pages” by Ash Maurya

Validation Method: Blind Product TestsThe classic Pepsi vs Coke test. Leverage your competitors or other existing apps to better understand how to differentiate. You can use their existing products or mock up versions of their experiences with no branding. How does your product stack up against a competitor? Or, how do your competitors rate against each other? Good for: Finding out how different your product or service really is Learning customer attitudes about competitor products Learning how usable competitor products areHow to do it: Identify competitors to include Determine what you’re testing for (is it attitude or task-based?)  Create a discussion guide along with any scales you’ll use to score interviews Create a prototype or visual reference (Optionally) Remove competitor branding so it’s anonymous Recruit target customers Run them through the prototypes or visuals Score each interview and analyze sentiments Identify opportunities for improvement in your product/serviceWhat it validates: Attitudes about competitor products/services when brand is not a factor Attitudes about your product as compared to competitors How usable your competitors’ products areResources: “Blind And Branded Product Testing Can Tell Completely Different Stories.” on the ACCE blog “Measuring Simple Preferences: An Approach to Blind, Forced-Choice Product Testing” academic paper by Bruce S. Buchanan and Donald G. Morrison

As you define how you’ll test a design idea, start a validation plan where you capture all the key aspects of your test.Every validation plan should include a testable prediction, how you will test your hypothesis (aka your test method), and what you’ll measure to determine a winner. It’s also helpful to include images of the customer experience or design changes along with detailed instrumentation of how success is measured in the test. After the test is complete, you can add the results, what your team learned and how you’ll move forward.No need to obsess over this documentation but I personally find the process of writing these plans helpful in driving important team discussions and in reducing errors in the tests.  

Additional Resources

My former colleague, David Bland, is writing a book on this topic called Testing Business Ideas. You can subscribe for updates on the book on the Strategyzer website.

The Experiment Canvas was developed by Ash Maurya as a one sheet to help teams formulate tests.

Brant Cooper offered a few ways large companies can test ideas in his post “Enterprise Lean Startup Experiment Examples.” 

Lex is an independent Growth Designer. She's the primary voice behind GrowthDesigners.co. Interests include product analytics, validating design decisions and watching dance videos on Tik Tok.

EventsAB Testing for Growth DesignNovember 13-14 (San Francisco)*They are looking for speakers. Reply to this email if you're interested.The Growth Mentor SummitNovember 18-22 (Online) 

JobsProduct Designer, GrowthGusto (San Francisco)Lead Product Designer, GrowthPinterest (San Francisco)Director of Product Design, ExpansionSlack (San Francisco) 

This is the Growth Design Newsletter. We produce monthly issues on topics relevant to growth design. Want to write or edit an issue? Hit reply!In the October issue

Leadership Buy-In for Growth