3 Tips for Testing Product Assumptions in a Regulated Environment

Overcoming barriers to ‘getting out of the building’

“Get out of the building” is advice that every founder and product manager has heard at some point. It’s a simple philosophy, popularized by Steve Blank’s Customer Development process: teams should “get out of the building” and talk to customers so they can identify their customers’ needs and validate whether they’re building the right products to address those unmet needs.

3 tips for “getting out of the building” when operating in regulated industries

Play In Safe Mode 

With regulations constantly changing, it might be difficult to stay on top of what you can and can’t do to test your new product or service with real people. By playing in “safe mode” you can ensure you’re never entering regulatory grey areas. To ensure you’re playing in “safe mode:” 

• Don’t capture any Personally Identifiable Information (PII) or Protected Health Information (PHI). 
• By leveraging research panels, you can still target users who match your target profile without needing to access or store any PII or PHI, such as name, address, email, phone number, or medical history.  
• Don’t blur the line between research and marketing.
• Both the healthcare and financial services industries have regulations to protect customers from marketing for products that have not been cleared and approved for sale. Exercise caution when conducting research to ensure that what you’re doing doesn’t come across as marketing. A safe way to do this is to:
• Explicitly tell consumers coming into an online study that what you’re showing them is either a hypothetical scenario, a simulation, or a mock-up and that you are requesting their feedback. Generally, research panels are structured in such a way that respondents are remunerated to provide feedback, so there’s a in-built mechanism to ensure a study is not perceived as marketing.
• Avoid using any commercial branding in tests, like potential ad creatives or website designs. Keep testing focused on the concept and not the brand.
• Do have your legal team on standby.
• Proactively engage with your legal team to define clear thresholds of acceptable risk. All organizations are different and, as such, have different risk appetites. For example, a start-up working in a single jurisdiction for a simple healthcare device may have a higher risk appetite than a global pharmaceutical company launching a drug dispensing device in multiple jurisdictions.   
• Make sure your legal team understands what online research is. Invest time in learning the difference between a “research participant” that is sourced indirectly through a third-party panel provider or sample marketplace, and a “customer” or “patient” that your business has a direct relationship with.

Focus on Assumptions 

New ideas are essentially clusters of assumptions. No idea is inherently good or bad. Good ideas are those where the underlying assumptions (e.g. assumptions about users, competitors, positioning, features, design, and usability) have been validated before going to market and bad ideas are those where the underlying assumptions have not been validated. In the latter scenario, most of the validation occurs in-market, late in the process, where the cost of making a mistake is significantly amplified. 

With that being said, the vast majority of feedback about a new product or idea can be gathered without actually ever talking about the product itself. In fact, by stress-testing the underlying assumptions, you’re giving your product a greater chance of success in the market. 

For example, imagine you’re launching a new type of heart-rate monitor for athletes that’s ultra thin, see-through, and sticks directly onto the skin. There are a number of assumptions you’d want to stress test, such as:  

• Athletes are more interested in tracking their heart rate compared to the general public.
• Athletes feel that existing heart-rate monitors are too bulky. 
• Athletes would be willing to pay a 50% premium for a heart-rate monitor that is smaller than the smallest device available today.
• Athletes would be willing to pay a 50% premium for a heart-rate monitor that is more accurate than the most accurate heart-rate monitor available today. 
• Amateur athletes are just as interested in tracking heart rate as professional athletes. 

Each of the above assumptions (which, if proven to be false, could ultimately impact market demand for the product) can be tested without ever fully revealing the end product being developed. This may be beneficial when operating in a regulatory environment and is also an approach that can be used when developing highly confidential products. 

Stay in the Public Domain 

Once a product has been approved for sale and released to the public, getting feedback on it becomes easier. If you’re releasing a new type of product (one that has yet to be cleared and approved for sale), look for equivalent categories of products that are already in the market. Testing against similar products can provide insight into how consumers perceive existing solutions. As you build your product and prepare it for launch, this type of competitive insight can help differentiate your product in the market. 

Summing it all up

Product development in a regulated environment can still be rapid, iterative, and embed continuous user feedback. By erring on the side of caution, playing in safe mode, focusing on assumptions, and gathering intelligence on competitive or substitutive solutions, teams operating in regulated industries can “get out of the building” and create products that solve real, unmet needs.