Product Research - Part 2

Product research comes in many flavors: vision and strategy, pricing and packaging, field enablement, new features (early discovery, concept validation, feature validation), white space offering, etc

We previously explored product research with a focus on competitive products. This is a common product research when you enter a new space or need to reposition your offering.

Product research comes in many flavors: vision and strategy, pricing and packaging, field enablement, new features (early discovery, concept validation, feature validation), white space offering, etc.

In this post, we explore research techniques that work on specific project types and that can complement a competitive research.

Passive Field Research

Software companies usually have processes to collect customer feedback. At Salesforce, we use Known Issue, Q&A forums, Collaboration Groups,  IdeaExchange among other channels to engage with our customers. Internally, we have the VOC (Voice Of the Customer) process to collect feedback pre- and post-sales.

Known Issues

Knows Issues is a neat concept that drives trust. When support identifies a product bug or issue, we publish a public Known Issue article. It becomes instantly visible in search engines like Google or Bing for easy discovery. Customers who are impacted can click on the "This Issue Affects Me" button and be added to the list of "Reported by". When the issue is fixed, customers who reported being affected get notified.

You may wonder what this has to do with research. In the software world, a known issue may take five minutes to a month or several releases to fix; sometimes, it may require a full re-architecture making it a non candidate. As PM, we can use more data points to prioritize specific issues. The Known Issue reporting mechanism plays that role and acts as a feedback loop. Trust me, if 30% of your customer installed base is impacted, you look at your Known Issue... differently!

Q&A and Idea Forums

On the same token, Q&A forums contains tons of information from your users. Such forums are technically oriented and questions tend to lean on very specific topics. It is great to understand where your implementers struggle!

Next, we have customer ideas. The Salesforce IdeaExchange site is public, searchable on Google, and contains all ideas submitted by customers. Those ideas can then be voted on by customers. Over time, the most critical ideas stand out. If your feature has an idea with 10K+ points (i.e. over 1000 votes), it is a clear signal that you have either an issue or an opportunity to improve your product.

I do not like binary thinking. Besides the relative ranking of ideas by points, I like to read the comments. There is so much to learn by reading comments from customers or implementers who are struggling with your feature. This approach has the merit of helping you understand how customers are using your product, and it can be eye opening.

Customers in idea forums are brutally honest. And they should be. The people who struggle the most with your feature tend to post pointed and valuable feedback. Usually, they tried everything to make things work, got frustrated, researched workarounds, and reached the bitter conclusion that the product does not work the way it needs to for their use case.

Reading through comments is humbling. But remember, feedback is a gift. Put yourself in your customers’ shoes and learn from it.

For software companies with large numbers of ideas or requests, I recommend doing keyword research and not limit yourself to your area of expertise. Idea classification through categories or tags is a great concept. In practice, many ideas are submitted in the wrong category, without topics, as duplicates, or as close cousins of another idea. Broadening your research will help capture related ideas.

Voice of The Customer (VOC)

VOC (Voice Of the Customer) is Salesforce internal process to capture customer feedback. Every software company has one. For us, feedback comes from pre-sales and post-sales. Both are invaluable. For example, a sales rep may log a VOC on a deal lost to competition and explain your product was lacking X, Y, or Z features; another rep may report a deal lost to competition due to pricing. The post-sales VOCs tend to be more feature centric and a reflection of implementation challenges. They are like ideas.

Field Research

So far we looked at passive research where one collects data points. A more direct approach consists of surveying SMEs on your products. I like to think of them as tribes and distinguish the pre-sales, post-sales, and customer tribes. Each tribe comes with its own perspective. Let's dig in.

Pre-sales tribe

In this context, pre-sales tribes represent your direct sales team (sales rep, sales engineer). They are great for research on market fit, pricing and packaging, competitive differentiators, and growth opportunities. Talking to account teams who have lost deals against your competitor can be an eye opener. Engaging with them on competitive deals is also invaluable.

You may conduct interviews with sales reps to assess new pricing and packaging models or gain a better understanding of the competition in specific segments (SMB, mid-market, enterprise), industries (e.g. High-Tech, Manufacturing, HLS) or geographies (AMER, EMEA, APAC).

Sales Engineers are also well versed in responding to RFPs and building demos that help close the deal. They will give you detailed insights on what helps sell, what customers are demanding and the FUD the competition is throwing at you.

Not all companies sell direct. Actually in industries like High-Tech and Manufacturing, 80% of the sales are done through channel partners (resellers, VAR, distributors). If this is your case, you will have to adapt your pre-sales research to this tribe. It is much harder to connect with external sales team. Make a priority to build long-term relationships with your channel eco-system and develop feedback mechanisms on a regular basis (hint: get help from your internal channel leaders)

Post-sales tribe

Post-sales tribes are in charge of the implementation and support of your product. Some companies will have a strong professional services arm and do most of the implementations. Large enterprise software companies nearly always work with SIs.

You should also consider success/account managers who manage the customer relationship and knowledge experts in your support agents (tiers-3 preferably).

When you engage with your post-sales tribe, identify early if their are functional or technical experts. Functional experts are not technical. Technical experts know the ins-and-outs of your product architecture and offering and translate functional requirements into solutions (dual expertise).

We mentioned in a previous post the relevance of specialist blogs for your research. Similarly, your solution/technical experts and SIs accumulate a treasure trove of insights over the years. When interviewing this tribe, pick experienced ones with diverse background (by market, geography, industry). Focus your research on their area of expertise. Listen carefully when they describe product gaps and customer challenges.

Customer tribe

Every PM I know loves to engage with the customer tribe. You learn a lot from your interactions with customers. They write the check for your product and represent the ultimate decision maker in a deal. Make sure to differentiate the buyer persona (CMO, CTI, etc) from the business or project lead who is usually the SME.

While I engage regularly with customers, I refrain from doing my research only through the customer channel. There are two reasons. First, a customer will present his view of the world at a specific point of time. He will tell you what matters to him and his team, and often surface tactical blockers for his project. From experience, I find that it rarely translates into broad innovation -at least in B2B software. This is not to say you should dismiss this feedback. I have had several times discussions focused on unlocking new cases that presented a great business opportunity.

Furthermore, the customer is usually not a true SME (exceptions apply). He usually comes with a single implementation of your product that may leverage 20% to 50% of your product offering. The feedback is bias and centered on their own use case and limited exposure of your product. Again, nothing wrong, just be aware of those factors when reaching out and apply critical thinking.

Trust In The Numbers

Facts and numbers are like a second religion to me, an integral part of my framework to make decisions. It also applies to research. Let’s say you have 10000 customers, the voice of a single customer represents 0.01% of your installed base. You don’t need to be a math major to realize this is statistically irrelevant.

Now, flip that around by doing research with a top SI partner. You can easily engage with SMEs that capture 100+ implementations. Input built over years of experience is usually invaluable.

Another way to bring relevance is to look at it from the lens of feature adoption. Let’s say you want to understand the limitations of your notification system, a specific feature set in your product offering. Using feature adoption dashboards, you can can identify the top adopters of this feature and select them in your target audience.

Net-net: there is many ways to skin this cat. Think in term of relevance and the effort needed to execute your field research. Identify the right SMEs for your research. It will serve you well! 

Finding Your Tribe

In the context of B2B software, a partner can be an SI (implementer), and ISV building app on top of your platform or an OEM provider.

The post-sales partner tribe is my favorite because they excel at one thing: implementing your software. Every year, they go through dozens or hundred of implementations, often across geographies and industries. They know what works and what does not. They are also a trusted customer advisor and know competing solutions; some partners may even be certified implementors across many vendors.

If your company has its own implementation practice, make sure to befriend them. Include both the functional and technical experts. In addition of being SMEs on your product, they are easy to access.

For our research, I ask my teams to create a shortlist of partner SMEs; it is usually a mix of global SIs (e.g. Deloitte, IBM, Accenture), top-tiers partners, and a few smaller partners who specialize on our product.

Depending on the research project, we reach out to the business or technical architect. You could consider the practice manager for revenue growth discussions where the GTM involves the partner. This crowd is always happy to engage with PMs. They usually have a solution for each of the challenge you will through at them. Be a good listener.

The MVP tribe is worth your time if it exists in your organization. At Salesforce, MVPs can be customers or partners. They are usually very engaged in Q&A, forums, and communities. It helps to know their specialization in advance of your research (business vs technical, admin vs developer). 

This concludes part two. In my third and last article, we will discuss a research framework I use to qualify and prioritize research projects. We will also talk about the multiplier effect, the art of doing more with less.