Product Research: Part 1
July 15, 2012, I enjoy a lazy morning after our Bastille Day party. Suddenly, the French revolutionaries karma strikes one more time. Not only did I successfully passed the first round of interviews
July 15, 2012, I enjoy a lazy morning after our Bastille Day party. Suddenly, the French revolutionaries karma strikes one more time and a recruiter's email lands in my inbox. Not only did I successfully passed the first round of interviews at Salesforce but I am selected for an all-day visit at their HQ. Time to speed date with Engineers, PMs, UX, and Execs. And like all good movies, the end promises to be epic: an hour long product presentation on Community Moderation, with everyone I will meet that day.
I waited five years to get a shot at joining Salesforce. My future in my favorite Saas company will be decided in the next 48 hours. What an irony! I know the presentation is key to land my dream job. Unfortunately, I know NADA about Community Moderation.
Luckily, with five years of entrepreneurship under my belt, I am accustomed to start at ground zero. First, I set a goal to get a solid draft by the end of the day. By lunchtime, my research is complete. I understand the core concepts around Community moderation and the main challenges faced by large brands. I identified a few pure-play vendors in the moderation space and understand their value-proposition and key features.
I spend the afternoon bringing relevancy into my research. I study Salesforce offering. Unfortunately, I am interviewing for a product that does not exist yet. I make the assumption that they have no moderation capabilities. I know moderation is top of mind since they picked that assignment. Finally, with PMs, Engineers, UX designers, and Execs in the audience, I have to pick a target audience. I play it safe. After a moderation 101 for the audience, I will qualify the business problem (relevance, scope), move to the solution, a.k.a. my product vision and strategy, and close with a three releases roadmap.
By the end of day one I have a draft version. I sleep on it. Day two, early morning. I refine my slides and talking points. I spend the rest of my day rehearsing for my presentation. On day three, I survive five hours of interviews and get to the presentation. Like a good striker, I score and the rest is history.
In retrospect, I owe a lot of my success that day to a well executed research and nuggets of critical thinking. The research provided me all the ammunition I needed to build a cohesive vision and strategy. It had enough depth to keep me out of trouble when asked tough questions. It was relevant.
To Research Or Not - That Is The Question
Product research plays a critical role in shaping strong product vision and strategy and shipping successful products. It is necessary step, maybe the most under-rated. In this article, we will explore different techniques to execute a research.
So how do you decide you need a research project? Well, it is really up to you. In my books, it is a must for strategic projects (2+ years) and it is highly advised for critical product capabilities (new or updated) or when there is a lot at stake (new pricing and packaging, product pivot, high customer impact, etc). I try to avoid research on small projects (cursory research is ok, a couple of hours top).
You identified a research project: what's next?
First, be specific about your (or your team’s) time commitment and set clear deadlines. Stick to it, no excuses. Similarly, it can be tempting to postpone the writing of your research report and recommendations. Just do it. Defining a clear scope and setting goals will let you avoid those pitfalls. Identify the important questions you need to answer. Write them down. I learn that lesson over the years, sometimes the hard way: projects are made successful early on. There’s little you can do to fix them in the end.
Just Do It
You defined research goals, timelines, and priorities. How do you go on executing your research?
Many people would answer "I block time, Google X, Y, Z topics, read, and take notes on my favorite notebook" and that's fine. What many PMs may not realize is that they are barely scratching the surface. It is critical you research with breadth and depth.
For instance, when entering new markets or playing catch up with the competition, I like to browse the key competitors website, understand their positioning, go to market, pricing and packaging, technical architecture, and read analysts reports. It's rare to see PMs who check all those boxes; I have seen PMs using pre-canned analysis and reports of the competition. Why spend hours reading when you can get the info in five minutes from an untrustworthy source, right?
A common mistake consists of taking everything you read at face value; each software company is world leader at something, they all have happy customers, plenty of success stories, work across all industries, all segments -SMB, mid-market, enterprise. Don't be like Candide in Voltaire's book. Being a critical thinker is hard. Getting different point of views from your peers and teams can help.
I personally spend limited time on a competitor's website and much more reading their user, developer, and admin documentation. I like to find examples of implementations. I also like to review the last three years of innovation and key announcements they made. I find forums very insightful. If you can access a developer forum, identify discussion threads relevant to your topic and read them. This is usually unfiltered real life feedback. Reddit is another good source.
If you have a hard time finding what stack they built their IP on, just look on LinkedIn and browse a few developer profiles. Job postings can also be very informative. Do not limit yourself reading PM or Engineering job postings. Check every function and find clues: Are they hiring a PM to expand in a new industry? Do they sell direct or indirect? Do they have their own implementation service or leverage mostly SIs? Are they expanding and recruiting sales in new regions?
All You Need Is A Few Keywords
Usually a few well chosen search keywords will do wonders. Many people stop at the first search results page and often only open sites for companies they know. I prefer to check everything and paginate to the next set of results until I hit a page where nothing is relevant anymore. I don’t read at this stage. I scan sites, search results and copy relevant links in my research doc.
I love to append "blog" to my search keywords. People writing blogs on a product, feature, technology are usually experts on the topic. Another bonus is that they are independent (usually). I am attracted to developer or implementer blogs with practical expertise on a product or solution. You learn the most from people who have to implement a software.
At the risk of stating the obvious, use Google images to get screenshots of the product. Even better, go on YouTube and use the same keywords to find demos or tutorial on the research topic. Whether it is bloggers or YouTubers, there is tons of prime knowledge accessible at your fingertips.
Bringing it to life
Sometimes, you need to expand your competitor's research beyond the obvious candidates. At Salesforce, we have a full solution. Obvious competitors include corporations like Microsoft, Oracle, Service Now. The software vendor market is much bigger and fragmented. I try to identify niche players, pure-play vendors, or software products that address a subset of the product space I am researching. It typically gives me a much better understanding of what state of the art means in term of product and what are the latest innovation trends.
A few years ago, my team researched low-code performance and scale test product suites. It was your typical build vs partner vs buy question. We looked at direct competitors like Adobe, Microsoft, SAP, and ServiceNow offering. Our short-list of pure-play vendors included JMeter and Load Runner. We put all other smal players in a third bucket.
Those two lenses gave us a broader perspective: How does best in class look like? How do we fare against direct competitors and pure-play vendors?
We covered a lot of ground to get you started with research projects. In the next post, we will explore several research approaches that can deliver great results when used on the right project.