As more and more B2B companies become product-led, how quickly and precisely a company identifies new customer needs and ships product ultimately determines its success. To develop product ideas faster and get shorter time to MVP (minimum viable product), companies have to rapidly prototype, test out several viable paths in parallel with potential users, and then focus engineering efforts on the winning direction (or at least, the direction they think is most likely to win).
For almost every phase of the product development process — ideation, prototyping, designing, and measuring — a suite of tools has emerged and multiple billion-dollar companies created. Yet user research has remained a fragmented market.
We’ve now reached a point where the market demands user research tools that are more real-time, contextual, collaborative, and easily digestible. Here, we look at the current state of the user research market; the drivers pushing this category to the forefront of product development; and some critical capabilities for next-generation user research tools.
The current state: what’s missing?
User research goes hand in hand with product design. Since there’s never a “finished” product, user feedback is indispensable for both ongoing product improvements and new product launches. This has been a secretively fast growing field fueled by the product-led trend and talent growth.
Broadly, there are two types of user research: 1) generative, which looks for opportunities and problems; and 2) evaluative, which tests a particular solution. While most of the focus has been on evaluative feedback, generative feedback is just as critical — in fact, 93% of designers conduct user research before designing anything.
Moreover, in smaller, often earlier stage, companies, there are rarely dedicated user researchers, so it often falls to product managers or designers, who spend a significant amount of time — over 30% of their day-to-day on user research. Not surprisingly, user researcher, UX designer, and product manager are some of the fastest growing roles in tech, growing ~20% YoY.
Despite the importance and growth of user research, 80% of product managers still feel they don’t spend enough time talking to their customers. And most companies only hire dedicated user research teams and perform formal user research late in their product lifecycle, after they reach a certain scale. What caused this current state?
The existing market is fragmented, tools are manual and disjointed. Most product teams use general-purpose meeting tools such as Zoom and Google Meet to conduct/record user interviews; SurveyMonkey, Qualtrics, and Typeform to run surveys; and Google Docs and Notion to capture the notes; and spreadsheets to analyze the results. Earlier purpose-built tools such as UserTesting and UserZoom tend to be point solutions, bought and used by user researchers for specific tasks in the overall workflow, such as user recruiting or UI usability tests.
First-generation research tools often refitted the processes and features built for physical products to digital ones. For physical products, finding the right demographic to test out a new product idea has always been tricky, so there was a lot of value to owning the audience. As a result, most vendors on the market today are heavy on things like panel recruiting (recruiting people for these studies), but light on workflow improvements.
Long research cycles and hard to prove ROI. The fragmented landscape of user research tools leads to long, expensive user research cycles. Research tools are disconnected from various product management, design, or work/note suites (such as Figma, Jira, Notion) so user feedback often lives in a silo. UX researchers or product managers are carrying the burden of transferring learnings and performing manual analyses — often with spreadsheets, shared docs, or even sticky notes — on hundreds of survey results, hours of video recording, and pages of detailed interview notes to find useful insights.
It’s a process that is not only cumbersome, but also prone to error and bias. It often takes weeks to define the right approach, budget, and target groups. Depending on the actual methodology — survey, focus group, or user interview — feedback collection can take anywhere from days to months. After that, synthesizing the data will take another few weeks (at least) before the insights can be shared with a broader team. As a result, user research project budgets are often five to six figures and only spent on the most critical projects for 1 – 2 times a year. The frequency is far below what product teams need, while the cost is prohibitive to improve it. Yet, with all that upfront investment, it’s still hard to correlate the impact of research to the actual shipped product.
No current system of record. Because each user research project is treated as a single-purpose event, there’s no central record, tacit knowledge sharing, or a standardized workflow to support a repeatable process. The raw data and insights are often scattered around several tools, accessible to only a small group of people.
The New Wave of User Research Tools
In terms of both market demand and technological maturity, the user research industry is ready for a new wave of innovation. Following our discussions with both incumbents and startups on purpose-built user research products, we categorized the players into three main buckets based on their primary user interface mode: video, survey, or panel, plus a few incumbent players who span all three. These categories, though, are just initial wedges to enter the market. Whatever the starting point, the next generation user research tool will likely differentiate by the following product components.
1. Native, continuous, rapid research
The benefit of testing a digital product is that you can access — and even segment — your users right when and where they are experiencing the product. Instead of spending weeks to find users who are “marketers working at a 5000-employee company already using feature x” in an external research platform, new tools are providing the ability to access these users and ask simple questions directly in the product, instrumented with analytics.
UserLeap, for instance, conducts research through micro-surveys that product managers and researchers can configure easily through a rich template repository, to target when and who to send surveys. Because the surveys are triggered by certain events (e.g., users stuck on a screen for 10 mins after onboarding), they can target the users at the right moment without having to reproduce this journey. For a product manager or designer, the hardest question to answer is often the “Why a user did x”, since “What” and “How” are often well captured by analytics products like Amplitude or Fullstory. Micro-surveys make it possible to answer the “why” without being intrusive to the user journey.
2. Integrated with existing design and analytics ecosystem
User research tends to be perceived as a process between users and researchers. It is true for the most part, but there’s also tight correlation and dependencies among research, design, and development or even to growth and revenue. Taking research out of context often results in broken workflow and introduces friction to the research conductors. It also stops the feedback loop to the existing ecosystem if some of the insights can be funneled back into the design/product management tools, or interpreted with additional data points.
For example, during the ideation and prototyping phase, designers often test their ideas with mockups or early prototypes. It used to be a cumbersome task that involved exporting the files into a slide deck and walking users through manually, or observing users click through the prototypes and asking questions simultaneously. This process is streamlined by Maze through a one-click integration with design tools, such as Figma and Sketch, which allows designers to rapidly build tests, get answers on anything from prototypes to copy, or round up user feedback in one place.
3. Automated, advanced analytics and reporting
Product feedback comes in different formats and mediums, ranging from low volume, high information density interviews to high volume, low information density public product reviews or NPS surveys. While quantitative research can reveal what is or isn’t working, it’s most often qualitative research that explains why.
But even though support tickets and video recordings are known to be a treasure trove of customer insights, there’s no effective way to combine or distill this information and present it in a coherent format. Amplitude, Mixpanel, or Fullstory capture user journeys through clean, structured data, but some of the most valuable feedback in open response survey questions, researcher notes, or audio/video files is by nature messy and disorganized.
Luckily new products are doing the heavy-lifting through transcription, tagging, searching, and clustering, so the data can be used effectively without having people spend significant time on cleaning and analysis. Dovetail, for instance, automatically transcribes user interview videos uploaded to the platform. Once the video is available as text, it can be combined with other text formats, such as notes and surveys, and tagged and annotated by the researchers. The key to reporting is providing a birds eye view but also drill downs to the point-in-time with context. The original video — which captures tone and facial expressions valuable for context and understanding — is still preserved. Reporting then becomes richer and more dynamic: Instead of showing only synthesized data, researchers can toggle between aggregated user insights and individual user response.
4. Collaboration and creating a research repository
To ship a successful product, the research process requires input and participation from multiple organizations/ functions, such as product design, engineering, marketing, data science — not to mention sales and customer success, who feed data to product teams.
It’s not just the process of collecting user feedback that’s collaborative. If the raw data can be preserved and shared effectively, the result is valuable when connected across teams. For example, a user interview about a checkout process is important for the design team to improve usability, and for the merchant team to refine the product offering.
But there’s not yet been a central place to consume this information, so breaking the silos of user research data to help everyone in the organization better understand their users will go a long way. Dovetail, for instance, is building a system of record for user feedback through integrations with Typeform, Zoom, Intercom, and Github. Every piece of user feedback is analyzed and stored in the research repository. Once research results are searchable and shareable, insights become self-serve and carry value beyond a single isolated department, giving user research the tools it needs to be a central part of product development and the organization.
We believe we are in the early days of new tooling for user research and there will be several valuable companies created in this space. As the product-led trend continues, the speed and precision to understanding customer insights will be the most important competitive advantage. We are looking forward to speaking with founders and teams building the future in this space!
Thanks to Ryan Glasgow, Benjamin Humphrey, and Jonathan Widawski who generously shared their industry knowledge to inform this piece.
###
The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein.
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.
Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.