# What exactly is a “data-first” approach to product development? In a product development context, a data-first (also called data-informed or data-driven) approach is a way of building an app or software that uses quantitative and/or large-scale qualitative data and iterative testing to guide decisions. This is in contrast to an approach that relies on things like prior knowledge, gut feeling, or purely anecdotal feedback. We prefer the data-first approach here at Mixpanel because it’s a more efficient and systematic method that helps us get to a better iteration of the product faster. To take a data-first or data-informed approach, you need to know exactly what baseline metric or metrics you’re working with. Your baseline is your starting point and it could be something like an activation rate or conversion rate that you’d like to improve. Knowing your baseline is critical because it helps you create an effective solution and prove meaningful impact. At Mixpanel, we spend quite a bit of time getting a good understanding of this baseline before we execute any experiments. That includes being aware of things like seasonal fluctuations and other important contextual information. For example, imagine a hypothetical situation where we agree to tweak the product to achieve a 40% conversion rate. In that situation, it’d be helpful to know the baseline [conversion rate](https://mixpanel.com/blog/conversion-analysis/) we’re trying to improve, especially if it is comparatively low, say 20%. Having this baseline information allows product builders to push back and recommend a more reasonable target in this hypothetical situation instead of going along with an overly ambitious and unrealistic jump from 20% to 40%. Baseline data helps you set up your team for success. ## The benefits There are three main advantages that a data-informed approach gives you: 1. **A way to prove impact**: Executing a project or experiment is only half the battle—you also have to be able to show the impact. A data-first approach provides visibility into the effectiveness of every iteration or new feature. 2. **Ideas for improvement**: It’s much easier to pinpoint where things went wrong if you have data. (Did an experiment fail because of an execution issue in the code, or was it an unexpected flaw in the user experience?) 3. **A good alternative when users don’t know what they want (or like)**: Everyone wants to be user-focused. The problem is users don’t always tell us when they like or dislike a feature. Sometimes they can’t articulate what they want and we’ll get feedback like, “I just want this to be easier.” Data is the perfect complement to often-imperfect qualitative user feedback. ## The obstacles So why don’t more teams take a data-first approach? It’s probably not because they don’t want to. There are a few barriers: 1. **Not having (good) data**: Product builders may be hesitant because they don’t know if their data is accurate or robust enough. 2. **Not having the resources or skills**: Even if a team has the data, they still need someone to analyze that data and use it to inform ideas for improving the product—and extra bandwidth is not something that most product teams have. 3. **Not having enough time**: This is probably the most common challenge. Often we just don’t have enough time to dedicate to collecting baseline data beforehand, executing the experiment (some tests might require weeks or even months to reach statistical significance), and doing post-analysis. ## How we follow a data-first approach at Mixpanel Our product and engineering teams use a multi-pronged approach to gather data to find out what users don’t—or can’t—tell us. We use four key tools to support the process: ### 1. Product analytics The first step in our workflow is usually to look at Mixpanel for any interesting quantitative product usage data. For example, we recently made some changes to our invite modal, which prompts users to invite other people to try out Mixpanel. Initially, our primary metrics were “User opens modal” and “User clicks ‘Send invite’”—this was essentially the two-step funnel we were tracking. When we analyzed all the individual steps between these two metrics, we saw that at least 5% of the drop-off was occurring at the loading screen. This helped us come up with our hypothesis, which was quickly proven right: If we remove the loading screen, we can increase the CVR of our primary metric by 5%. ### 2. Session recordings From there, we can drill down into problem areas using screen or session recordings from a tool like Hotjar, which lets us see what the user sees, from their perspective, as they click through our website or software. Hotjar even highlights the user’s cursor so you can follow where they’re hovering or pausing (which might tell you they’re puzzled or taking longer to read through something). These recordings are extremely useful because together with the product analytics in Mixpanel, they give us a 360-degree view of how someone is navigating or using a product. ### 3. A/B tests Besides tools like Hotjar and Mixpanel, basic [A/B tests](https://mixpanel.com/blog/experimentation-isnt-an-add-on-why-you-need-a-strong-a-b-testing-solution-in-your-data-stack/) (or “split-tests”) are a key part of any engineering team’s toolkit. We use them frequently to experiment with ways to convert signups to paid users and help users [get to “aha” moments faster](https://mixpanel.com/blog/getting-to-the-aha-moment-fast/). Typically, we’ll make a feature available as a test to a certain subset of users and see if that test results in better results (like higher engagement, more conversions, and so on). ### 4. NPS NPS, or Net Promoter Score, tells you how likely your users are to recommend your product to their colleagues and networks. We use Sprig to collect NPS and verbatim feedback from our users via targeted surveys within the product, and this has been one of the top sources of qualitative feedback for us. Beyond just gauging NPS, you can set up custom survey questions as well, with response options like open text, multiple choice, and rating scales. While Hotjar and Mixpanel provide implicit signals that hint at where we should dig deeper, Sprig gives us very explicit cues that help us extend our analysis of those implicit signals further. Leveraging a good balance of these explicit signals and implicit quantitative data gives you the best chance at overall success—if you over-index on explicit signals, you’ll typically end up catering to a very small subset of users (either extremely happy or extremely unhappy), whereas using implicit signals comes with its innate challenges, such as interpreting the data correctly. ## Use both quantitative and qualitative data to keep users happy Contrary to what many people think, you don’t need a complicated or even super-robust process to start building in a more data-driven way. You can start with a simple one- or two-step experiment or funnel—for example, testing whether “Share” or “Invite” converts better on a CTA button—and add more steps as you think of them later. Remember, you don’t have to get it right the first time. What’s more important is you have a solid starting point that you can iterate on as you continue to gather more data. [Sonya Park](https://mixpanel.com/blog/author/sonya-park/) Senior Software Engineer @ Mixpanel ## Builders Sprig’s Kevin Mandich on a decade of building with ML and AI [Read Article](https://mixpanel.com/blog/sprig-kevin-mandich-building-ml-and-ai) ## Analytics Product Analytics and the data warehouse: A long road to a perfect pairing [Read Article](https://mixpanel.com/blog/product-analytics-and-data-warehouse-history)