Thinking about an Analytics Maturity Model
Does your organization have a plan to improve analytics? Here's one way to think about getting better at making data-driven decisions. Read: "Everything Starts Out Looking Like a Toy" #153
Hi, I’m Greg 👋! I write weekly product essays, including system “handshakes”, the expectations for workflow, and the jobs to be done for data. What is Data Operations? is a post that grew into Data & Ops, a team to help you with product, data, and operations.
This week’s toy: a game controller based on your physical movements. Remember Microsoft Kinect? This open-source tech lets you turn arm and body movements into inputs for games. Sort of silly, and sort of brilliant. Edition 153 of this newsletter is here - it’s July 10th, 2023.
Brought to you by Pocus, a Revenue Data Platform built for go-to-market teams to analyze, visualize, and action data about their prospects and customers without needing engineers. Pocus helps companies like Miro, Webflow, Loom, and Superhuman save 10+ hours/week digging through data to surface millions in new revenue opportunities.
If you're reading and haven't subscribed (yet), give it a try! And if you have any comments or are interested in sponsoring, hit reply.
Data Operations (“Everything Starts Out Looking Like a Toy”) is a reader-supported publication. Please consider subscribing.
The Big Idea
A short long-form essay about data things
⚙️ Thinking about an Analytics Maturity Model
Analytics is a hot topic at many companies, and it’s often hard to clarify what people mean when they talk about this practice. At a high level, the process of analytics provides a structured framework (or at least a starting point) for organizations to assess and improve the way they are doing business.
It’s pretty straightforward to talk about the overall goals for analytics:
Benchmarking and progress tracking: how are we doing?
Strategic decision-making: do we have the right information to inform decisions?
Data quality and management: is there unintended bad data in our systems?
Resource allocation: do we have the right people (or capacity) pointed at the right problems?
Risk management: have we evaluated the unknown unknowns?
It’s less clear to measure the progress of an organization thinking about improving data awareness along an Analytics Maturity Model.
Here’s a simple framework to start that conversation. The goal is not to explain every decision you might make, but to demonstrate the kind of transformation that data-aware organizations make as they transition from counting data to informing decisions.
Why should you care about this topic? You want to use new tools like machine learning models and AI. If you don’t know how to tell them what’s important, you’re going to get some unexpected results.
Let’s put some pieces together.
Stage 1: Just starting data journey
Most organizations thinking about data and how to improve their performance start here, where they understand that data is important but haven’t done much to make it more than a report.
They work to establish common definitions for information that are shared across departments, like “what is a Marketing Qualified Lead” or “what are the minimum fields necessary to deliver a lead to a seller”, and how often that information is gathered and cleaned.
The goal here is awareness and making sure things are counted in a consistent way so that a person looking at a data source and another person elsewhere in the organization will get the same answer when they ask a core business question.
For Saas businesses, the metrics they are tracking at this stage look like conversion rates, acquisition costs, or customer lifetime value. It’s likely that you have copies of the data everywhere, and that reports are most effective for a single part of the business.
Stage 2: Basic Instrumentation
When people say “we need a dashboard to track our metrics”, they are likely thinking about basic instrumentation, where a central database (a data warehouse) tracks and counts raw data and aggregates it into reports and data cards.
A dashboard might include reports from a single system like Salesforce or Hubspot, where there is deep data for part of the business, but struggle to connect data across multiple departments and objects.
It’s possible to measure simple things like the number of closed-won deals or the breakdown of the sales pipeline across multiple campaigns.
It’s not as easy to compare the current performance against a goal, especially when the goal moves frequently.
Stage 3: Data Informs Action
Now that you’ve got data in one place, what do you do with it? Ideally, you’re going to recommend what should be done next to fill a gap or to meet the pacing expectations for a goal.
One way to think about this is to ask: what would you do in response to a positive or negative result for a metric? This implies that you have set thresholds for your important metrics and identified what values correspond to a good or bad result. If you know what should be done to respond to a good or bad result, you have the beginning of a feedback loop based on the data you’re capturing.
Taking our example from above: if you’re approaching the mid-point of the month, have an average sales cycle of 45 days, and haven’t reached your goal for pipeline generation, that’s a problem for next month.
Using scaled projections and rates to project the answer to “are we trending to goal” helps you act before there is a problem.
Stage 4: Automated Action Triggers
When you’re tracking the progress of multiple metrics and understanding how they relate to each other in the buyer journey, you have the ability to design automated actions. These triggers – based on rules that you establish – make it easier for the team to focus on higher-value actions than counting or looking at a dashboard.
If you know that every time a prospect spends time on a pricing page and doesn’t request a demo or ask to learn more, a message is sent to their account manager or SDR, you can go beyond these simple actions.
Action triggers get more effective when they span objects in your organization. For example, when accounts in a certain segment take an action in your product during a trial, you can compare the impact of these product-qualified leads against others that do the same thing. Does a product-qualified lead indicate readiness to buy, or is it simply a result of activation?
Stage 5: Insight from Data
The goal of a data-enabled organization is to be able to pose a hypothesis, measure data to prove or disprove that idea, and put instrumentation in place to make that journey a standard operating procedure.
When you take a question like the one we posed above about product-qualified leads and measure the impact over time, you’ll learn (at least directionally) when those actions really mean something.
Do you need to prove the correlation between action and result for every hypothesis? Probably not. But if you provide decision support for your team by building insights out of the data that you’re collecting, you’ll be far ahead of the typical company that has not thought about how to enable data in their environment.
The goal? Make better decisions, faster. In Jeff Bezos’s 2016 letter to Amazon shareholders, he wrote: “...most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow. Plus, either way, you need to be good at quickly recognizing and correcting bad decisions.”
What’s the takeaway? There is no one best stage of the analytics maturity model. But you will get better at analytics and making meaningful decisions with data when you start focusing on the definitions for data, the thresholds for metrics, and the next actions you take when you learn about results.
Links for Reading and Sharing
These are links that caught my 👀
1/ How many ways can you send data? - Ethan Aaron of Portable has made a list of the Top 100 ETL tools to send data to systems that need it. Whether you’re just starting out in evaluating this space or you want an opinionated survey of available tools, this is a great resource.
2/ Will AI take your job? - Benedict Evans writes about the next industrial revolution: the one happening right now in AI. The fundamental question: will AI automate all of the available work? If you are a student of history, it helps to look at past technological revolutions to get an idea of what this one might hold. And yes, it might take the current job you have, while also creating new ones.
3/ Calendars enable time travel - This is a fantastic piece about calendars by Julian Lehr. What’s cool about it? He does a great job of pointing out the layering effect of combining different functions (meetings, personal time, focus time, blocking) in the same space. There’s definitely room in this space to improve.
What to do next
Hit reply if you’ve got links to share, data stories, or want to say hello.
The next big thing always starts out being dismissed as a “toy.” - Chris Dixon