Data-driven stories: a product that should exist
Data-driven stories: a product that should exist
It's easy to create a report. Why is it difficult to build value through business intelligence? Reports don't tell you what and when to change. Read: "Everything Starts Out Looking Like a Toy" #197
Hi, I’m Greg 👋! I write weekly product essays, including system “handshakes”, the expectations for workflow, and the jobs to be done for data. What is Data Operations? was the first post in the series.
This week’s toy: the World Wide Web was a toy in 1991. Here’s what it felt like to use it. Things that stand out in 2024: very intentional navigation, based on intent, with an extremely simple design that loads quickly. It’s all about the information (still good advice!) Edition 197 of this newsletter is here - it’s May 6, 2024.
Feeling lost in a sea of sales tools? Apollo.io has your entire GTM stack covered.
A true end-to-end sales platform with a huge (and very accurate) lead database, enrichment, outreach, call intelligence, scoring, calendaring, and email automation. They integrate seamlessly with your CRM and existing workflows. The quality of Apollo’s data is unmatched; they are ranked #1 for contact and company data accuracy on G2 with over 6800 reviews and a 4.8 star rating. Book a meeting to get started for free.
If you have a comment or are interested in sponsoring, hit reply.
The Big Idea
A short long-form essay about data things
⚙️ Data-driven Stories: a product that should exist
As a product enthusiast, I get excited at the possibility that products will live up to the hype. One of these ideas that comes up over and over again is the idea of an auto-generated story that starts with a metric and creates a visually appealing card with context that delivers true insight.
You’re probably looking at this and saying: “this is a dashboard report,” or “this doesn’t look that complicated.” This product should absolutely exist, and it looks a lot less complicated than it is.
What exists today: automated reporting that counts items and provides a description of what’s happening without a linkage to bigger insights.
What I’d like to see: automated reporting that takes a hypothesis into consideration and helps you see whether a report proves or disproves that hypothesis.
We need data-driven stories - a way to productize and deliver business insight from the work that we do every day - to create valuable next actions for a business.
What’s wrong with the current way we build reports?
We’re stuck in a rut. Building reports is impossible without knowing what we know, how we think, and a concept of insight.
What we know means the schema of data and how we count and collate information in a business process. For example, what qualifies a lead to be “sales-ready”? Once you know this you can count how many of these appear in a time period.
How we think is a bit more complicated, spanning the institutional knowledge of the organization and might be better said as “how do we capture the way we work in data definitions.”
Insight then becomes an observation of how we think and what we know about a business process, resulting in a narrative that makes sense to someone in the business who sees that report or reads that information card.
Next Actions could then be possible as a flywheel-like effect if the insight tells you about a deviation from a standard business process that you need to correct with workflow.
Think of the last time you tried to build a report. If you’re using most software, you need to know a lot about the schema of the data to make sense of the report. Simple counting of information is not challenging, and even the kind of trending report suggested in this card mockup is not too hard to produce in most BI systems.
Moving one step out from “counting”, how could we arrive at stories that tell you what to do (and when) to achieve goals based on business process, the structure of the data, and the actual data that’s arriving and changing in our systems?
We need a semantic layer for business, visualized in a data-driven story.
The concept of data-driven stories
A data-driven story shows a change (or lack of change) in a process over time and identifies the next action needed based on the semantic goals of the business.
What does that mean in English? It could be a story like this:
“We are trending over our lead goal for the month and it is time to evaluate our monthly budget for next month’s promotional efforts - consider spending more on source x which is converting at a higher rate for the same lead cost”
In the hypothetical example, we go beyond counting monthly leads and considering the cost per lead per source to looking at that performance closer to real-time and adding insight to the regular cadence we would normally take in considering marketing spend.
What are the components that would make this possible?
A data schema that describes a customer journey or a work product
Enough repetition to know what the “happy path” looks like for a process and the data that triggers each step in the flow (and what data signals a need to remediate or stop)
Initial caution to avoid outlier data and obviously wrong recommendations
Good examples here look like strongly correlated statistics that can be proven independently, e.g.
Trialers who use the product a lot in the first five days of trial are more likely to purchase than people who don’t use the product at all
Leads that get a response from sales within the first 10 minutes of their interest are much more likely to be engaged leads
Customers who come from certain sources stay longer and spend more than other customers
So far, these “stories” are relatively simple observations, and are “if this, then that” kind of ideas. How can we move beyond that?
Data-driven stories start as useful reports
Report generation is a well-trodden path. It’s easy to create bar, line, column, and area charts based on time series. You can use Metabase, Highcharts, or something similar and do these things almost for free. But the real knowledge in the reporting is understanding how metrics fit together.
I love the concept of a metrics tree and think it’s going to be hard for most organizations to adhere to an overall standard of governance around how metrics are built.
Wait, but isn’t the whole part of this idea that organizations should adopt a semantic layer that will automatically generate data-driven stories?
Yep. That’s why it’s hard. Different departments in companies behave in different ways, use different systems with different schemas, and aren’t necessarily using the same metrics or only link them in loosely coupled ways.
That’s why we want to start with the reports people value today and use that as a springboard to build new, useful insights.
How can you automatically generate something from an existing report?
I believe that teams focusing on metrics trees will win, but how we get to the tree definition is going to require some work.
Outcomes, defined as rules
No, this is not going to be a story about how generative AI will solve the problem, though I do think it will be a very useful partner in observing existing patterns and identifying possible outcomes for human operators to validate during training.
Like any good report (or a metrics tree), it helps to start with the end in mind: the output of the report card.
My hypothesis: we’re going to end up with a new primitive for reporting. It will be essentially an A/B test where you decide on the expected outcome and measure the result. By chaining the existing observations to outcomes and tagging it semantically as a good/bad outcome, you’ll start to be able to measure a process and identify whether it’s heading toward a good or bad outcome. This would allow you to create a story with impact, and to generate it automatically.
Until then, I suggest building reports that describe different points in the same customer journey. This should allow you to create segments in that journey and start to understand where different prospects fall out of of the funnel over time.
What’s the takeaway? Reporting is one of those tasks that appears easy to do right and depends upon a lineage of other knowledge. Much of the time “bad reporting” is also explained by a lack of clarity in business process and the associated data. Once we get better at linking the outcomes we want to the reports we build, we’ll improve our ability to generate automated insights from that reporting.
Links for Reading and Sharing
These are links that caught my 👀
1/ Code for everyone - The Nature of Code is a very readable handbook on how computers work. Even if you are an experienced technical professional, this book is worth reading. Check it out the next time you need a simple explanation to complicated things.
2/ Commodotize your complement - How do you become the leader in your category? Make a great product. How do you become a monopoly leader? Make everything around your product cheaper.
3/ When will we find the killer app for AI? - We’ve had ChatGPT for over a year. No one is quite sure how AI is going to help them in the long haul, or how it’s going to change everyday work. Ben Evans suggests the solution is going to be mundane but profound. (It’s not just add ✨ to everything)
What to do next
Hit reply if you’ve got links to share, data stories, or want to say hello.
The next big thing always starts out being dismissed as a “toy.” - Chris Dixon