A discovery engine > a generic demo
"Everything Starts Out Looking Like a Toy" #115
Hi, I’m Greg 👋! I write essays on product development. Some key topics for me are system “handshakes”, the expectations for workflow, and the jobs we expect data to do. This all started when I tried to define What is Data Operations?
This week’s toy: a game that teaches you how to repair older electronics. Is this the future of learning? Edition 115 of this newsletter is here - it’s October 17, 2022.
The Big Idea
A short long-form essay about data things
⚙️ A discovery engine > a generic demo
A first-time user experience aims to introduce the user to the software, teach them basic navigation and concepts, and demonstrate value in a short period of time. As a product manager or marketer, building that demo is a challenge. You want to deliver the real feel of the application along with data that makes sense to the user.
One popular way to demonstrate Saas software is to use a chrome plugin to record progress in a typical flow, adding tooltips and a small amount of interactivity to create the equivalent of a “choose your own adventure” text game. An unengaged user doesn’t have to do much here beyond clicking next, next, and next to reach the end of the demo.
How do you populate the empty space of the demo (or the initial use of the product) with meaningful data that looks like the kind of data you would use in the “real-life” usage of a tool? If you could deliver a demo along with data that seems familiar, the path to value is much shorter. Combining good data with relevant use case discovery would get the first-time user to relevance even faster. Today, users are stuck navigating an experience they don’t care about or using an app from scratch.
A thought experiment: discovery as demo
The problem with the existing experience that demo-building tools provide is they set up a situation that requires context to be relevant. If you are selling a project management app and your demo asks the user to set up a project, add tasks, change the status of a task, and see the impact on the schedule, this is a useful comparison for someone who has managed a project of that type before.
Many demo-building tools ask you to go to each step in the process, record what you do, and repackage the result as a kind of interactive prototype where the user clicks specific areas or types information to move on and complete the prototype. Other tools provide “guides” or navigable steps that overlay the actual application and watch for changes in key fields and screens to confirm that the user has reached the next step of the process. But both of these processes neglect the need to personalize the experience for the prospect.
If you were meeting in person with a prospect, you would use the tried-and-true method of discovery questions to learn more about the customer’s use case. What are you hoping to achieve? How will you measure your success? What does a typical project management process in your company look like? Who is involved and how do you know how to move a project to the next stage? Finally, what doesn’t work with your current solution?
A perfect solution would identify the data that you need, the structure it has, and key elements that would allow you to generate meaningful synthetic data. In this case, “meaningful” means data that feels like the kind of data you would use in the product if you were using it for real and “synthetic” means the system used your input to create or synthesize information that looks meaningful, even if that data hasn’t been generated yet in real life.
Here’s what you’ll need to discover from the prospect:
The actors - who engages with the software, what roles do they have, and what are key attributes to store (like their name and email address for starters)
The objects (or entities) - the structure of the information that we store, and things that might be related to each other (like a project and project task items)
The attributes of objects - for a project, the name; for task items, the due date, and owner
What happens - the order of expected actions, the change in objects or attributes that results, and the sequence of events that happens
Discovery engines gather information in context. If you knew all of this information, you could create a simulated product experience (or a real product experience with simulated data) that closely matches the way the prospect would use the application, while removing the effort and time it takes to get to that value.
This sounds like a solution that is grounded in science fiction or perhaps a little too dependent on a trained, unpredictable model like GPT3. So how do you prototype this solution with a more realistic set of steps?
How to build a discovery engine
The good news with our approach above is that it resembles a decision tree. This is a very well-known format. If we squint at the problem, this is similar to the process of building processes for call center agents, where they need to gather valuable information in the shortest period of time with the goal of resolving a customer’s inquiry and achieving high service levels.
If we borrow from the idea of conversational chat, a discovery engine might engage a prospect in an interview, walking through a few questions and answers to build the data model necessary to populate or generate synthetic data for a demo. But be cautious. Prospects lose focus quickly. They don’t want to answer endless questions from a bot.
So we can’t present an interview to create a decision tree to the prospect to help them populate a demo with custom data that makes sense to them. But that process would be valuable as a scenario author to answer a targeted interview that helps me populate the synthetic data for a scenario that seems relevant to a persona.
The discovery engine in action
Here’s where we’ve landed:
customized demos with relevant synthetic data are superior to canned demos
prospects don’t want to do a lot of work to assemble this data for us
scenarios resemble a decision tree and have a known data structure that can be populated ahead of time for a persona
Dear demo-building software product managers, please make it easier to build data sets of relevant data, link those data sets to scenarios and screens, and wrap all of this up in a simple interview for the prospect.
What’s the takeaway? Prospects deserve better demos with data that makes it easier for them to imagine using those products. Building these examples is challenging with the current set of demo-building tools, that don’t handle the hardest part of this process: creating a relevant data set.
Links for Reading and Sharing
These are links that caught my 👀
1/ Tips to improve data viz - Here are a few excellent tips to improve any images you make to tell data with stories. One of my favorites: adding notations to data charts, close to where the important inflection in data occurs.
2/ Learn R: a primer - this open-source introduction to data science will help you to get started building and manipulating data with R. With these skills, you can run tests for probability, analysis, and transform and update data to achieve data goals. If you’ve wanted to learn R and didn’t know how to start, this is an excellent place to get going.
3/ When one team leaves the playoffs - as a Seattle Mariners fan, I found this comforting. I’m not sure it’s backed by data but still, it’s relevant.
What to do next
Hit reply if you’ve got links to share, data stories, or want to say hello.
Want more essays? Read on Data Operations or other writings at gregmeyer.com.
The next big thing always starts out being dismissed as a “toy.” - Chris Dixon
Generating synthetic data on the fly using prompts can be such a game changer! ;)