Search is now a multi-player application
OpenAI's new search feature changes most things you know about search. You're still responsible to build a better search result. Read: "Everything Starts Out Looking Like a Toy" #223
Hi, I’m Greg 👋! I write weekly product essays, including system “handshakes”, the expectations for workflow, and the jobs to be done for data. What is Data Operations? was the first post in the series.
This week’s toy: using an 8k monitor as a desk display. If things get much bigger, this will remind me of the walls described in Fahrenheit 451. But seriously, this size of display is competitive with the cost of four 32” 4k monitors. You can imagine a future where large displays pull down into place or unroll from a hidden location. Edition 223 of this newsletter is here - it’s November 4, 2024.
If you have a comment or are interested in sponsoring, hit reply.
The Big Idea
A short long-form essay about data things
⚙️ Search is now a multi-player application
OpenAI just launched their implementation of search. This fundamentally changes the concept of Internet search and turns it into … well, whatever want it to be.
You can use OpenAI as a traditional search engine. You can also riff. For example, you can ask the software to tell you the History of the American Revolution as a rhyming poem.
Who knows whether this was useful or an LLM parlor trick, but it does give you a jumping-off point of real information to learn more.
What’s the main difference that search offers from “typical” ChatGPT answers? You’ll see a list of resources used to create the summary. Each of these summarized links is similar to the result you’d see in a regular search.
You can even change how the software responds to you by giving it formatting and templating instructions (try “Provide your response in the format of Google circa 1998” for a fun version of this trick).
The epiphany is that OpenAI has broken search into a fractal of a million billion searches where you can interact differently with the results. Wow! Many applications are waiting to be born from a structured rephrasing of search and it’s exciting to see the potential.
Search UI is now UX
The good news? If you don’t like how search results are organized, you can ask the search engine to change how it provides these results. ChatGPT retains the ability to tell you about the sources for its citations, but it’s not super clear why an individual item is highly ranked.
Address this issue by asking “why is this item highly ranked?” You’ll get a subjective answer from the LLM explaining the basic reasons an individual result was highly ranked. This is slightly unnerving as it’s somewhere between a conversation and a structured result.
This fungibility means that more than ever, the search results one person sees are not going to be similar to the search results seen by another person. Asking a question will result in a similar general answer but it’s really easy to twist that answer by giving different input.
In the future we might start talking about “search arguments” or some other new term rather than blindly believing the results we get from search queries in tools like ChatGPT. The citations might be roughly similar if the software is working well but the way we describe the “answer” will be different.
The User is responsible for output
What’s the bad news?
As a search user, you are now responsible for validating the quality of the output.
SPOILER: you were always responsible for having a critical thinking mindset and evaluating whether the results you got from a query “made sense” by looking at the citation (link) and assessing whether it was a reliable source. The main difference is that now you can add questions to your search engine.
For example, you can add instructions to your search result that improve the usability of the result or transform it into an image or another kind of summary. If you’re evaluating local driveway contractors, you could build a visual flow of the logic.
Could you please provide a image flow chart in mermaid.js mapping your citations to the reasons you applied them to this example?
The possibilities are almost endless. It’s going to be a lot easier to converse with your search results, getting much deeper into the facets of the conversation, or to build structured results.
Ask better questions, get better answers (maybe)
Here’s where the multiplayer bit and the responsibility bit come into play. Unlike a traditional search engine, ChatGPT is going to listen to you and format the results in a favorable way to your prompt.
If you ask ChatGPT to respond as a Magic 8 Ball, you’ll get a result, but maybe not in the format you expect.
The good news is that ChatGPT will tell you why it’s ranking information in the way that it does, and provide sources. You can even ask probing questions about sources if you have questions, or ask how recently they were crawled.
Request for feature: publish your Search prompts for ChatGPT and let others use them. You can do this by copying prompts, but I suspect this will become more popular over time.
Deliver more transparency for your searches
Based on the marketing for this feature, I think most people will search OpenAI conversationally, checking the results only if they find out that they are super wrong or hallucinatory.
I want to be deliberate about the way I use LLM search. I’m going to proceed cautiously.
To do this, I wrote this prompt:
"Respond to search queries in either 'Google circa 1998' format or 'Magic 8 Ball style' based on my instructions. For 'Google circa 1998,' give a directory-style response with concise descriptions, and add source links or search terms when direct links aren't feasible. For 'Magic 8 Ball style,' reply with short, mystical statements while still making relevant recommendations. Validate your responses by ensuring results are from reliable sources and meet the query’s criteria (e.g., quality feedback, local relevance). Summarize your approach after each response in a table to confirm accuracy and structure."
I did this for a couple of reasons:
I’m not sure I trust search results from an LLM, so I want to see how it will respond and how it build citations
I’d like to automatically fact check whenever possible so that I can be alerted to potentially biased content
It sounds like fun to engage in a search Old School style. Perhaps in the future there will be other fun modalities, like playing a casual game as you do your searches.
Still an open question? How you can know if the algorithm you’re using for search gives you filtered information or the “right” answer. The only way to ensure you know what you’re getting is to ask questions of your sources and triangulate good results.
What’s the takeaway? Search just changed in a big way. But the responsibility of understanding the results hasn’t been altered. It’s more important than ever to have the equivalent of a nutrition label on our searches to find out when we’re consuming junk information.
Links for Reading and Sharing
These are links that caught my 👀
1/ The chronicles of Busytown - Chris Ware writes a charming retrospective of Richard Scarry’s illustrations. If you’ve ever read or appreciated these books, this will be an enjoyable read.
2/ Fixing the messy inbox - One of the use cases with the most promise for AI software is finding structured information inside of unstructured content. One of the most obvious places? The inbox. When this strategy moves from VCs talking about it to developers incorporating it into everyday GTM workflow, it’s a true shift.
3/ Finally, read my grocery list- Google is preparing a model that learns how to vectorize handwriting by watching humans write. With this knowledge, you can create reasonable transcription (even for messy handwriting) and read and write handwriting samples in vector format which takes up much less space than big images and Optical Character Recognition (OCR).
What to do next
Hit reply if you’ve got links to share, data stories, or want to say hello.
Want to book a discovery call to talk about how we can work together?
The next big thing always starts out being dismissed as a “toy.” - Chris Dixon