What role can generative AI play in information retrieval? A few months ago, I set out to learn what was possible using the current generation of AI models. I found some surprises, both good and bad. In this 3-part series, I’ll delve into the lessons I learned along the way.
Recap
In part 1, I tried a straight chatbot as an interface to search a product catalog. I learned two big things along the way:
- Using generative AI can be a game-changer when the user is looking for ideas rather than specifics
- The chatbot pattern leads people to ask anything. This can be bad when your agent doesn’t know how to answer everything.
In part 2, I built a more restricted chatbot interface, with response prompts, and discovered:
- AI approaches to recommendation pull ahead of conventional approaches when lots of detail is provided, and it’s possible to build a UX that triggers users to provide detail.
- Pre-canned responses did not effectively restrict what users would ask the chatbot to do.
Taking the Chatbot out of the Chat
Discussing the over user experience and problems with my colleague John Choquette, we articulated the benefits. The value provided by the AI came from interpreting text to make very precise recommendations, and from thinking of questions to ask the user as follow-ups.
However, the chatbot experience also depended on the user using free text to execute the next UI action and reach the next step of the experience. Here, very little value was being added over ‘conventional’ navigation and design. In fact, depending on the user to say what they wanted next was much worse than traditional UX design. Clear UX that educates the user on what actions are *possible* next is still much better than an open-ended prompt.
This led to the biggest change in the overall user experience – casting out the chatbot-style UI entirely! Generative AI was still to play a big role in solving the gift-finding problem: eliciting details of the gift recipient, brainstorming ideas for them, and communicating why a gift might be a good fit. But user actions themselves would be recast ‘traditionally’ – with action buttons and icons, rather than as a conversational input.
Putting it All Together
To try to pull in the best parts of generative AI, I settled on the following approach: I’d ask the AI to continuously brainstorm gift ideas based on the information it had so far. In parallel, I’d have it interview the shopper to better get to know the person getting the gift. Breaking the chatbot paradigm, I’d still have the questions being asked dynamically generated by the AI, but presented in the style of a questionnaire instead of a chat dialog.
To demonstrate what this looks like in practice, I went in to do some hypothetical Christmas shopping for Wednesday Addams.
Out of the gate, it looks for the basics: age, gender, price range. At this point, it only knows Wednesday as a ‘teen girl’ – and starts giving ideas very similar you what you might expect from a ‘traditional’ gift finder tool:
But then it prompts me to describe her in more detail. Here’s what I told it:
“She is stoic and mysterious, and has a very dark and deadpan sense of humor. She dresses in an antiquated goth style, and hates all color, preferring black. She is an avid writer, and is a fan of dark and macabre writing. She’s an excellent crossbow marksman. Her interests lean towards the morbid, with a love for dark poetry, ancient rituals, and creepy crawlies. She has a disembodied hand for a pet.”
Right away, you can see it getting a much different angle on its gift recommendations – stuff you’d never find if just searching for a teen girl:
Note the generative technique at work here. You can see the AI is trying to come up with a category of gift ideas first (the heading) and then looking for products that fit the bill.
After a couple more questions, it found this gem:
I wasn’t aware that there were Nintendo Switch games out there which might appeal to a Wednesday Addams-type, but there was no point in going any further. I’ve found my gift 🙂
Conclusions
Is this a revolutionary new stand-alone shopping tool? No, probably not. After all, most searches are quite a bit more directed, and the majority of us, like it or not, are closer to ‘mainstream’ than we probably care to admit. 90% of the time, ‘traditional’ shopping and browsing is faster. (And cheaper to host!)
Are there techniques in here that can be used to enhance discovery experiences? Absolutely. “Brainstorming at scale” is a huge strength of generative AI – while it doesn’t nail every result, it can keep going, tirelessly. Even with only 1 in 4 suggestions being on the mark, it’s going to quickly outpace me as a human coming up with ideas. I would never in a million years have thought to look in the Switch gaming section for something for Wednesday Addams, but there it was! There is absolutely a place for flexing this muscle anywhere discovery is needed.
If you’re curious (and especially if you’ve got someone quirky to find a gift for), go ahead and try it out as you’re doing your holiday shopping this year. It only has about 20,000 Amazon best sellers, but as you can see, sometimes that’s still enough to find some really on-the-spot ideas. (Full disclosure: I picked up about 10 gifts for my wife while testing, so let the buyer beware 🙂
MeshThink 
Leave a comment