MeshThink

When your software approaches the realm of not-quite-human, it needs a not-quite-human personality.

This new generation of AI-driven chat interfaces comes with a curious side effect. As chatbots start sounding more human, we ‘actual humans’ often assume they can think the way we do.

If someone asks a large language model (LLM) something like, “What would be a good face cream to use if I have an allergic reaction to glycolic acid,” and it responds by talking about three face creams and two toothpastes, many of us humans are going to rate it poorly. “Stupid bot – it doesn’t even know that face cream and toothpaste are different things!!!”

Put the same question into the search box of an online pharmacy, and we’d be pretty impressed if three out of the top five results actually meet our criteria. In most cases, online store searches will fall far short.

So, the LLM-driven interface performs better against the goal but feels worse to the user. Why? It all comes down to expectations. Once our computers are capable of conversing with us, it tends to trigger two assumptions:

1. Anything a typical person knows, this computer knows.
2. It also knows every fact available on the internet, because… computer.

That’s a pretty high bar! I suspect no chat-driven user experience will be able to meet it for a while. Are we doomed to have a generation of technology that’s more capable and, at the same time, more disappointing?

One thing I’ve started exploring is the personification of the interface. Is the key to making user-lovable chat interfaces actually making them seem less human, so that we humans assume less about their capability? If you’re creating an automated customer service bot, is personifying them as a human customer service agent hurting the user experience? Might they be better portrayed as a robot? An animal? A super-intelligent shade of blue?

It will be interesting to see if there is a role for great fiction writers in our next generation of tech, as ‘creating a great character’ for the computer becomes just as important as writing great code.