Some technologists hope that one day we will develop a super-intelligent AI system where people can talk to each other. Ask it a question and it will give an answer like something a human expert has created. You can use it to ask for medical advice or help plan a vacation. Well, that’s the idea, at least.
In fact, we are still a long way from getting there. Even today’s most sophisticated systems are pretty stupid. I once got Meta BlenderBot’s AI chatbot telling me that a famous Dutch politician was a terrorism. In experiment where AI-powered chatbots are used to give medical advice, they tell patients to pretend to kill themselves. It doesn’t fill you with optimism, does it?
That’s why AI labs are working hard to make their conversational AIs safer and more useful before turning them loose in the real world. I have published a story about Alphabet-owned AI lab DeepMind’s latest effort: a new chatbot called Sparrow.
DeepMind’s new trick to creating a good AI-powered chatbot is to ask a human to tell it how to behave—And force it to back up its claims using a Google search. The participants were then asked to rate the reasonableness of the AI system’s answers. The idea is to further train the AI using human-machine dialogue.
In reporting the story, I spoke with Sara Hooker, who leads Cohere on AI, a nonprofit AI research lab.
She told me that one of the biggest hurdles in securely deploying conversational AI systems is their brittleness, meaning they perform brilliantly until taken to unfamiliar territory. , which causes them to behave unpredictably.
“It is also a difficult issue to deal with because any two people can disagree about whether a conversation is appropriate. And even if we do agree that something is relevant right now, this could change over time, or based on shared contexts can be subjective,” Hooker said.
Even so, DeepMind’s findings highlight that AI safety is more than just a technical fix. You need humans in the loop.