I demoed Google’s Project Astra and it felt like it was the future of synthetic AI (until it wasn’t)

As I waited in line for the journalists and entered the small testing room, my eyes were glued to the wall screen and the Pixel 8 Pro in the hands of one of Google’s two product experts. The showcase was pre-recorded by Project Astrastand out in the company’s introduction I/O speech an hour earlier, was well-received — and a difficult act to pull off. Now, with the phone tucked away in my breast pocket, real-world testing is about to begin.

Also: Google Glass and Project Astra: Sergey Brin on AI wearables and his top use cases

Project Astra is the brainchild of Google DeepMind; The company’s vision is for a super-charged, multi-modal AI assistant that can process visual information, reason, and remember what it’s told or shown. It will not be as available as the new one Gemini features coming to Android devices, but the ultimate goal, at least for now, is to get this technology into phones and maybe wearable devicesbecome your daily assistant for everything we do.

For the demo, I was introduced to four use cases: Narrator, Dictionary, Alliteration, and Freeform. It’s all quite understandable and nothing exists Innovative AI models impossible to do, but the depth, speed, and adaptability of the responses is where Project Astra really shines.

First, I placed a pepper on the Astra’s camera feed and asked it to make an allusion. “The golden group sparkles splendidly,” it answered confidently, though not exactly. “Wait, that’s a pepper,” I said to Astra. “Perhaps the shiny chili is posing serenely.” Much better.

Also: The 9 biggest announcements at Google I/O 2024: Gemini, Search, Project Astra, etc

Then I added a toy ice cream cone and a banana to the mix and asked Astra if they would make a delicious lunch. “Maybe packed protein provides pep,” it suggested, understanding the nutritional imbalance between the three foods and, to my surprise, I was sticking with the alliterative words. By the way, Astra’s responses are relatively quick, enough to make me not want to pull out the paper. Rabbit R1 for comparison.

Perhaps more notable is how natural the AI ​​sounds – sharing a similar tone to that of OpenAI GPT4-o — as I panned the Pixel 8 Pro’s camera around and asked random questions about various objects in the room. Natural voices go hand in hand with Narrator and Narrator abilities, both of which entertain children, students, and those with free time.

Also: The best AI chatbots of 2024: ChatGPT and its alternatives

One issue I encountered during my roughly 5-minute demo was the way Astra frequently paused responses mid-sentence, possibly interpreting the sounds of outside chatter and nearby soccer activations ( where Google demonstrates how its AI can evaluate your kicking style) when I interrupt it. The ability to interrupt voice assistants is the latest step towards achieving more natural conversations.

However, in this case, the high sensitivity of the head-worn microphone of one of the employees may have affected the demo. That leads me to believe that in busier environments, such as when I’m traveling on the NYC subway or at a trade show, communicating with Astra might be more difficult than talking to a real person next to me.

Also: Innovative AI can transform customer experience. But only if you focus on other areas first

The other problem with Project Astra is its memory capacity. Currently, AI only remembers and tracks the locations of objects presented to it during a chat session (for just a few minutes). While the AI ​​might be able to recall that I put my phone in my jacket breast pocket when I started the demo, it theoretically wouldn’t be able to tell me where I left the TV remote that night. before — when such a feature would be most beneficial.

One of the researchers told me that expanding Astra’s memory capacity — which runs in the cloud rather than on the device — is definitely possible. The trade-off for such a performance feat will likely be battery life, especially if the goal is to put this technology in a wearable device as thin and light as glasses.

Ultimately, Google DeepMind gave me a clear vision of what the future of AI interactions will look like. They just have some wrinkles that need to be ironed out before I’m ready to introduce another voice assistant into my life.