Google has used its own project Astra to swing against OpenAI's GPT-4o. This universal AI agent is designed to be your assistant for daily life tasks and utilizes your phone's camera and voice recognition to give a response. 

Google also demonstrated Project Astra using smart glasses. 

To be clear, Astra is coming to the phone first and will be called Gemini Live, but may move to other form factors over time. But I can say that the demo shown in Google I/O2024 is impressive. 

Google says Project Astra can understand and respond to the world like people do, understand the context, take what you see and hear to take action, and also speak naturally without experiencing delays to remember.

power Project Astra was built on Google's Gemini model and other task-specific models. It can process information faster by continuously processing video and speech input. 

During a Project Astra demo, a person held an Android phone and left the live video of the camera open, asking a series of questions. Project Astra did not miss a beat.

For example, when he pointed the phone's camera at the table and asked what made the sound, Astra found the computer's speakers. Then the woman circled the top of the speaker and asked Astra what it was. She reacted correctly to the tweeter.

From there, Astra was able to provide a creative allusion about a bunch of crayons, identify which parts of the code do when directed at a computer monitor, and correctly identify the King's Cross area of London when the camera is directed out the window.

Then things got really interesting. A Google employee put on a smart glass and asked what this reminds me of when I looked at the board and stared at the illustration. What does this remind you of? Schrödinger's cat, Astra, replied.

Project Astra was asked to come up with a band name on the spot while holding a stuffed tiger next to the Golden Retriever. Answer: Golden stripes. Overall, Project Astra's spatial understanding and video processing is impressive, and we're excited to see where Google is adopting this AI agent. It's coming to the Gemini app later this year and we can't wait to test it out.