Multimodal AI and AR: The Vision of Brilliant Labs’ Frame Glasses

Imagine a pair of glasses that can recognize objects, translate languages, and search the web, all with a simple voice command. That is the vision of Brilliant Labs, a Singapore-based startup that has launched Frame, the world’s first glasses featuring an integrated multimodal AI assistant. combines AI and AR.

combines AI and AR Frame is not just a technological marvel but a fashion statement. It pays homage to historical figures like John Lennon, Steve Jobs, and Gandhi, who shaped culture while donning iconic circular spectacles. Frame seamlessly integrates advanced technology into a lightweight, stylish package, virtually indistinguishable from conventional eyeglasses.

Frame is equipped with a discreet camera and a micro-OLED color display projected onto the lens. Using a voice assistant and multimodal AI models, it can do a real-time online search for whatever one is looking at, as well as answer queries and translate text or audio. combines AI and AR.

The voice assistant is named Noa, and it is powered by an integrated multimodal generative AI system capable of running GPT-4, Stability AI, and the Whisper AI model simultaneously. Noa performs real-world visual processing, novel image generation, and real-time speech recognition and translation. Frame’s multimodality empowers multiple AI systems to collaborate, unlocking limitless applications for this technology in everyday life.

For example, one can use Frame to identify plants, animals, or landmarks, get information about products or services, or learn about the history or culture of a place. One can also use Frame to communicate with people who speak different languages, or access online content or entertainment. Frame can also be used for education, work, or leisure purposes, enhancing one’s learning, productivity, or creativity.

Brilliant Labs has raised $6 million to date, with significant support from AR pioneer and Niantic CEO, John Hanke, whose company made Pokémon Go. The roster of advisors and investors also includes Coho VC, Brendan Iribe (cofounder of Oculus), Adam Cheyer (cofounder of Siri), Eric Migicovsky (founder of Pebble), Steve Sarowitz (founder of Paylocity and chairman of Wayfarer Studios), Nirav Patel (former core team member at Oculus and founder of Framework), Francisco Tolmasky (member of the original iPhone team), Plug and Play Ventures, and Moveon Technologies.

Under the leadership of ex-Apple executive and Brilliant Labs CEO, Bobak Tavangar, Frame introduced what it calls a paradigm shift in daily living. “Frame truly offers a new paradigm in the way humans interface with the digital world, and with each other,” Tavangar said in a statement. “Noa, a totally new kind of AI virtual assistant, empowers users to engage more with the world around them, not less. Our hope is that Frame enables people to connect with each other in entirely new ways, and opens up new experiences in everyday life, the workplace, the classroom, and more. This is the unique promise of bringing generative AI in front of the human eye. combines AI and AR.

Frame is designed to be open source and its design files and code can be found on Github. Frame is also compatible with an app called Noa, which contains an AI assistant that uses OpenAI technology to process visual analysis. At launch, Noa is free to use but is subject to a “daily cap,” the limits of which the company doesn’t share on its website. Brilliant Labs does intend to charge for a subscription to the AI services to lift the daily cap, but how much that will cost hasn’t been revealed yet. The company says it will announce it soon but it will keep a free tier in place. combines AI and AR.

Frame is available for pre-order from Brilliant Labs for $349 and the company intends to ship its first batch on April 15. Frame is a revolutionary product that combines multimodal AI and AR to create a new way of interacting with the world. Frame is not just a pair of glasses, but a window to the future. combines AI and AR.