This Facial Recognition Experiment With Meta’s Smart Glasses Is a Terrifying Vision of the Future

Two college students have used Meta’s smart glasses to build a tool that quickly identifies any stranger walking by and brings up that person’s sensitive information, including their home address and contact information, according to a demonstration video posted to Instagram. And while the creators say they have no plans to release the code for their project, the demo gives us a peek at humanity’s very likely future—a future that used to be confined to dystopian sci-fi movies.

The two people behind the project, AnhPhu Nguyen and Caine Ardayfio, are students working on computer science at Harvard who often post their tech experiments on social media, including 3D printed images and wearable flame-throwers. But it’s their latest experiment, first spotted by 404 Media, that’s probably going to make a lot of people feel uneasy.

An Instagram video posted by Nguyen explains how the two men built a program that feeds the visual information from Meta Ray Ban smart glasses into facial recognition tools like Pimeyes, which have essentially scraped the entire web to identify where that person’s face shows up online. From there, a large language model infers the likely name and other details about that person. That name is then fed to various websites that can reveal the person’s home address, phone number, occupation or other organizational affiliations, and even the names of relatives.

“To use it, you just put the glasses on, then as you walk by people, the glasses will detect when somebody’s face is in frame. This photo is used to analyze them, and after a few seconds, their personal information pops up on your phone,”  Nguyen explains in the Instagram video.

Nguyen and Ardayfio call their project I-XRAY and it’s pretty stunning how much information they’re able to pull up in a short amount of time. They’re quick to point out that many of these tools have only become widely available in the past few years. For example, Meta’s smart glasses with camera capabilities that look like regular eyeglasses were only released last year. And the kind of LLM data extraction they’re achieving was only possible in the past two years. Even the ability to look up partial social security numbers (thanks to all those data leaks you read about every day now) was only possible at the consumer level since 2023.

As you can see in the video, they also approached strangers and acted like they knew those people from elsewhere after instantly looking up their information.

“The system leverages the ability of LLMs to understand, process, and compile vast amounts of information from diverse sources–inferring relationships between online sources, such as linking a name from one article to another, and logically parsing a person’s identity and personal details through text,” the creators say in an explanation document posted to Google Drive. “This synergy between LLMs and reverse face search allows for fully automatic and comprehensive data extraction that was previously not possible with traditional methods alone.”

The creators list the tools they used in their release, noting that anyone can request that those services remove their information. For reverse facial search engines, there’s Pimeyes and Facecheck ID. For search engines that include personal information there’s FastPeopleSearch, CheckThem, and Instant Checkmate. As for the social security number information, there’s no way to get that stuff removed, so the students recommend freezing your credit.

The students didn’t immediately respond to questions from Gizmodo on Wednesday morning. Meta also didn’t respond to a request for comment. We’ll update this post if we hear back. But in the meantime, we should all probably get ready for this kind of tech to emerge more widely since this kind of technological mash-up feels inevitable at this point—especially if any of the new smart glasses that guys like Mark Zuckerberg love so much really become mainstream.

It may take quite a while for the biggest tech companies to get behind it, but just as we saw OpenAI essentially shoot the starting gun for consumer-facing generative AI, any small upstart could plausibly make this product happen and start the dominoes falling for other larger tech companies to get this future started. Let’s cross our fingers and hope for the best, given the privacy implications. It really feels like nobody will have any semblance of anonymity in public once this ball gets rolling.