How it works: You can replace a raw camera feed with the filtered stream in your app. The filter processes a live video stream, applies privacy protections, and outputs a privacy-compliant stream in real time. You can use this processed stream for AI apps, social apps, or anything else.
Features: Currently, the filter blurs all faces except those who have given consent. Consent can be granted verbally by saying something like "I consent to be captured" to the camera. I'll be adding more features, such as detecting and redacting other private information, speech anonymization, and automatic video shut-off in certain locations or situations.
Why I built it: While developing an always-on AI assistant/memory for glasses, I realized privacy concerns would be a critical problem, for both bystanders and the wearer. Addressing this involves complex issues like GDPR, CCPA, data deletion requests, and consent management, so I built this privacy layer first for myself and other developers.
Reference app: There's a sample app (./examples/rewind/) that uses the filter. The demo video is in the README, please check it out! The app shows the current camera stream and past recordings, both privacy-protected, and will include AI features using the recordings.
Tech: Runs offline on a laptop. Built with FFmpeg (stream decode/encode), OpenCV (face recognition/blurring), Faster Whisper (voice transcription), and Phi-3.1 Mini (LLM for transcription analysis).
I'd love feedback and ideas for tackling the privacy challenges in wearable camera apps!
Most of these features are aimed at protecting bystanders, but I'm also interested in exploring privacy protection for the wearer. For example, automatically shutting off recording in bathrooms, or removing identifiable landmarks from the video when they could reveal the wearer's location, depending on the use case.
I can taste the irony.
Have you tried running this on a phone or standalone smart glasses? 30fps is horrible performance on a laptop given that it's probably 10-100x more powerful than the target device. And what laptop? Based on your screenshot, I'm guessing you're using a new Apple Silicon mac which is practically a supercomputer compared to smart glasses.
The 720p 30fps figure is from a PoC implementation, so there is still significant room for improvement. And yes, the demo is on an Apple Silicon M2.
I don't know this would actually make others any more comfortable/guaranteed, which is where the bulk of the privacy concerns around smart glasse lie. I'm also not sure if that's actually a problem for more technology to be able to solve. The nice thing about e.g. recording consent is you do it on your device, or in some cloud device by a 3rd party. The moment you do it through the other person's device it's no different (from a trust/easiness/guarantee factor) than just asking the person themselves to not record you (again, beyond the benefits to the operator themselves around that). If some guy walked up to you with a literal video camera pointed at you and said "it only records your face if you give it consent":
- Do you even trust the person is telling the truth?
- If you do, do you trust the technology to work 100% of the time?
- If it is a lie or doesn't work and they post something, are you any better off than if you weren't asked for consent and said no? (note: separate this from the value to the person operating the camera, who does get value from those that say yes).
- If you say no, are you really not going to feel awkward compared to not having someone recording the rest of the situation?
Maybe the answer for you to all of these is "Absolutely!", I'd bargain to bet it's not the case for the vast majority of those concerned with the privacy implications of the technology though (which I don't consider myself a part of that group necessarily, I'm just putting myself in their shoes for a second).
.
So I'd split my thoughts into two main sections: I'm not sure technology is going to solve any external privacy concerns here, but I think it's an interesting approach to internal privacy concerns with the tech.
I fully expected this to be about solutions for empowering the user to ensure that her audio and visual experiences weren't being exfiltrated to a third-party, not that they were being censored before even getting to her.
Censorship is telling everyone else they have to avert their eyes as you walk past.
Whether or not privacy is proper in a particular situation, or whether censorship is, that's another matter. But it's frustrating to see them confounded in this way.
Crippling the user's device (which is ultimately an affront to freedom of general purpose computing) in order to censor data which is plainly available through sensory input is not, in any sense, "privacy".
This does nothing to alleviate my privacy concerns, as a bystander, about someone rudely pointing a recording camera at me. The only thing that alleviates these concerns about "smart" glasses wearers recording video, is not having "smart" glasses wearers. I.e., not having people rudely walking around with cameras strapped to their faces recording everyone and everything around them. I can't know/trust that there is some tech behind the camera that will protect my privacy.
A lot of privacy invasions have become normalized and accepted by the majority of the population. But, I think/hope a camera strapped to someone's face being shoved into other peoples' faces will be a tough sell. Google Glass wearers risked having the camera ripped off their faces / being punched in the face. I expect this will continue.
Perhaps your tech would have use in a more controlled business/military environment? Or, to post-process police body camera footage, to remove images of bystanders before public release?
The only way for this to work are legal regulations. But those can be easily dismissed as not possible to implement. So this is good PoC to show what is possible and way to discover how this could function. Without such implementation I don't believe you are able to convince anybody to start working on such regulations.
Something like this tool is ridiculous against companies like Google or Meta. Just with their phone apps and OS control with a video like the displayed those companies could know exactly who each person in the video is, what are they doing, who they are with, and record that information forever.
In the video I see three young women, another woman near the zebra crossing. A young woman with a young man, a woman walking with two men on the sides, and another young couple. I know their heights, if they are fat or slim, the type of their hair and so an AI could know that and with that information and a little more like someone of one group having location activated it is enough for a computer to automatically decode the remaining information.
If enough people wear those stupid glasses it means in a city everyone is surveilled on real time with second and centimetre accuracy, included indoors in places like restaurants or malls.
This is too much power that no company or institution should have. If Meta or google have the ability to do that, they will be required by the US government to give that info automatically with some excuse like "national security".
However, even with this uneasy feeling, one has to recognise a street is a public space and I don't see how one can have reasonable expectation of complete privacy there. There is nothing rude about recording what you can see.
The privacy expectation I have is not that my picture will not be captured, but that such recordings from many unrelated people will not be aggregated to trace my movements .
So in summary, I think everyone has a right to photograph or record whatever they like in a public space, but the action of pooling all such recordings, and running face tracking on them to track individual people (or build a database of movements, whatever) is breaching these people's privacy and there should be laws against it.
If I understand correctly how this works consent can come from camera operator and be attributed to recorded person
Verbal consent is just one example. Depending on the situation, other interfaces may work better, such as having a predefined list of friends who are always consented.
The problem isn’t consent, the problem is that the gun is being needlessly pointed at you in the first place.
https://en.wikipedia.org/wiki/White_Christmas_(Black_Mirror)
Note i love my Meta Ray Bans yet they aren't durable and after 2 breaking after 20 months Im not sure I will buy another pair (maybe the upcoming Oakleys but still same not reliable to non-durable tech in those). But for me they are great for taking pics and videos (asking the time too when my phone isnt on me) of my life and when traveling solo using the Live AI feature to learn more about my surroundings. Like what's the history of the train tracks Im walking alongside.