How Webcams Detect Your Heart Rate: rPPG Signal Processing Explained

So, there's the human envelope if you have any questions. Like, subscribe, you know what to do. Uh this video generated by Notebook LM. We gave it essentially a YouTube channel the website, but also this specific tool that that uh we've been testing for the last like 6 months or so. The one that the that takes your heart rate using the camera. So, we'll just have a hopefully somewhat critical review of that. >> You know that little camera on your screen? The one you use for meetings, for calls with family? Well, what if I told you it has a hidden superpower? A power so sensitive it can literally see something happening inside your body. Let's find out what it is. I mean, really think about that for a second. Can that simple, everyday camera actually detect one of your body's most fundamental vital signs just by looking at you? No wires, no special equipment, nothing. Believe it or not, the answer is a resounding yes. And in this explainer, we're going to break down exactly how this science-fiction-level tech works using nothing more than the device you're watching this on right now. All right, so let's dive right in. This isn't some far-off futuristic concept. It's a real capability that's been unlocked in everyday devices, all thanks to some incredibly clever software. The specific piece of magic we're looking at today comes from a web application built by Bioniq Shayos. They've created a tool that basically turns your webcam into a kind of signal microscope, letting it see these tiny, invisible changes that our own eyes miss completely. So, to get how this works, we first need to understand the science it's all built on. And it all boils down to something that happens every single second of every day, completely unseen. Every time your heart beats, it pushes a little surge of blood through your body. Now, that rush of blood causes the tiny vessels in your skin to expand just a little bit, which in turn causes a microscopic, totally imperceptible shift in the color of your skin. You can't see it, I can't see it, but your webcam can. Now, there's a big scientific name for this technique, remote photoplethysmography, or rPPG for short. It's a mouthful, I know, but the idea is actually pretty simple. Remote, cuz it's not touching you, and the rest is just a fancy way of saying measuring changes in blood volume using light. But, why a color change? What's so special about color? Well, it all comes down to hemoglobin. That's the protein in your red blood cells. And it just so happens that hemoglobin is fantastic at absorbing green light. So, when more blood rushes to your face, more green light gets absorbed, and your webcam sensor sees a tiny, tiny dip in the green channel. That is the signal it's hunting for. Okay, so the webcam sees this little dip in green light. How in the world does the software turn that into a measurable heartbeat? Well, it's a surprisingly straightforward three-step process. First, the software uses face detection to find your face and locks onto a small patch of skin, usually your forehead. Then, for every single frame of video coming in, it calculates the average green value of all the pixels inside that little box. Finally, it plots those averages over time. But, at this point, that signal is a complete and utter mess. And this, right here, is where the real magic happens. Getting that raw data is one thing, but turning it into something useful is the real challenge. The software has to find that tiny, rhythmic pulse that's buried deep inside all of that noise. I mean, think about it. That initial signal is corrupted by everything. A shadow moving across your face, the room lighting changing slightly, your camera's auto exposure freaking out, or even just you shifting in your chair. All of this noise completely drowns out the tiny heartbeat signal we're actually trying to find. And check this out. It perfectly illustrates the transformation. On the left, you see that raw, chaotic signal. It's basically useless. But then on the right after the software does its thing, you see this clean rhythmic wave. That is your pulse. This is the real aha moment of the whole process. So, the big question is, how do we get from that messy graph on the left to that beautiful clean one on the right? The answer is some very smart signal processing, specifically using something called digital filters. So, it cleans up the signal using two really clever digital filters. First, a high-pass filter gets rid of any slow changes. Think of it like ignoring a shadow that's slowly drifting across your face. Then a low-pass filter gets rid of any super fast jitter from the camera sensor itself, stuff that's way too fast to be human heartbeat. And what you're left with, right in that sweet spot in the middle, is the clean signal of your pulse. And here's another brilliant little trick. You know not all webcams run at the same speed? To make sure the math works perfectly every time, the system takes the incoming video and resamples it to a stable 30 frames per second. It's super smart because it means the whole thing just works, no matter if you have a brand new webcam or an older one. So, we've got this incredible technology, but it's really important to put it in context to understand what it is and just as importantly what it isn't, at least not yet. And the potential here is just wow. I mean, imagine future versions that could track multiple people at once or be so smart they can keep a lock on your pulse even if you're moving around. They might even use deep learning to pull out a signal in really low light or other tough conditions. This is really just the beginning. But, and this is absolutely crucial to remember for now, you got to read the disclaimer. The creators are very, very clear about this. It's a technology demonstration. It's an educational tool to show what's possible. It is not a medical device. You should never, ever use this for a medical diagnosis. It's a cool showcase, not a stethoscope. So, what this technology really reveals is that our world is just filled with invisible data, invisible signals. And if a simple webcam can see your pulse, it really makes you wonder, right? What other secrets are hiding out there in plain sight, just waiting for the right tool or the right lens to bring them into focus? >> It's pretty It's pretty decent. It's pretty decent review. And it's a uh pretty accurate uh review as well. The visuals uh pretty good. The website does have uh So, you can go on the main um On the main page, you have this drop down. And it has uh all tools is selected by default, so you just see a list, a very long list of everything we've done in the last 3 years. But then you can also select feature tools. Those are ones that were marked by the original creator. Yeah, those are like my favorite. Then you have neurological tools, so covering EEG. So, you have that advanced EEG signal simulator that we reviewed multiple times. Go check out previous videos. Uh we have some optical illusions. Optical illusions are relevant to neuros- relevant to neuroscience because they show you how the brain works. We have interactive MRI simulation, multiple MRI simulation, interactive TMS, transcranial magnetic stimulation simulation. That's quite a mouthful, isn't it? Transcranial magnetic stimulation simulation. That's right. We actually, just as a side note, we currently have something in development that is using the same brain model. So, it's quite an accurate brain model with about like 100 parts. A 3D model and we used that to actually map sound onto it. So, it's more you'll have probably The idea was that you'll have two separate versions. We actually can bring it up. Mike Brain, it's called Mike Brain. Yeah, so this is kind of stuff that will So, you have to Mike Brain audio driven brain activation visualization. Okay, so you have to activate and start your microphone. You have to give that permission. Everything is happening locally. In this case, all the code is on the page. All the code is on the page. And it's already running with only six activated regions of the brain, but then you can select up to 100 uh 150. Yeah, there's different uh amplitude and mapping, but uh yeah, it's working pretty well with my voice. You kind of have to It's kind of saturated. You can see it's being uh saturated at the moment with the So, there's both mapping of the of the uh frequency and the amplitude. There's something weird with the zooming in and out. Yeah, so if you see if I'm doing higher frequencies you get the back of the brain activated. If I'm doing lower frequencies Anyway, different parts of the brain get activated. That's why it's So, it's more of artistic. Obviously, it doesn't It's not scientifically accurate. There is a description at the bottom of page, you know, making the disclaimer of why is it not scientifically accurate. Well, because because if it was it will only activate the auditory path auditory regions of the brain, but currently it's activating the entire brain. Yeah. So, So, it's kind of nice for audio visualization. Eventually, yes, we'll have a more complex So, this will be a more artistic version of it and we'll have a more complicated, more complex more complex uh scientific version of the same thing.