How AI Reads Your Heartbeat Through a Webcam: rPPG & Eulerian Magnification

Right now, uh the webcam staring at you from your laptop could actually be reading your heartbeat. No wires, no chest straps, just code tracking these invisible microscopic pulses of blood under the skin of your face. But, you know, before you start panicking about some sci-fi dystopia, you need to know a slightly hilarious secret about this technology. That same genius AI gets completely derailed, like totally confused, if a cloud happens to pass over your window and changes the lighting in your room. >> Yeah, it really is the ultimate paradox of modern medical technology. I mean, we have these algorithms that can seemingly perform magic, but they're incredibly fragile the second they hit the chaotic reality of the real world. >> And that contrast is exactly that we are getting into today. So, welcome to today's deep dive. For you, the listener, we are pulling from a truly unique stack of sources today. We're looking at the website and the YouTube channel for a platform called Bioni Chaos. It's run by a biomedical engineering PhD. And their whole mission is to take hard science, you know, dense, intimidating medical data and make it highly visual, interactive, and honestly pretty fun to play with. >> Right. And Bioni Chaos is like the perfect name for it, too, because when you see commercials for health tech, it's always pristine. Blue holographic data, sleek spaceship interfaces. But this platform completely strips away that Hollywood gloss. >> Okay, let's unpack this because they have built tools that turn abstract data into things you can actually see. And I want to start with that webcam thing I just mentioned because it absolutely blew my mind. They have a web app called the real-time signal amplification microscope. And it uses something called remote photoplethysmography, rPPG. Tell me how this actually works cuz I'm stuck on the basic physics of it. I mean, how can a camera lens detect blood if my skin isn't transparent? >> Well, it's not looking through your skin, it's measuring the light bouncing off it. Every single time your heart beats, a fresh pulse of blood travels through your vascular system and flushes into the tiny capillaries just beneath the surface of your face. >> Okay, but I look in the mirror all the time. I definitely don't see my face flashing red with every heartbeat. >> Right. Yeah, human eyes can't perceive it because the change in color is microscopic, but a camera sensor actually can. That influx of blood slightly alters how ambient light is absorbed and reflected by your skin. >> Oh, wow. >> Yeah, so the camera captures those minute variations in pixel color frame by frame. >> So, my face is actually flashing like a biological beacon, I just can't see it. But even for a camera, that signal has Oh, hey there. How can we >> I can just mention quickly that I'm actually getting a good reading. I'm testing it right now. There's a human in the loop. If if you have a question or something, but I'm testing it right now and measurement is similar to what my smartwatch, Garmin watch is showing. So, it's seem to be accurate within the one plus minus one beats per minute. >> That is fantastic. We absolutely love when you interact. >> Oh, that's excellent feedback and thank you for testing it live for us. >> It's actually a great demonstration of the power of the tool, right? >> Absolutely. The fact that you are getting a reading with that one beat per minute of your Garmin watch is a testament to the core math of our PPG. >> Okay, because that reading, the one you're getting right now, that signal has to be impossibly tiny. >> It is, which is what makes it so impressive. >> Is my laptop camera basically a Star Trek medical tricorder or now? >> Well, it feels like it, but what's happening is a massive mathematical amplification of the signal you are getting. >> It feels like trying to measure the moon's gravity just by looking at a video of ocean waves. >> That is actually a brilliant way to think about it. And to stick with your ocean wave analogy, if you wanted to measure that invisible gravitational pull, you couldn't just look at the raw water. >> Right. >> You'd need a mathematical tool to exaggerate the movement of the tide. >> And that is exactly what this app does. >> What's fascinating here is how it amplifies the invisible. It uses a technique called Eulerian video magnification. >> Eulerian video magnification. Okay, what does that actually mean in practice? >> It means the algorithm takes the video feed and mathematically isolates those specific invisible color changes and then aggressively amplifies them until they become a measurable visible signal. >> It turns up the volume on the microscopic world. >> Yes, exactly. >> Okay, that sounds like absolute magic, but this is where the cloud passing over the window ruins everything, right? >> Yeah, unfortunately. And that's why your clean environment is giving you that excellent reading right now. >> Ah, the controlled conditions. >> Because going back to your Star Trek analogy, the Enterprise didn't have to deal with the messy, unpredictable environment of an Earth-bound home office. >> True. >> The math behind Eulerian magnification is brilliant, but it amplifies everything, including the noise. >> So, if I just shift in my chair or if the sun goes behind a cloud, the camera sees that lighting change and thinks my heart just skipped a beat. >> Yes, exactly. The algorithm suddenly struggles to distinguish between a shadow moving across your cheek and an actual pulse of blood. Engineers call this noise. >> Noise, got it. >> Right. Motion artifacts and lighting changes are the absolute bane of remote monitoring. >> In one of the Bionic Chaos YouTube videos, the creator actually live codes with an AI, specifically Gemini, trying to fix these exact monitors. >> They're constantly battling this noise. >> And the video talks about using things like sub-pixel camera monitoring and something called FFT. >> Fast Fourier transform to track respiration and filter out that noise. >> But again, to a layman like me, that's just a bunch of alphabet soup. How does an FFT actually separate my breathing from me just squirming in my desk chair? >> Okay, imagine you are listening to a massive symphony orchestra and you want to isolate the sound of just one single oboe. >> By trying to find that one signal in the chaos. >> The overall sound wave hitting your ear is incredibly messy. >> Oh, yeah, definitely. >> Right. It's the violins, the drums, the coughing in the audience all mashed together. >> That sounds like my brain on a normal Tuesday, but go on. >> Well, what a fast Fourier transform does mathematically is take that messy complex wave and break it apart into its individual frequencies. >> It separates the deep thrum of the bass from the high pitch of the flute. >> Oh, I see. >> So, in the context of your webcam, the symphony is all the pixel data. >> Your heartbeat, your breathing, your chair squeaking, the sun moving, the FFT separates those rhythms. >> Okay, because breathing happens at a slow, specific frequency. >> Exactly. And the heartbeat is a faster frequency. >> By isolating those specific rhythmic frequencies, the algorithm can find the actual biological signal hidden inside the chaos. >> Which is why you are getting such a clean, accurate reading in a stable environment. >> Okay, that makes so much sense, but it also sounds exhausting to program. >> It is. >> Wait, this makes me think of something else. If these algorithms are this easily confused by a shadow on my face, how does the AI even learn what a heartbeat looks like in the first place? >> It must need perfect data to train on. >> And you have just hit the foundational problem in all of biomedical engineering. >> The underlying data used to train these models is almost never perfect. In fact, it's often a total disaster. >> The classic garbage in, garbage out problem. >> Worse than that, honestly. The creator of Beyond the Chaos posted this amazing, deeply frustrated commentary on their YouTube community tab about this exact issue. >> Everyone, you know, investors, the media, patients, they all want the developer to push the big red button that says develop life-saving product now. >> Right. We all want the Tony Stark montage where the AI just perfectly renders the cure in a glowing 3D model. >> Yeah, but the reality they point out is that before you can even run a basic algorithm, you have to spend six painful months acting as a digital janitor. >> A digital janitor? >> Yes. You are cleaning a data set that is full of inconsistent formatting, missing values, and like duplicate records from 2013 that someone forgot to delete. >> So, building medical AI is less like Tony Stark inventing in his pristine lab, and more like a janitor cleaning out a hoarder's digital garage from a decade ago. >> I'd argue it's actually harder than cleaning a garage because a garage doesn't handily lie to you. >> If you don't clean healthcare data properly, it will mathematically trick you. >> Wow. >> The creator's exact sentiment on this is perfect. They said, "Great products aren't built on clever code. They're built on clean, reliable, boring data." >> You cannot machine learn your way out of a data dumpster fire. >> And they have a visual on the platform that proves this, right? >> Sorcerer doesn't. I was looking at this and my jaw just dropped. >> Can you explain this to the listener because it completely shattered my trust in basic statist >> Yes, so with Testing This Tools, everything we do is available on bunnykills.com. Go check it out. Uh provide your feedback. So, there's a human in the loop if you have any questions. Like, subscribe, you know what to do.