Video summary
The video introduces an innovative interactive tool called the Phonetic Simulator, which visualizes the complex biomechanics of human speech synthesis in real-time. This application allows users to manually control and observe the movement of anatomical structures such as the tongue, jaw, teeth, and lips, effectively turning the screen into a flight simulator for piloting a human vocal tract. By manipulating sliders that adjust parameters like jaw depression and tongue height, viewers can witness how physical changes in the oral cavity directly alter the generated sound, bridging the gap between abstract audio output and tangible biological motion.
At the core of this simulation is an explanation of acoustic filtering theory, where the vocal tract acts as a resonant chamber similar to a flute rather than a vibrating string instrument. The tool demonstrates how specific sounds are produced by changing the volume and shape of air cavities; for instance, raising the tongue lowers the first formant while pushing it forward affects the second formant. The interface includes an integrated IPA vowel quadrilateral that maps these movements geometrically, tracking the highest point of the tongue to show exactly which sound is being articulated based on the precise positioning of the anatomy within the simulated throat and mouth.
Beyond passive observation, the simulator offers active engagement by allowing users to type custom sentences or manually drag the pink tongue body to perform articulatory gymnastics. The system breaks down complex words like "bionic chaos" into their constituent phonetic parts, illustrating events such as bilabial bursts for 'b' and velopharyngeal port openings for nasal sounds. While the current version focuses primarily on visualizing the kinematics of the mouth and tongue rather than perfecting synthetic audio quality, it successfully uses built-in browser text-to-speech to play back sentences, enabling users to hear how their manual adjustments influence the resulting acoustic output.
The video concludes by pondering the future implications of this technology, suggesting that digital voices may soon evolve from simple audio files into entities literally "spoken" by virtual lungs and vocal tracts. By simulating the precise physics of biological speech production, the tool provides a unique educational platform for understanding how a combination of physical cavities and aerodynamic turbulence generates the entire spectrum of human language. The developers invite feedback on improving the audio clarity and pronunciation algorithms, emphasizing that this project is an ongoing experiment to better visualize the intricate mechanics behind every word we speak.
Read the full video transcript
Okay, we have this new tool on the
website. It's a bio acoustic
articulatory
phonetics and vocal track animatics.
It's a bit of a muff. Could have came up
with a better better name. Well, the
better name is phonetic simulator.
So, you can see
we can control this mouth and tongue. I
don't know if it's clear that this is
mouth. So the teeth, the tongue, the jaw
should be pretty clear. If you have any
ideas how to improve this,
uh do let me know. You can control it
from the
from the control panel as well.
You can pronounce
certain words like bionic chaos.
It's not super clean, but this is just
um essentially building a a text to
speech from scratch. Yeah. When you have
the sound on, it keeps vibrating the
road. But also when you change the
when you change the position of the
tongue or the jaw, the mouth open and
close, it changes
the sound
as
your mouth does.
and we will do an overview by notebook
and lamb and let me know if you have any
questions or comments on it.
>> Right now there is a tornado of
pressurized air vibrating inside your
throat, bouncing off the wet walls of
your mouth and escaping your lips as
language. We rarely think about it, but
you are constantly operating a highly
complex biological wind instrument.
Welcome to this deep dive. Today, our
mission is to act as your audio guide
while you run the demo mode of the
English speech articulator and phonetics
visualizer over at bioniccast.com,
pronounced bionic chaos.
>> Yeah, we're basically decoding the
fascinating biomechanics of human speech
right alongside the simulation you're
watching.
>> Okay, let's unpack this. If you look at
the main visualizer canvas, it's well,
it's pretty mesmerizing. You've got this
pink tongue body moving in real time,
>> right? And you can see those blue
formant monitors tracking the
frequencies plus that uh labial view
showing the actual lip shapes.
>> Exactly. And this interface feels
exactly like a flight simulator but you
know instead of flying a Boeing you are
manually piloting a human mouth.
>> That is like the perfect analogy. And
that grid in the upper right the
integrated IPA vow quadrilateral. It's
basically a literal map of your oral
cavity.
>> Wait really I mean looking at it just
looks like a geometric map.
>> Well it is. The top edge is the roof of
your mouth. The bottom is your jaw
dropped. The left side represents your
front teeth. And the right side is your
throat.
>> Oh, so when that yellow IPA cursor moves
around, it's literally just tracking
where the highest point of your tongue
is.
>> Yeah, you got it. It's tracking exactly
what the anatomy is doing to create a
specific sound.
>> Okay. But if the canvas is the
windshield, the sliders on the right are
the cockpit controls, right? Like jaw
depression and tongue dorsome height.
>> Exactly. and tongue backness too. Moving
those is what actually changes the
physical shape of the mouth.
>> But how do these physical movements
actually create the sounds? Is it like
pressing frets on a guitar string?
>> Well, it's actually much closer to
blowing into a flute. You aren't
shortening a vibrating string. You're
changing the volume of a resonant
chamber.
>> Oh, right. This goes back to Gunnar
Fance acoustic theory of speech
production, doesn't it?
>> Yeah, exactly. Speech is really just
acoustic filtering applied to
aerodynamic turbulence. Your glauus is
the source just vibrating away in your
throat.
>> And then that 17.5 cm vocal tract acts
as a complex acoustic tube acting as the
filter.
>> Exactly. When you move those sliders,
you're physically squeezing and
expanding different sections of that
tube.
>> Here's where it gets really interesting.
Wait, so the mysterious formance like F1
and F2, they aren't just abstract math.
F1 is directly controlled by how high my
tongue is.
>> Yep. They are intensely physical. And F2
is just well how far back it sits in
your mouth.
>> Oh wow. So it's like blowing over a
glass bottle, right? A smaller volume of
air means a higher pitch.
>> You nailed the physics. Dropping your
jaw opens the front cavity which raises
F1. Pushing your tongue forward shrinks
that front cavity raising F2.
>> Okay, so let's connect that physical
theory to the demo mode you're watching
where the graphine to foam engine
orchestrates these exact movements.
>> Yeah, it translates text into precise
physical kinematics. It's wild to watch.
>> Watch how the tool breaks down the
custom word bionic chaos. You see the
complete kinematic sequence like the
bilabial burst for the bee. Right at the
start.
>> Yeah. The lips slam completely shut. And
then uh pay attention to the N where the
veloperial port opens.
>> Right. Which routes the acoustic energy
right through the nose.
>> Exactly. And then for the S, you get
that high velocity airream directly
across the incizers. What's fascinating
here is well, you don't just have to
watch the demo run.
>> No, totally. You can actively click and
drag the pink tongue body manually or
type custom sentences to watch these
complex articuly gymnastics unfold in
real time.
>> We've basically just explored how a
simple combination of physical cavities
and a 17.5 cm acoustic tube generates
the entire spectrum of human language.
>> It really is incredible when you break
down the actual physics of it all.
>> It really is. So leave you with this
final thought to ponder while you keep
playing with the demo. If an interactive
module can perfectly synthesize speech
by simulating the precise physics of
physical lips, tongues, and pallets.
>> Yeah, literally simulating the biology,
>> right? How long until the digital voices
we interact with daily aren't just audio
files, but are literally being spoken by
a virtual set of digital lungs?
>> Okay, this was not a lamb overview. 4
minutes, 4 and a half minutes. I'm not
sure if it did well on showcasing the
tool in terms of the especially in terms
of the sound the synthetic sound
activation.
So playing with our sales.
So this is it.
I guess that's so that's it pronouncing
bionic chaos if that wasn't clear. There
is a description at the bottom of the
page. Let's take this uh paragraph.
Pop it. If you pop it into this text
box, it will pronounce
Okay, that was difficult to understand.
So, currently it's just focusing on how
the tongue and the mouth open, close,
the position of the mouth and the
tongue.
Um, we will we could try improve the
pronunciation.
>> Yeah, it's not stopping the sound
because the throat keeps vibrating for
sound generation.
Okay. So, you get the picture and
streaming the sentence will uh
essentially in this browser uh there's
no uh there's no text to speech built in
into the browser. If you're using Chrome
or in our browser that has a text to
speech, uh this uh stream sentence
button will activate that will repeat
whatever is in this text box using your
inbuilt uh
uh text to speech.
Yes, I think you get the picture of what
it does. It's available. Go check it out
on bony.com/fonetics
seam. It's for phonetics simulator. It's
also on the landing page
and we have many many other tools on the
landing page as well. So go check them
out as well
and I'll see you next time. Bye-bye.
trying to wave to like a
you can actually activate it with a
keyboard
or press the buttons here. Yeah. So,
check out this EMG simulator as well,
but