Video summary
The video introduces an innovative sign language translation glove developed as a hackware project in August 2026, designed to bridge the communication gap between deaf individuals and non-deaf people using American Sign Language (ASL). The team faced significant constraints, including only two days to build the prototype, yet they successfully created a functional device that translates spoken text into speech for hearing users while simultaneously interpreting sign language gestures into audible descriptions. To achieve this without overburdening the wearer with excessive weight or bulk, the hardware was strategically divided between an open-ring glove and a forearm case; the glove houses five flex sensors to track finger bending and one IMU for hand orientation, whereas the forearm unit contains a Raspberry Pi for processing data, an ESP32 for analog input handling, additional microphones, speakers, a display screen, and another IMU to monitor overall arm movement.
The development process involved overcoming substantial engineering challenges related to space management and component integration within such a compact form factor. Initially, the team considered placing all electronics directly on the hand but abandoned this idea because it was too cramped and uncomfortable for users wearing compression sleeves or gloves; instead, they opted for open rings that are easier to wear and less hot than fully enclosed designs. The software side relied on training data provided by a teammate who knew sign language, utilizing a random forest model due to limited dataset availability, with plans to upgrade to more complex AI models in the future. A key design decision was to keep all processing units within the device rather than relying on an external smartphone connection; this ensured that users could simply turn on the system and use it immediately without needing to carry or connect a phone during communication sessions, thereby enhancing accessibility and inclusivity.
Looking beyond the immediate prototype, the creator envisions future improvements driven by artificial intelligence to simplify both hardware design and software development workflows. The speaker highlights how AI agents can be integrated with open-source CAD tools like KiCad to allow users to plan circuits via natural language commands, effectively automating repetitive tasks in hardware creation that are currently difficult for humans to manage alone. Additionally, the goal is to develop systems capable of understanding context better by integrating directly with various applications, which would provide tailored suggestions on what skills or components a user needs to learn next rather than leaving them to figure everything out independently. This forward-thinking approach aims not only to refine the current glove's capabilities but also to democratize hardware design through intelligent automation and contextual learning assistance.
Read the full video transcript
Essentially what this is is a sign
language translation glove. So ASL is a
language that deaf people use to
communicate. And so we wanted to build
something that could actually allow
people to use to communicate like de
people to like non-deaf people. So
that's why we built this glove and it
actually worked. We only had like two
days to build it. So our whole team was
very proud when we finally managed to
make it. And I can talk a bit about how
it worked right now.
No.
All right. So, this is essentially the
hardware control flow that went into
making the ASL gloves. So, we had
something that was on the glove like
actually on the hand and something that
was on the forearm like as the photo
showed. Uh, initially we were thinking
of placing everything on the glove, but
then it just like too much space just
like everything was too crammed in.
Plus, it would be too heavy for the
person wearing it. So, we decided to
separate it into like rings that people
could wear on on the on the hand and
also a forearm thing. uh so we could
store a lot more circuitry especially
since we were using offtheshelf
components we weren't actually creating
our own PCBs yet uh so essentially on
the glove we had five flex sensors which
measured how much the fingers bent uh as
well as one IMU which could detect the
orientation of the hand um and on the
forearm we had uh a Raspberry Pi which
which process everything and also an ESP
which allowed us to take all the analog
inputs uh I know someone was from ESP
yes
>> uh and we also had a microphone uh
speakers to actually display and then a
display screen and another IMU on the
forearm so it could not only track this
motion but also like the whole
orientation of the arm. One big
limitation is we couldn't see the face
but uh but we we had to make do. Uh and
essentially what this allowed is so so
someone could talk and it'll translate
that text into speech uh that speech
into text which would be displayed on
the forum. Uh and also it would speak
out what the person was doing via sign
language.
Uh and on the software side we
essentially just trained a bunch of
different commands. Uh we we had someone
in our team or who actually spoke sign
language. So that's that's how they
train the model. Uh we used we we
couldn't get that we couldn't get that
much data so we used a random forest
model. Uh but in the future we want to
we going to upgrade to more complex like
AI models but right now this is just
what we did.
Uh for the CAD, uh initially we were
thinking of something like that where it
would be on your hand, but that was
really awkward to place on uh especially
since we were using like a you know how
people have those compression sleeves.
So that's how we were attaching the the
the case onto our arm. Uh so we decided
to model it after someone's forearm. Uh
so we could easily place fit on really
really easily. Uh and as to you know the
the software uh programming pretty uh
pretty uh generic stuff uh uh common
stuff uh but the planning of all the
components was like really hard because
we had to fit it in such a small space
and we had so many different inputs and
outputs that uh it made organizing
everything like a nightmare but luckily
we managed to figure it out uh which
took a lot of time. Another question. So
once you have the SP30 once the fax
sensors and the orientation equipped,
why do you need the four AR files? Like
one would think that you could just
connect to your phone and run the rest
of the pipeline of the phone and then
you don't need the extra ars.
>> What we wanted was for the person not to
have to be able to take out their phone
when they were using it. So like it'd be
fully enclosed. Uh so like you'd have to
set it up, you'd have to connect each
time versus like just turning it on and
using it over there is sort of the
reasoning that we went behind. Uh we
were actually thinking of outsort like
all the we were thinking of another way
which is keeping the microphone uh and u
the microphone on the
on the on a phone. So you give the for
the phones the like the hearing person
uh and and the love is just to the deaf
person which would make everything a lot
easier but essentially we went we went
the other way because we wanted it to be
completely inclusive.
>> Uh
so this is uh sort of how we iterated
through it. Initially, we just took one
of the the soldering iron gloves and we
placed everything on. Uh, and as you can
see, I mean, this is not even half the
components and they didn't really fit.
So, we had to move on to having a
forearm case. Uh, and we chose open
rings instead of instead of a full close
gloves just because it' be easier to
wear and also be like less hot on the
hand. So, that's why we chose these like
open case rings. And obviously we were
going for something like that, but
obviously like it wasn't it's a
prototype. So uh and we didn't fully
enclose the flex sensors yet, but you
know in the future that's what we plan
on doing.
>> And this is a sort of demo still works.
And just just a quick note, the speaker
was connected to the ESP, not the
Raspberry Pi, because we didn't have a
Pi speaker on on hand. So the audio
quality was a bit bad.
It's playing through my
Oh, I think the the audio is just
playing through my computer instead.
It said hot milk.
>> Hi, my name is Hershey.
translating a thing.
>> I'm good. What about you?
>> So that's essentially uh how it worked.
And also just a quick other thing is
that from this it's like a sort of
another like idea I was I was thinking
about was that there was a lot of stuff
throughout the entire uh throughout the
entire building the whole glove that I
realized could be simplified uh with
with uh like an AI system like because
you have a bunch of AI and agentic
systems for code for software but on the
hardware side it's not as much. So I was
I also started to create a bit of a
solution to sort of mitigate this. And
what this is is basically it's a I took
an open- source CAD uh software and in
and uh connected an agent to it. So you
can CAD via via text and natural
language. Same thing with uh Keycad
because that's already open source. So
uh I connect another agent to that. What
I'm trying to do right now uh is have a
way to eliminate a lot of the the sort
of repetitive stuff with creating
hardware uh with hardware projects uh
and also the the stuff that is that
could be taken away through external
agents uh through this solution. So I I
this is a very brief thought but this is
sort of what I'm trying to do uh right
now. Uh and there's two two different
ways I'm thinking about this. one is you
know I a few friends said that plan the
planning stage of hardware is is really
really hard uh because you just it just
you you know what the end goal is but
you don't know how to start so that's
something we're trying to do uh and also
a second thing which I actually spoke to
I think with side was actually
integrating with these apps uh so it
could understand the context a lot
better uh and therefore uh pose like
suggestions to what what you need to
learn rather than you having to figure
that all out by yourself. So yeah,
that's what she's getting.