Video summary
The podcast session focuses on the evolution of Pedagogical Agents (PALs) from isolated, one-on-one desktop environments to dynamic classroom settings. Traditionally, these systems were criticized for being isolating because they relied on a strict individual ratio where a single user interacted with an agent on a personal computer. The discussion highlights the significant challenge of transitioning PALs into chaotic classrooms filled with multiple students and activities. In this new context, the role of the teacher is redefined not as a replacement by technology, but as the central "pedagogy module" who utilizes data collected by the system to make informed instructional decisions. While traditional one-on-one systems allow for deep individual control over content, classroom-based PALs shift focus toward collecting real-time process-level data through cameras and sensors, enabling the tracking of cognitive engagement, peer interactions, and note-taking behaviors across the entire group.
To address concerns about teacher displacement, particularly among veteran educators who might feel threatened by AI capabilities, the dialogue emphasizes that technology serves as a supportive tool rather than a substitute for human expertise. Experienced teachers possess deep insights into student motivation, background, and social dynamics that current AI systems cannot fully replicate; therefore, the system is pitched as an assistant that highlights data points the teacher might otherwise miss, such as students who are struggling or disengaged. By positioning the teacher as the "final boss" or decision-maker who interprets this aggregated data to determine remedial actions or teaching strategies, the technology aims to reduce the cognitive load on educators while enhancing their ability to provide personalized feedback. This approach ensures that even teachers with decades of experience feel empowered rather than obsolete, as the system acts as a second set of eyes that complements human intuition without attempting to replace it.
The conversation also explores the potential and limitations of Virtual Reality Learning Environments (VRLE) in educational settings, specifically for middle and high school students. While VR offers immersive experiences, significant barriers exist regarding cost and physical comfort; devices like the Apple Vision Pro are prohibitively expensive for widespread classroom use, and even cheaper headsets can cause nausea due to sensory conflicts when users move without their bodies physically moving. Beyond these practical constraints, the transcript delves into how learning is measured within VR through detailed log recordings of millisecond-level actions, such as joystick movements and object interactions like grabbing magnets or changing coils in a physics experiment. By analyzing sequences of these actions, educators can distinguish between random trial-and-error navigation and meaningful experimental strategies, allowing the system to classify effective learning behaviors and provide targeted feedback based on the student's specific interaction patterns.
In conclusion, the integration of PALs and VR technologies into education requires a balanced approach that leverages data collection to support rather than replace human teachers. The ultimate goal is to create a hybrid ecosystem where the teacher acts as the decision-maker who synthesizes vast amounts of behavioral and cognitive data provided by the system to guide instruction. For VR specifically, while the technology holds promise for immersive learning, its adoption must account for hardware costs and user comfort issues while utilizing advanced action recognition to validate genuine learning outcomes. The podcast reinforces that successful implementation depends on convincing educators that these tools enhance their capabilities by automating data gathering and assessment, thereby freeing them to focus on higher-order teaching tasks like motivating students and addressing complex emotional needs that machines cannot yet understand.
Read the full video transcript
Welcome back for this week's podcast. We
have Amit with us for the podcast
questions.
>> Hello sir. Uh thank you for again
inviting me as a host uh for the podcast
session. Uh so
uh welcome back everyone again. So for
years intelligent tutoring systems were
criticized for being isolating. Uh
meaning that uh the system is designed
for a single person uh use right uh so a
person is staring at desktop and the PAL
works on the desktop itself. So uh today
we will discuss the PALS out of that uh
uh rectangle uh RCBLE or our computer
desktops and then we are taking it to
the classroom.
So uh we know then that the traditional
paradigm uh of the pedagogical agent is
built on strict one-on-one ratio right.
>> Yeah. So when we take away that isolated
desktop environment and throw it uh into
a classroom setting okay and classroom
has lot of chaos people are sitting then
few people are like discussing lot of
things so there are so many things are
happening and then there is one
instructor or teacher is present okay so
uh how do you see that transition of pal
from one-on-one to one to many also I
want to add one more point to is what
will happen to the teacher?
>> Okay,
interesting question. I think we
discussed this in this week. Um
when we are saying pack for the
classroom,
we are considering teacher as the
pedagogy module.
>> Okay. The learners are there. The
learner module can be collected for
learners. The domain model a teacher can
have from their own you know knowledge
or teacher can use domain model from the
computer know from the from the vast
internet they could collect the data
they can curate and keep it use lm to
generate content that's not a problem so
here the teacher is not removed I think
I'm answer the second question first
okay um in a classroom
we might not able to collect data which
we can collect in the one-on-one system
where you know it's how do we say it's
like they have their own uh agent built
on the system. They try to understand uh
we try to understand the content. They
have their own control on the content.
They learn they can do everything. That
is true. In a classroom that is not
possible but we could collect the
process level data in real time in the
classroom. How? We may not have the
clickream data but we have the camera
data in the classroom. We could track
the objects that is track all the
students, identify students in the
classroom, track them, identify them. We
could record their cognitive engagement.
Are they taking notes passively engaged?
Are they, you know, constructing
something? Are they interacting with the
peer correctly? Are they not knowing the
help? You could you could track each and
every individual separately. That data
can given to the teacher
>> and then the teacher can use the data
abstract all students data each
individual data students can look at it
then the student can interact. So stu
the teacher is the pedagogy module
>> right
>> the data from all the students and ind
students shown in the dashboard become
the learner model which is stored domain
model can come from the internet or the
teachers idea. So yeah, so we are not
replacing teachers, we making teachers
as the uh decision maker where u where
the system is helping with the data
collection and uh content.
>> Okay. So sir uh I have one kind of a
different question considering a human
uh element. Okay. So suppose there is a
a teacher uh who is very old suppose 50
50 years 52 years old okay and he has
been teaching that one particular
lecture or that uh two to three subjects
for more than 20 plus years right and
all of sudden the school tries to
implement the PAL system or like with
webcam based uh uh systems right uh and
he feels that AI cannot read my
classroom room and understand my
students uh emotional needs better than
I do. Right? So uh to cater to the
teacher
>> uh how uh we can develop a PAL system or
IT system
>> that uh that particular professor won't
feel threatened by the PAL system or any
AI system.
>> Okay. I think uh is it connects back the
previous question answer also. So right
uh the teacher is better than it which
means he has been huge experience and
teacher is kind of know already knows
how to teach 20 years of experience is
huge. So convincing them hey don't worry
we have a can detect when the students
fails we can provide a content to them
you should be involved only when they
needed it's not not easy. So in a
classroom kind of setting it's very
simple. We could simply collect the data
in the teacher's classroom. Later we can
show hey these are things happen in your
classroom. How much you noticed? How
much you did not notice. As a
experienced teacher they quickly notice
everything a pal can show but they can't
be looking at the students all the time.
>> Right?
>> So we could pitch them saying that you
will be writing in a board. You know
you've been 20 years teaching or using a
slide if they have moved to the slides.
uh we could collect all the data of all
the students in a one single set then
ask the teacher to make a decision this
system don't worry about decision or
anything let the teacher be the decision
maker
>> so then teacher is not left out and they
can do we talked about this uh thing
before also that in a in a
self-regulated learning and any theory
you could see that um the people have a
cognition effect meta cognition
motivation. We have a systems to deduct
you know cognition effect of
metacognition but there's no system to
understand students motivation on a
particular skill
>> to understand the skill even for the
experienced teacher they need to know
the student for know at least for 6
months understand students background
the family uh you know their social
circle the friends in the classroom how
he's performing other classrooms to
understand the students motivation on
every action so a teacher is the only
person can give feedback
He cannot replace a teacher. So we have
to explain maybe with this concept to
teacher. One we need to show that you
are good at your work but our data can
give also that work. So your work can be
reduced. Second we have to show that you
are not replaced. We are giving data to
you are the decision maker. You are the
final boss. You should do it. Then the
teacher will try to take it. That's the
idea. But if the teacher is not
interested in a one-on-one pal then we
have to say it's not teaching you teach
the content use the pal as the test
taker assessing the learnage
>> right
>> so as the you teach everything now pal
assess the knowledge gives a complete
data feedback
>> now you take a decision other I should
go for remedial content should I teach
how to teach it that is also good
>> okay so you mean that this PAL system
will work uh twofolds
It will also help the learners and uh
getting the feedback from the students
and showing it to the teachers uh maybe
veteran uh so they will get uh what are
the exact pain points or different
points and it will actually assist the
>> uh teachers right?
>> Yes.
>> Oh that's a good tech. Uh again sir I
want to ask about the VRls right?
>> Yeah. So uh it's very fascinating
someone mounting that VR headset and
doing lot of things. Okay. But uh all
the good things that we see comes along
with some limitations. Right.
>> So first uh do you want to like uh uh
shed a light on what are what could be
the limitation of using VRLE u in
suppose um 8th 9th and 10th grade. Okay.
Uh I I think there's a big limitation I
should talk about this. The technology
is so much advanced. VD we could
interact immerse uh fantastically. But
that's a limitation such that even with
the latest technology you could use the
the meta quest or the normal VR less
normal sense the one which is not really
costly like Apple Vision Pro for 10 to
15 minutes. Okay. That's a one thing and
a lot of people have um felt nausea when
they they wear the VR when they when
they see they are sitting in the V
element from driving the car um it's not
that I am also moving also the side
everything the the thing moves which I
without my body moving everything moves
actually it's body is not used to it
they feel nausea
um for some of the games also there uh
so that is the first limitation We can't
keep everything in V.
>> Second the as I said if the good
immersive VR environment comes with a
huge cost say minimum 35 to 50,000
>> I can practically go to the classroom
and do it and already lot of schools and
lot of colleges have the VR but they
just see it
>> they don't use it they won't create
content it's because
>> they don't have enough for every
student. M
>> so yeah the VR is a good one but other
than these two limitation not think
quickly the other limitations. Yeah.
>> Okay. Okay. So uh in continuation with
VRLE is uh so when analyzing the
interaction behavior in VR we see that
log recordings
uh are like uh we are recording from the
milliseconds
uh and then uh we have some actions as
well right like grab action grip action
uh and then uh the person is
particularly in that we are really
showed by Sorry. PRL is shown by
Anthony. Uh so there is two turn coil
experiment. There could be multiple
experiments like that. Right? So uh a
person is doing lot of things lot of
data can be collected with that. Uh so u
can you can you tell me that if a person
is in VR and if he's doing lot of random
things
are we really able to understand the
learning that is happening inside that
maybe few people are only doing trial
and error basis or experimentation right
>> so how to justify all these actions uh
and come up with that Yes. Uh one
particular person has uh learned
something from the VR something
>> cool. I think I think Anthony tried to
show um how do we collect data in the
virtual reality environment. So as in
the any computer based learning
environment student also interacting
with children but maybe with the know
different kind of interaction.
Um so they have a controller you know
the press the joystick everything they
control the thumb everything is possible
we have collected every movement in the
uh you know joysticks
>> you know the actions
>> right
>> and we map to the student is in this
experiment the student is turning two
turn coil or four turn coil or six turn
coil how much uh quickly they grabbing
the magnet and dropping it or the iron
or how how much they're quickly moving
it you know to find those theirh you
know the falling coil principle and
everything so we collected this data now
we look at it which is the which is the
good I said which is a good strategy
which is not a good strategy between
trial and error we kind of consider as a
uh in the computer based learning model
a bottom layer we collected these
actions then we try to combine and make
a meaningful action for example if a
student change the coil to the two-ton
Next immediate action is grabbing the
magnet and moving and looking at the
graph.
>> If I have a three action in sequence I
mean there's a meaningful action
performed students. For example, student
change the coin to check the speed. Now
he goes back change the coil to four
checks it.
>> So now student is experimenting the
coil. If I know the sequence of actions
which is record in the VR then I
classifies of actions as the you know
good strategy. Whereas somewhere student
is simply um not that first time going
around is good after the game simply
moving to other steps sitting there come
back and it's not spending any
meaningful actions
lot of navigations
>> then I would say it's not there then we
could continue this you know
understanding of the learner model then
provide some feedback so that's the idea
>> okay thank you thank you sir for this uh
podcast session
Uh I think whatever uh the questions can
come from this week n I have tried to
get like answers from the sir but again
if you have few questions post it on
forum we try to resolve all those uh
questions. Uh thank you thank you for
joining us.
>> Yeah thank you Amit and thank you
everyone.
>> Thank you. Thank you.