Video summary
The lecture focuses on transforming raw data collected from Virtual Reality Learning Environments (VRLE) into meaningful insights for personalization. While previous sessions covered the types of VR environments and methods for logging data, specifically within devices like the Meta Quest 2, this segment explains how to process that initial data. Raw inputs such as grip pressure, trigger states, and controller movements cannot directly provide personalized feedback; instead, they must be converted into constructed features that describe specific actions. For instance, a sequence of picking up an object, holding it for ten seconds, and dropping it is synthesized into a single feature representing "dragging," which carries more pedagogical value than isolated button presses.
To create a comprehensive picture of student behavior, the instructor demonstrates how to combine multiple action features into complex scenarios. By merging data points such as dragging a magnet through a coil, selecting different coils, and simultaneously looking at a graph via infrared ray tracking, the system can infer high-level learning activities like "exploring electromagnetic force." This approach allows the environment to recognize when a student is struggling or succeeding by analyzing patterns, such as a learner repeatedly trying different coils before consulting instructional notes. These synthesized features enable the VRLE to assess performance and provide remedial feedback similar to computer-based learning environments, utilizing models like BKT to measure skills.
The core methodology for personalization in VRLE mirrors that of traditional computer-based systems but requires an additional initial step: effective data logging. The process involves collecting raw interaction data, constructing meaningful features from those logs, and developing a learner model based on this information. Once the learner model is established, strategies such as pattern mining or rule-based systems can be applied to generate hints, scaffolding, and tailored content. The lecture concludes by emphasizing that while the pedagogical logic remains consistent with existing CBL environments, the unique challenge lies in implementing robust data logging mechanisms to capture the rich interaction data necessary for these advanced personalization techniques.
Read the full video transcript
[music]
[bell]
[bell and music]
>> Welcome back to designing eye driven
personalized adaptive and learning
environments course.
In last videos, Anthony Rogash has
explained what are different types of VR
environments and how to collect log data
in the one type of VR environment that
is 60 of freedom meta quest two.
So in this video we use how to use those
data which we collected to personalize
it. Before we go to personalization,
um the data we collect is called raw
data. We collected all the raw data and
since the grip, the trigger, the
pressure, you know, the left
left trigger or right controller in the
VR devices. But that is not
cannot be directly used to provide a
personalized you know feedback. To
provide personalized feedback, we need
to convert these features and construct
a feature from this so that those
constructed features can inform
something about actions they're
performing in the
uh environment. So they use the actions
to provide a personalized feedback. For
example,
uh controller and button actions, you
know, all the objects we consider we
take actions. For example, a student is
uh pressing the left grip and uh the
object is picked this magnet at the
timestamp uh 0.01.
Now the left controller is picked, the
grip button is also selected, also the
the magnet is picked. This can be
classified or converted into action
called magnet grabbed.
Now the
Now the left controller is there and the
grip button is still on and the magnet
is released. So now the magnet is
dropped.
Together magnet grabbed and magnet
dropped that is from the time 0 01 to 10
seconds. That's 10 seconds the time.
We can consider this as a magnet dragged
for 10 seconds.
By writing a Python script after
collecting the raw log data, we could
create a set of actions.
If you try to use this data like magnet
dropping, magnet drop dropping, you
won't get a meaningful information. You
have to construct features such that
for the last 10 seconds, the student
might be dragging the magnet. During the
magnet dragging, the student also
changed the
turn coil for two turn coil, you know,
pressing the button.
So, use those information, we could
combine the two coil turn coil selected.
Now, we can say that
the student actually dragged the magnet
and in the two turn coil selection and
dragged for the 10 seconds. This is a
very meaningful information. Suppose a
student does it for four turn coil, then
tries for six turn coil, then the
student looks at the graph. We can make
a pattern sequence of pattern saying
that the student first exploring each
turn coil, how much the, you know,
electromagnetic force induced would
change in the particular, let's see,
scenario. Those data is useful to
provide a personalized learning content.
Similarly, what we could see is that
whenever the student's looking at the
graph using the invisible infrared ray
which Anthony talked about in the
previous video, we could say that the
student is looking at the graph or not.
So, whenever the student looks ahead
ahead and they're looking at the the
graph,
the infrared ray will be displayed on
the particular object called graph
object. Whenever they're noticing the
object that is a graph,
the time can be notified. We can say the
student's inferring about the graph.
Similarly, when the student is looking
at the object of instruction board or
the some notes were given to them, we
can say student is reading the
instruction.
So, that's called inferring the graph
for the four seconds. So, this is very
important to capture these data. So,
now,
if you combine all three actions,
a student has dragged the map for 10
seconds. In the 10 seconds, student is
standing coil for a couple of seconds.
During turning the magnet, dragging the
magnet in the coil, he's also looking at
the graph.
So, by combining all this information,
we could say that the student is
exploring
the electromagnetic force by looking at
the graph and dragging. Maybe you would
change the you know, the direction of
the magnet, change the turn coil. All
these things are possible to combine
uh this information to provide a
personalized feedback.
So, what we call is a parameter
manipulation
to the experimentation to evaluating the
student actually manipulate the
parameter, experiment using the magnet
coil, and evaluating its performance
using looking at the graph.
Consider you have collected all the raw
data, then you have constructed the
features. The features like
grabbing the magnet, inferring the
logic. The features like the student is
navigating, is looking at the
instructional material.
Student is trying to grab something, is
not understanding, going back to
instruction material. If you have these
kind of features coming into picture,
then it is same as the computer-based
learning environment where you have
collected a data such as sequence of
actions. Use the sequence of actions to
provide a feedback.
You could also have a assessment.
Student do perform some actual task. You
can measure the performance. You can
provide the skill measurement like a BKT
within the VRLE. It's all possible. The
only thing stopping us
to use personalization VRLE is the data
logging mechanism. That is what we are
display discussing last two videos. I
think if you are interested in you know,
providing personalization in VRLE,
consider data logging using the
mechanism we provided. Then use the data
to provide a remedial feedback. So,
first thing is develop the learner model
using learner's interaction data. That
is we collect the data, we construct
features, and develop the learner model.
Once you have the learner model, you
could have a pattern mining or some
strategies created from this. By using
these strategies or constraints or the
rule based systems, we could create a
hints, feedbacks or remedial content for
scaffolding.
So, the steps are exactly same as
computer based learning environment.
Only the first step should be that how
to log the
interaction data in the VR early. I hope
this
two videos which Anthony talked about
and would help you to understand how to
log data in the VR early.
Then you could start with the pedagogy
logic which we discussed in the CBL
environment.
I hope it is clear
in this videos how to personalized
learning environment or learning content
in the virtual reality learning
environments.
So,
in this particular videos, the last two
videos, is to collect log data.
So, we have written many papers and we
have listed three papers here. These
papers would help you to understand more
in detail how to log data, how to
understand students, you know, patterns
occurring during the learning
environment in the VR early. How
students
patterns differ when the high versus low
performing learners.
Which particular interactions is more
impacting the learners outcome. All
those things has been discussed in these
papers. So, these
papers are detailing whatever that we
discussed in last two videos. These are
the sources for you. If you are
interested in more understanding more
about this, please I request you to go
and read these papers.
Thank you.
>> [music]
[music]