Video summary
The video introduces the concept of data logging within Virtual Reality Learning Environments (VRLEs), emphasizing its critical role in facilitating personalized and adaptive learning experiences. Since VR systems rely on head-mounted displays and handheld controllers for user immersion, these devices generate a wealth of interaction data that can be systematically collected. By capturing every action performed through the virtual reality system, educators and developers can model learner behavior to better understand the learning process. This comprehensive data collection is essential for tailoring educational content to individual student needs, ensuring that the learning environment responds dynamically to user actions and preferences.
To implement this logging mechanism, the lecture outlines a specific logical framework designed for tethered VR systems, often utilizing game engines like Unity with the Virtual Reality Toolkit. The system first checks for an existing report folder and CSV file on the computer, creating them automatically if they are missing. Once established, the logic monitors inputs from handheld controllers, recording detailed information such as whether the left or right controller is used, which specific buttons are pressed (e.g., grip, trigger, thumbstick), the state of those buttons, applied pressure levels, and the precise XYZ axis movements of the thumbstick. Additionally, the system logs interactions with virtual objects, ensuring that every physical manipulation within the digital space is timestamped and stored for later analysis.
Beyond manual controller inputs, the video explains how to capture "noticing actions," which refer to what a learner looks at or focuses on within the environment. This is achieved through ray casting techniques attached to the main camera of the head-mounted display in the Unity engine. An invisible ray is projected from the camera; whenever a learner turns their head to view an object, this ray collides with the corresponding VR object, triggering a log entry that records the specific object being observed along with the exact timestamp. This method allows researchers to track visual attention and engagement patterns without requiring explicit input from the user, providing deep insights into how learners navigate and interact with complex virtual scenarios.
In conclusion, the session successfully details the operationalization of data logging in VRLEs, covering both the technical implementation using C# scripts and game engines and the specific types of data generated by controller interactions and visual attention. The lecture presents an excerpt of the resulting CSV data to illustrate how button actions, object interactions, and gaze tracking are consolidated into a structured format for analysis. By understanding these logging mechanisms, developers can effectively gather the necessary metrics to support adaptive learning strategies. The video concludes by previewing the next session, which will focus on extracting meaningful insights from the logged data to further enhance the virtual reality learning experience.
Read the full video transcript
[music]
[bell]
[bell and music]
>> Welcome back.
In the previous session, we discussed
about virtual reality system, different
types of virtual reality systems, and
the softwares through which the virtual
reality contents can be developed.
In this video, we will discuss what are
the how the data logging can be done in
virtual reality learning environment.
You can pause this video and answer this
question.
VRLs are accessed using VR devices
comprising of head-mounted display and
handheld controllers. Now, think for a
while, what data can be collected to
enable personalized and adaptive
learning in virtual reality learning
environment?
You might have answers for the activity
questions that we discussed. Here, we
need to collect the data that are
suitable to provide personalized and
adaptive learning in VRLE.
For that, we have to collect all the
interactions happening through the
virtual reality system. Here,
this is the head-mounted display which
the learner wears and get immersed into
the environment.
And these are the handheld controllers.
One is right handheld controller,
another one is left handheld controller.
There are various buttons in the
controller such as grip button, trigger
button, then thumbstick is there, other
controller buttons are there. So, grip
button, the learners use this button to
grab the objects, trigger button to
select the options in the interface, VR
interface, and thumbstick they use to
teleport or to
move from one point to another point.
So, capturing all these actions
happening through these buttons
and happening through these controllers,
we can
model the learner's behavior, which will
be helpful to understand the learning
process of the learners, through which
personalized and adaptive learning can
be done.
How to do data logging in virtual
reality?
Here is a logic for
logging data in a tethered virtual
reality system.
The logic checks if there is a report
folder already available in the
computer. If it is not available, it is
created. And also it checks if
the file in which the data has to be
appended is available. If not, it
creates.
This logic is created using C sharp
script using game engine used to develop
the virtual reality content, for
example, Unity game engine.
Next, to log the actions that are
happening through the buttons of the
handheld controller, we use this logic.
We use the package virtual reality
toolkit.
And that gives the information such as
controller used, whether it is a left
controller or right controller, the
button used, whether it is a grip button
or trigger button or thumb stick, then
the button action, whether the button
has been pressed or released, the
pressure on the button, the thumb stick
axis in XYZ.
All this information are appended to the
report CSV file, which was created using
the previous logic.
Also, the virtual object interacted
using the handheld controller also
logged, including with the timestamp.
We also need to log the noticing action
of the learners.
To log the noticing action, we use ray
casting technique, in which the main
camera
which is the
head-mounted display. Head-mounted
display is named as main camera in the
Unity game engine. To the main camera,
we attach an invisible ray. Whenever the
learner turns his head to see any
object, the invisible ray, it comes in
collision with the VR object. That
triggers the logging, and that also
logged into the CSV file along with the
timestamp.
Here is an excerpt of data logged using
the data logging mechanism, giving
information related to the controller
button and the action and the object
interacted.
In this video, we have seen what is data
logging mechanism, how data logging
mechanism can be operationalized, and
what are the data logged in virtual
reality learning environment. In the
next session, we will see how to extract
actions from the data that has been
logged. Thank you.
>> [music]
[music]