TAISIG Talks #33: AI meets Virtual Reality - Tilburg University

[music] >> Tilburg University and Mindlabs present TaSIC Talks, [music] the podcast that keeps you updated on the latest developments in the field of artificial intelligence in just 1 [music] hour. >> Hello. Welcome to this edition of the TaSIC Talks, TaSIC the Tilburg Artificial [music] Intelligence Special Interest Group of Tilburg University. In these talks, we try to focus on artificial intelligence research done [music] at Tilburg University. We mostly talk about education topics, but today this is about research, and in particular I'm going [music] to talk about virtual reality and AI in virtual reality. >> [music] >> I have two guests here, Phil Brown and Evi Michaillidou. So, could two of you introduce yourself? >> Yeah, sure. So, my name is Phil Brown. I'm an assistant professor in the Department of Intelligent Systems, and I also help to co-run the Virtual Reality Lab at Mindlabs. Research interests are broad applications of virtual reality, often times in healthcare applications, but also quite interested in sensing using different sensors and different modalities as well. >> And my name is Evi Michaillidou. I'm an assistant professor at the Computational Cognitive Science Department. This is the Department of the Intelligent Systems, and my background is in virtual reality, extended reality, and multimodal sensing. I've worked for a few years before in the industry trying to develop actually some of the technologies used for sensing within virtual reality. And as Philip said, I'm also co-running the Virtual Reality Lab. So. >> Thank you. Yeah. So, when we talk about virtual reality, and we often other terms are brought in as well like augmented reality, mixed reality, immersive experiences. So, lots of terms which all relate to the same kind of topics. But, there is a a development over the last decades where I saw that um virtual reality was not really being used let's say 25 years ago and then it started to come up. And we got lots of enthusiasm and then evidently was a sort of a disappointment phase where people well and and then it spread out to other kinds of applications. So, maybe you can go over this this history a bit to set the scene and you can talk in whatever order you want. >> Shall I start? >> Yeah, please. >> [laughter] >> I guess we have a lot to say about this, but maybe we can highlight some of the key moments in the history of virtual reality because it's been I guess a history full of ambition and and prospects and lots of lots of funding and we managed to to get where we are today within 25 years or so. But, let's start from from the start. So, we have some prototypes like the sort of Damocles by Ivan Sutherland in I think in the 1960s and that really showcased what is possible, what can be done with virtual reality. And not so long after we start having a data glove, another very nice prototype that showed that you can actually touch in virtual reality before that was possible. And then we have advancement over an advancement and the reality is that for us we haven't been able to use this in in research at least when I was a researcher because it was used within very specific laboratories. It was very difficult to set up. So, in the in the 2010s is when we start having the first the virtual reality headsets. That's when we start having the Oculus. For example, when I was a master student, I remember we all chipped in so we can all buy one headset so we can start researching with it. And that was fascinating what we can do. Um then in the then later on as we can see from the Gartner's cycle of emerging uh technology, virtual reality had been for a while within the expectancy's and then we have the big hype towards the 2015 and 16. This is also the uh the hype of inflated expectations as we call it and it creates a big bubble and high ambition for everyone and we start having uh more and more headsets I think from a lot of companies uh which are big and and high-tech and so on and so on up until about 2020. I think that's when we start having the uh peak of the metaverse. Uh create a big uh sensation. That's another inflated expectation peak and we've learned a lot from uh overpromising and where the technology was. Um and now we are steadily outside of the emerging emerging tech cycle because uh the technology now is uh mainstream. It's out there. It has been developed and adopted uh by 170 million of people or so. >> Yeah, and continuing to be I think as well and you touched on it briefly there about 10 years ago with Oculus I think as well and when they made a really commercially available headset to get out of those um very expensive laboratory environments. Um and even if you take it back a little bit further I think to um what Peter mentioned at the start of this 25 years ago and then there was a bit of that disappointment. Um I I remember anyway growing up you had things like the the Virtual Boy and stuff like this which were things developed by you know in Nintendo for example, very um well-known companies in the field of gaming and I don't know if any of the listeners or anyone in this room has ever tried one of them they're absolutely horrible. You know, you've got about 30 frames a second, it makes you feel quite sick, and you want to take it off as soon as you've put it on. Um no, these issues still persist to this day, but if you really see that um the advancement and how we have maybe addressed these issues and how we're able to solve these issues partly because of the availability of this equipment. Um yeah, it's very interesting. >> But what was the hype then about? I mean, it's cool, but why was there a hype? What did people think they could do with it? >> Well, it's was a combination of things. We had better technologies, better sensors, we had more content, we had the people who were actually trained in virtual reality. They were engineers who worked on it, there was more funding being pushed, there was more commercially accessible uh headsets and sensing uh technology. So, all of a sudden, we had all the tools available to deliver some of the some of the visions we had in the past. So, for training, entertainment, and simulation for gaming, uh you name it. >> Yeah, I can but I can imagine that the hype was mostly that people look at that and think this is so cool, this is like the Matrix, and uh and and then I purchase that and I experience a whole new world, and then you get a couple of disappointments in the applications. I mean, I have been disappointed with virtual reality in games for instance where it also gives so many limitations besides the some aspects that also limitations but what you can do. While I can also imagine that um the serious applications of virtual reality, like you mentioned training, but uh um uh other application can imagine as well could be very useful, although that would not be a hype because that's not something that everybody then wants to have. Am am I looking at this in the correct way? And these serious applications, are they also a disappointment, or or what are good examples of that where VR is used? >> Yeah, um I think something to touch on later with some of the projects that we've been doing, but exactly that. Myself personally, um develop a lot of uh applications and with projects in healthcare domains, for example. So, how can we use these experiences, and quite successfully as well, I may say. There are many applications where traditional treatment, for example, may not be possible, and or may not be desired, and then VR you can actually offer an alternative to this. >> A treatment for what? >> Could be a number of things. Myself, for instance, we look at sort of prospective memory training. We look at maybe with um youths with learning disabilities, for example, or older adults pre-Alzheimer's in cases like that. So, how can you maybe offer something which they've either not used before, maybe they've not become accustomed to or interacted before, and offer an alternative a an alternative approach uh to that instead. But, take it back even a little bit further, for instance, when we talk about this 20 25 years ago, where this technology came from, the uh military has been very interested in this technology for a long time, not necessarily just virtual, but augmented mixed realities, which um you know, different modalities, same sort of principles that are applied there. Um and again, training applications, which maybe if you can talk a bit more about in a future, she's got a couple of projects running in that domain. Um again, very expensive um purposes, and then maybe how we can take it away to say flight simulations, we've had students who've graduated recently um from here who have done stuff with flight training, for example. Very expensive tasks, how can we maybe do these in a more um approachable manner, and a more sustainable manner as well. >> Yeah, I think it I if I may add as well, these uh the underlying principle of what virtual reality offers is why some of the visions that that we have initially might have been more disappointing, but ultimately, virtual reality is not just the medium, it's not just the technologies, it's not just a plugging thing that we could set up at any point. It's a whole system of interrelated nodes that manages to make an experience so highly immersive and simulated that and and controlled that we can use it for any kind of experimental processes and that's why it's so such a useful laboratory tool really. You can set up any kind of scenario and an idea, anything you want to explore, you can set it up safely and with high creative and controlled and high ecological validity within virtuality. >> Yeah. I can imagine that you can do that visually, but is it limited to visuals or should we look beyond that? I can especially when I think about augmented reality for instance where the actual reality should be there as well. Can you say something more about that? >> Of course. I mean for for us we started with mainly visual and motion activated interaction principles, but it's also the acoustic element. It's the haptics now that is more and more trendy. We have more consumer technologies available to us. We have olfactory induction devices so we can actually smell things. So if you can capture the majority of the senses, then you should be able to induce a more realistic, a more naturalistic scenario and in there we should have also more naturalistic responses by users. >> Yeah, and I can add to that maybe a little bit as well where sure we do often group uh virtual extended reality into visual experiences, but the benefit of that and maybe with some of the things you just mentioned there is that immersion, is that level of presence which these immersive technologies are able to offer. Yes, we could have maybe offered um, different types of training, or, um, medical conditions, or whatever it is, in a, uh, 2D display, for example, in a traditional sense. But, when you're able to actually have that immersive experience, and able to adopt someone into that instead, that's really sometimes where the benefit can be. How do you get to that immersive experience? Visuals, as I've just said, but then, you know, smell, sound, interactions, haptics, these all play a part of it. And, again, it sort of ties back to the start of this conversation a little bit, I think. That's where we've come from, where we have this primitive, so say, uh, technology, and how we're able to iterate upon this again and again, and really develop it into something now where, um, it's almost come full circle, I think, in a lot of ways. Uh, everyone saw this technology and thought, "Great, I can do anything with it." You have that level of disappointment then, oh, maybe I can't do everything with it. But, now as we're actually starting to develop it, people have had 10, 12, 13 years with these more commercially available headsets, we're learning how to actually work with this. We're learning what the pitfalls are of it, and what we can actually do to develop it further. >> Absolutely. And, and also it's some of the software that comes with it, right? APIs, SDKs, you name it. >> Yes, and I would agree, um, or argue even, that the software is something that has maybe been quite limiting with this for a long period of time. We have had large investment a lot of the time into the hardware of this, and many companies, from Samsung to Apple to Google, um, developing their own headsets. But, how much investment is actually made for the experiences, um, which people are going to be experiencing? Oftentimes, very expensive to develop. Um, where are you going to put your money maybe as a, uh, developer? Are you going to make it for the, um, for the movie, or for the Xbox, or whatever it is, which has X million number of people, or are you going to have it for that more niche technology potentially in virtual reality? Um, there's an argument there for what can be done for both, but an example maybe, once again, would be the Apple Vision Pro, which has just come along, very expensive headset, a very large niche in that regard. Although, you still do have major developers and may but major manufacturers behind it developing this content. That needs to continue and it needs to be sustainable so that people don't put their headset on and say, "Ah, it's the same two movies that I saw half a year ago." No, there's a constant stream of content and there's a constant stream of engagement there. >> Absolutely. And the what you mentioned about immersion and and presence, I think we need to re-emphasize that because it's a mental phenomenon that appears to be very prevalent in in virtual reality. And this the feeling of actually uh sensing as if you are in the room, as if you are in the virtual environment. Even though it's like Matrix, right? So, as you mentioned, so you you you plug in and you start experiencing this new environment. But it's safe and you can unplug at any point you want. You know it's fake. Right? So, so you know it's a fake environment, but somehow how people respond to the stimuli in the same way you they would have responded to a real stimulus environment. So, it's very interesting, a very intricate how this relationship between perception, cognition, response, emotional um perception actually affects your your responses and your experiential process within virtual reality. >> Yeah. Yeah, I can imagine that some of the disappointment is that that the walls is particularly in the area of the of game development because people then think, "Okay, we That's a big public, people who play games, and we can sell all that hardware, and then we can develop the games." But I think that game development that is what my experience is is very hard for VR, mostly not not even to make things 3D, but because in a game you often want to focus people's attention on a certain thing, and with a 2D monitor, it's much easier to focus attention on a specific thing than in a 3D world where people can look around everywhere. >> Yeah. Absolutely. And again, the hardware constraints that go along with that as well. Like you touched on it slightly there in that game environment, you can make a very pretty centerpiece for a game which if someone is going to be looking at that, but in a virtual experience 180 360° when you turn your head and it doesn't really look like it's supposed to, it's going to drag you out of that experience and you're going to say, "Ooh, that's not maybe how it's supposed to look." A game environment or something, you know, it's specifically manipulated to say, "Make this look pretty. Make this look nice." And again, it comes down to a cost and development cycle, I think, for this as well. >> Definitely. Graphics. Yeah. >> Yeah. You there was suggestion, but I don't know if you really meant that that that we have now reached basically a certain state static state-of-the-art, so that now the developments would be more in the software or are there still is there still developments in the hardware as well and then the and the the tools as well? >> Do you want to answer this, Salai? >> Go ahead. I can briefly mention, so I'll chuck it back over to you afterwards. I think this is constantly evolving and I take Apple as example once again with the Apple Vision Pro. This is their essentially version one which they've just released. I certainly hope they're not going to stop there, but this really is sort of ground zero for them and entering into this sort of a market. We got to a point where we thought, "Ooh, these developers aren't really coming out with much more. These hardware manufacturers aren't doing much else." Steam who uh created the Steam Deck people from gaming and you know, very popular platform online, just announced their new virtual reality headset. We're looking at extended reality goggles. Meta, you know, put millions and billions into this um throughout the years. >> Yeah, they did, but they now say, "Okay, we stop with that because it was uh it was a waste of money." Basically, that's what they're saying. >> Yeah, they're well, scaling it back anyway. Um and >> [clears throat] >> you could maybe uh have a whole conversation about how they've decided to implement this and uh the idea of the metaverse and how that's been portrayed to or received anyway. Um but specifically from a hardware to answer your question, then so I'll throw this back over to Iffy here. It's both absolutely. When we're talking about resolution with screens, when we're talking about field of views, battery life is a big one. This has been for a long time about um how long can something last? Um untethered experiences as well. The in virtual reality normally you want to track your space within the real world as well, so that you're not static in a single place, but you know, you can move around and you can um experience the world around you. Uh we used to have to have certain sensors for that. Now we have something called inside-out tracking, whereby the cameras on the headset itself can do that. We're constantly developing these technologies, so it's ease of comfort to the user, but then also the experience as well is being developed. Um and then likewise, like I've already talked about software. Um absolutely, bring it on. We need more. >> Definitely, and also um in the past we used to have gloves with specific sensors. We we used to then have the the controllers. I mean, I started working with the with the Polhemus tracker that you put on the top of the head in a in a in a in a motion capture room, so that yeah, the the evolution of the of the technologies is massive. Now we do hand tracking with the same system that Philip mentioned, the inside-out camera tracking, which is yeah, it's fascinating. There's constantly constantly new developments on that thing. >> Okay. Yeah, I think we've talked enough about the hardware and the possibilities. If you already mentioned that something of uh an application, in this case in the health domain, maybe let's focus a bit on that. So, what are we actually doing with it? And specifically, uh I would focus on Tilburg University there. So, what are you doing and maybe other researchers doing with this kind of technology? >> Maybe I can start and talk a little bit about the Elsa lab. >> Yeah, I know. >> This is an interesting project about artificial intelligence uh within the manufacturing industry, and how do we actually implement that? This is where the ELSA comes uh in place. So, it stands for ethical, legal, and societal acceptance aspects of artificial intelligence in and deployment in the industry. So, we have at the moment some hurdles, some difficulties in uh the development and application of artificial intelligence. Artificial intelligence and some of the tools that we currently have in research but also in laboratories could be uh humongously useful in the industry. And they would make that industry or European industry much more competitive against the international market. Um however, we do see things with a more strict, critical, and analytical uh point of view. Um not ourselves as Aalborg University, of course, we also do, but I'm talking about the overall European Union. And we have uh started looking at uh the AI Act, labor laws, and the artificial intelligence, and how that impacts on uh labor overall. So, we have a lot of data privacy, data security concerns, and the application of AI with sensors or with uh the utilization of data from the employees or from the industries can be very uh difficult at the moment. So, um the idea for for the ELSA Lab is that we will uh utilize virtual reality as a way to simulate an environment in which we can test different scenarios of AI utilization within uh industrial uh factory floor scenarios. Um we do that by utilizing digital twins, which is um in in our case is a a replication of the uh setting in which, let's say, a worker who is uh assembling some electronics on top of a of a of a PCB, as we call it, on a board. Um they would get some instructions, some assistance, some documentation. These are some real uh world use cases provided by the industry, our partners. And we can simulate how that would work and how people would react and uh ask for their feedback and see um long-term how that affects them and discuss the impact of AI and how we would implement it, what kind of data are needed, and uh how far can we go with the processing of that data. There's a lot a lot of uh parameters to investigate. We've started that and we started looking at these uh simulation opportunities within the virtual reality as a regulatory sandbox. >> Clear. Although I think what you mentioned is really virtual reality needed for that? What does it contribute? >> Yes, absolutely. I I will explain this. Um we need to simulate the scenarios that otherwise we wouldn't be able to uh to do in the actual factory floor. Cuz at the moment it's impossible. It's restricted. Uh it's not secure. But we can extract that data in a laboratory environment. And in order to simulate how the factory floor looks like or feels like, what kind of processes are taken into into action, we will do that in virtual reality. That's the idea. It's the is the facilitator of the laboratory setting creation. >> Okay. Good. >> Phil, yeah, I can go into a little bit then and it's um maybe not so much with an AI mind on this, but specifically how we are using virtual reality um in these health applications, like I did briefly mention earlier. Um for the last 3 years now, myself, um Eric of Gouda from computational cognitive science, that's the other department, um Mercedes Almagro from cognitive neurocycho neuroscience and Alexandra Haring from Developmental Psychology. The four of us have been doing something called the Digital Interventions Incubator for the last 3 years. The idea from this in its infancy was a internship agreement between a cross-school, cross-faculty so that we could have um, psychology students develop um, or ideation of intervention and then we can use our more technological minded students at CSAI to implement some of these as internship projects. Um, success of this is actually spawned out now into um, the involvement and collaboration with multiple other departments and schools throughout the university. Um, external sort of clinical um, applications within the Netherlands and beyond as well. And yeah, we're a good team now. We're writing grants and we're actually um, hiring developers to uh, create applications which we can use in clinical practice, scientifically backed um, and with the student focus in mind as well so that um, we really hopefully can show a pipeline from um, that ideation to um, clinical use. >> Okay. So, when we talked a little bit before and you also mentioned research into avatars or am I incorrect here? >> Yes, um, I think uh, we were talking about the the future of AI and VR, right? So, there's a lot of um, different um, avenues that we see virtuality and AI being used in I think uh, virtual avatars or virtual agents as we call them really are um, are taking a a big role here. Um, we spoke before with uh, Philip but with other colleagues as well. Um, in Tilburg University, how would it be possible to create more content? And this is a big bottleneck at the moment. We wish we had a, you know, a team that would be able to actually create uh, from from scratch some of the materials we need for for the experiment. Um at the moment we do that uh in our laboratory, but um it would be ideally a bigger effort. But going back to AI, I think this is where uh generative AI and the creation of content comes in place. And also the um realization of virtual agents and this communication. We already have some examples of that. So, we already have the uh integration of LLMs and existing um let's say just GPT 4.0 has been implemented within a virtual reality. My actual PhD Suhana Biswas has uh has done that. He's and she's uh presenting one of her paper next week uh in the artificial intelligence extended reality conference. So, there's a lot of advancements there. But what does it mean for us? I think um it allows us to create more realistic scenarios, more naturalistic interaction. Um yeah, I can talk and talk about this. >> Yeah, go ahead. >> Maybe as well with um the product you told me about before by your collaborating with Keana on sort of um virtual humans as well. And that's sort of a bit of a hot topic as well at the moment. >> Absolutely, absolutely. Um is a very hot topic. There's a lot of conferences that I have workshops about this, but also main tracks just on virtual agents. So, we see this evolution from uh focusing on certain applications to actually focusing now on specific components of uh virtual reality simulation. So, um I think there's a lot of aspects here that we are we are considering not just how an agent looks, but how it behaves, how it responds, how it it can actually uh induce uh emotional reaction to users, how we can appear more realistic. Um there's a lot of different avenues of research that we are exploring. Um we're also very passionate about physiological and objective measurement uh in our lab because in reality lab in in mind labs but also in all of these projects because it provides this additional um perhaps may I say more objective measurement of of the reaction of of users of participants in the side of virtually generated content but it can also give us a little bit of an insight what does that mean for the wellness and well-being overall so that connects and links back to some of the projects that you mentioned. >> Yeah I don't know I shouldn't be the one asking the questions I don't think but I'd like to ask you a little bit of a question on what you just said then the dangers of this right when we're talking about generative AI content we've seen this in the news a lot deep fakes and stuff like this is there an inherent risk in virtual environments for this sort of stuff but would I don't think we're at that stage yet where we're having generative AI created content necessarily for VR applications but you know if we said earlier about how immersive this is and present it is that's a danger right? >> Are we far from that? I don't think we are. >> No I don't think you are either. Actually I just reviewed a paper where somebody basically that was about adapting applications automatically from to the environment to a 3D environment so >> Yeah. >> And not too long ago there there was a lot of hype in the course in splatting and still is I mean in the in the IEEE VR conference which is a virtual reality conference on that aspect we were talking we had a keynote about this. So it's the the the ability I mean I'm not an expert on this but I'll try to explain it as best as I can the ability to take images 2D images and translate that into three-dimensional environment vertices. Um however this does not allow for the animation of those or the disarticulation of where one object ends and where it starts. Of course there's a lot of research now that is able to do object detection and surface extraction, etc. Um we are not there yet. So, I think we're not far from, but we're not there yet. There's a lot of research on this area, and I think a lot of the big companies will actually put some more effort into generating 3D content from from 2D very very soon. >> Yeah, I've seen it with movies already, which is not 100% accurate. If it would be hard to estimate this, but it's pretty good >> Yeah. >> actually. And surprisingly, just free applications which can do that for you. >> Yes. It can generate, as I said, the three-dimensional mesh, but it would be different if you would be if you would wanted to, let's say, pick this glass of water I have on the table, then it would mean that I have to have the whole three-dimensional object here, so I can look at it from every perspective and interact with it. >> Yeah, I understand that a movie would be very different from an application where you have to interact with your A movie can also fix if something is not really correct, but uh >> Exactly. >> Yeah, that's and might be actually that you often see that the the first big step where you do 90% of the stuff, that is the easy step, and then the last bit is the hard step to fix the last little bits, so and of course this is still a big step to go from 2D images to actually 3D representation. >> Yeah. Absolutely. >> Brings up another issue. Sorry, don't have to jump off this topic as well, but um I mentioned a lot of big companies earlier developing these sort of headsets. There are many other big companies training these models on data that we are providing to it as well. Um at what point though do we then create a dependency on this um and taking it away from the content creators and developers as well? It's maybe not a topic for virtual reality and AI, but it's certainly something when we are very reliant on that and we push all of our creative juices out to these models. When do we what control do we have at some point? >> Yeah, that's a very good question and one that I would like to do a a couple of uh podcasts on. >> I'm certainly probably not the person [laughter] to >> I don't know. The thing is with this this topic and I think actually virtual reality is a good uh view because that needs to be a lot of content needs to be creative for virtual reality. >> Sure. >> Actually you and and one issue may be why it is not uh that prevalent in virtual reality yet yet is that there is not that much content to train on. So that means that creativity is maybe limited to creating virtual avatars or something like that but not a whole new applications. But uh but yeah, I think this is this is indeed a big issue that we The things well there are lots of issues. One issue is people who work in this area what they enjoy is the creative aspect. They don't like they don't enjoy fixing what the computer does wrong. So that is one aspect you take away the fun part from people and say you do the stupid part the computer will do the fun part. And the other thing is the stagnation uh that everything will be the same especially maybe in a relatively young topic as virtual reality because you would you train on a few things and then those things will get imitated. So you don't see the new things coming in. But that maybe is still an opportunity for humans again. >> Absolutely. Absolutely. >> I think creativity will not stop in virtual reality so much needed and >> [laughter] >> for the past years I mean we've always had a team someone would do the 3D models, someone would do the textures, someone would do the shaders, someone would do the lighting, you name it. There was so so many specializations. Now I see a a more people who have a little bit of everything in order to manage to to to stay in virtual reality area and and this is actually fascinating because if AI can assist them, they can do they can produce faster. So I see this as a tool. I also see that we're talking about adaptive environments because the real-time aspect of virtual reality is what makes it so interactive and immersive and interesting, right? Um, so this adaptiveness, being able to create personalized experiences call it or user experiences and for example, we had projects in which the the content would change slightly based on the stress of the user. Heart rate, heart rate variability, or emotional reactions. And of course that comes with some limitations or some risks because of course personalized environments could be also used as as a manipulation process. Um, we have we have been from University stance very critical towards anything that utilizes personal data or could be used for the wrong reasons. Um, and maybe in that framework I can mention the REDE is the academic collaborative center for resilient and democratic society because it actually looks into these aspects for the four different themes and yeah, whoever is interested in in these four themes, they can they can come to us and we can discuss further some of these aspects. >> Yeah, so you say come to us. That was one question at here, okay. You mentioned a couple of times that there is a lab. So people come if you walk into the lab, what do you see? What what is there? >> So I'd I'd recommend for anyone going to Google and search virtual reality lab mind labs. I'm sure it'll pop up and well, might see my face good or bad, but um, see us there and there'll be some contact information at the bottom if anyone's interested in topics we're discussing here. And it'll give you sort of a brief look into what our lab is. Um, it's a multitude of equipment. It's headsets, it's sensors. We have a What was a six-camera, now it's an eight-camera motion capture system that goes around the perimeter of the lab now. Um, we've just had a great internship student and who's just even been able to create motion capture content and translate it into uh virtual reality environments, um, essentially at a click of a button. Um, and so this is sort of where we're going with it. It's that integration of all of these sensors, all of this equipment. We have a small, fairly primitive driving simulator setup as well. Um, in there, Effie, anything else I'm forgetting? >> I guess some of the physiological sensing and um how we create the custom setup so that we can track this because it's another thing saying, "Okay, I'm going to use um I don't know, electromyography or enceph- encephalography and record the signals." And another thing to do that in virtual reality. There's so many other factors involved. So, we uh we have expertise in that area and we're able to help and facilitate some of that research. >> Yeah, I can see a certain application here. When I came to Tilburg University, it was about 15 years ago, I believe. Um, people heard that I was interested in gaming and I got a couple of people coming to me and say, "Look, what we have Usually uh from the psychology or the sociology department." And they said, "Well, we do research. We want to see people's reactions to things. So, we give them a written description and they have to tell us how they feel about it. Can't we do this in a game?" Uh not that we have been able to do much there because I also think that at that time a game would not have added that much. A little bit because you can see things rather than a description. And maybe you can do some interaction. But you mentioned that the big advantage of virtual reality is the immersiveness. So, do you do you get people wanting to do this kind of research with you, or should they come to you to do this kind of research? >> I think so, absolutely, right? >> Well, maybe give you a quick example actually that maybe touches on this quite well. It's a another project myself and Ifiok involved in along with Nina Cooper over in TSB. Um and it's good for me, a lovely acronym that Ifiok came up with. And it's specifically looking at multimodal sensor integration in virtual environments and creating a framework for that. So, what that essentially means is we've got lots of information, how do we synchronize that, and how do we um are able to maybe look at an event output at the end of it and say this happened here and this was the response to that from the user. One such application um which we've looked [snorts] into before is the Trier social stress test. Um you traditionally would have a panel of people in real life, you interview them, and you maybe ask them quite tricky questions, and you measure how they respond to that. Now, having a panel of people to be able to do that in a in a real-world environment might be quite time-consuming, quite costly. Um so, if virtual reality is immersive, is it presence-inducing, why can't you do that in a virtual environment instead? And so, maybe that's exactly what we're going to be doing. And uh you can look out for that if the students listening to this to a study upcoming soon. I've not just given the game away. >> [laughter] >> But, it's perfect examples, it's psychological experiments sometimes that you really can manipulate, that you can control yourself. And um you know, okay, maybe we actually want to change this, we want to see that response instead, we want to look at this instead. Um that's the beauty of it. >> Yeah, absolutely. It it's a little bit like a a a Plato's cave, virtuality, isn't it? But, you know that everything is fake, and all and all the shadows are there, but they're not really real, and then you can unplug at any point. But, you can create what whatever cave you want. Um yeah, so for us it's so so fascinating, and I I because of our experience with all the problematic factors in the past, all the hardware issues, all the um middleware issues, I think that's why we came up with a solution. Let's create something that other researchers can use in the future and they can develop without any issues or they can design um without having to worry about how one hardware will communicate with uh with the other. And of course, we're not going to go and and develop something that is customer-ready, but we are we're talking about a framework here and we actually should invite other researchers who have been working maybe along this line for open-source software and they would be happy to collaborate with us. >> Yeah, absolutely. Yeah, something I'm got a conversation this morning about this. It's uh creating the systems open for other people. So, it's all well and good us developing something like this, but um how can this extend beyond our lab? How can other researchers access this and how do we do this in a sustainable fashion as well so that we don't keep reinventing the wheel every time so a researcher wants to be able to do something. We can make an adaptive system whereby we can we treat on that. >> Okay. Well, I think that's a very good place to uh to end it unless you have something that you want to add here. >> For myself, I think we covered all of my topics. >> We thank you very much for the invitation. >> Yeah, thanks for [laughter] having us. Yeah, absolutely. >> Yeah, there was a lot of information compressed compression about 45 minutes, but I hope that this is inspiring for especially people who do research, but they need this kind of immersive technology that they that they would see that's possible at least in the lab. >> Absolutely. Yeah, and like I said earlier, search the lab, you'll find all the contact information there if any of this does sound appealing to you and get in touch with either you or myself as well. We're always open. >> Okay. Thank you very [music] much. >> Thank you.