Ousama Saghie | Pedestrian-Oriented Traffic Light Controller

[music] >> Hello everyone. I am Osama and today we'll be presenting my capstone project, which is the pedestrian oriented traffic light controller. So, here is the table of contents. First, the introduction, the overview of the device and its capabilities, proposed location, modes of operation, software components, the 3D model, hardware components, conclusion, and future work. So, the main problem for this device to be uh relevant is that in Yerevan, we have, for example, tens of people dead on crosswalks each year on crosswalks, and we have hundreds of people hit. Um and 9% of pedestrians overall are victims of crashes on crosswalks. And a study from France shows that 64% of pedestrians do not wait for 40 to 90 seconds, and probably most of you who have crossed the street know that a lot of traffic lights in Yerevan hold pedestrians for even more than 90 seconds. So, the problem with the current systems in Yerevan is that they have fixed timings, and they have long waits for pedestrians. So, the main purpose of this device is to improve, first of all, pedestrian safety by reducing wait times and by reducing the amount of pedestrians crossing on red. Um it should also improve pedestrian comfort to promote pedestrian travel, which should positively impact congestion and air quality. And it also brings some comfort to drivers because the amount of unpredictable pedestrian behavior will go down, and um it will remove unnecessary waits for drivers who are waiting sometimes too long at a crosswalk that has nobody crossing on it. So, the overview is of the system that it have it has the main controller, camera, and it inputs weather, time, and it outputs to um uh the main controller of the traffic lights that the controller module that controls the separate lights, and it can communicate with other controllers, too. So, in more detail, it takes inputs from its environment, mainly pedestrian and vehicle traffic, weather, and time of day, and it also can monitor its own location and position in case of a fault. It has Zigbee and ethernet connectivity, and also Laura, so that it can wirelessly communicate and communicate through a cable. It can send sound warnings, and it has debugging and setup tools. It also has backup power, so it can survive power outages for up to um 15 hours. So, the main components of the device are the main computer, which is a Raspberry Pi 4 Model B, its camera and human presence sensor, the communication modules, the secondary controller, which is Raspberry Pi Pico, and its input-output devices, and the power management system. So, the proposed location of the device for this specific version is at lights in pedestrian call mode, meaning the ones where you have to approach, press a button, and then it will start countdown, and then you will cross, also known as mid-block crossings, so not at intersections. Also, at one-way streets, because it doesn't have a camera looking the other direction. >> [snorts] >> And yeah, not at intersections, so that there are only two primary phases, one where pedestrian is green, vehicle is red, and pedestrian is red, vehicle is green. So, more specifically, it is mounted on a pole like shown here. One camera, the green triangle, is looking on the road, and the other camera is looking on the crosswalk. So, to overview how Yerevan's pedestrian crossings work right now, we have the default configuration, which is pedestrian red and vehicle green. Then, for vehicles, it will go to amber, and then it will go to red, then pedestrian will get green, and then it will go again to the default mode. So, this device like this will allow for the addition of a new mode that is currently not present in Yerevan, and that is a pedestrian default mode, meaning that instead of the traffic light being red for pedestrians, green for cars at the start, it will be red for cars, green for pedestrians, and cars will activate it by approaching the traffic light and being visible to the camera. So, the factors influencing the face time will be pedestrian traffic, more specifically quantity and speed, the weather, the temperature, humidity, and rain, time and date. So, for example, if it's day on or night, if it is school hours, if it is in front of a school, if there are any special events, and also it will be influenced, of course, by vehicle traffic from live camera input, from usual traffic patterns, and it will be influenced by the previous phase, and by input from other traffic lights. So, the modes of operation, it has six modes. It has the capture mode, which is the main mode where it is capturing video of the road and analyzing the footage, and outputting the face times. The second mode is the same, but it also saves the photos for future reference, if it is needed. This is not the primary mode because saving the photos takes additional resources. Um it has the setup mode, which is for setting up the zones, which I will show later, and a debug mode, low power mode, which turns off primarily primarily the communication devices, and it has idle mode where it does nothing, but it is powered on. So, the software components of the system are first the main code, which handles all of the switching of the modes, the analysis of the photos, and all of that stuff. It also has a setup web page, which is shown on the bottom. This is the real web page, but the photo is not real because I didn't take a photo while I was testing the last time, and it allows, as you can see, to set the zones for each part of the road. So, we have the roadway zone, we have the crosswalk zone, we have two waiting zones where pedestrians are waiting to cross, and we have two pavements. Um These zones will be will be later used to count the number of cars and pedestrians in each zone and make decisions based on that. It also has a simulation app that you can see on the top. It has just a simple two traffic light and a button where you can press it and it simulates a pedestrian call. So, the main circuit For the main circuit that is connected to the Raspberry Pi, we have two cameras looking at the roadway, which are here. We have a Zigbee and Laura communication devices, which are here. Uh we have a thermal camera sitting right here. Uh and the camera that looks down onto the crosswalk, which is sitting right here. And it has the ethernet port, which is connected to the traffic light controller or to the computer for debugging. So, the secondary circuit it it is controlled by the Raspberry Pi Pico. It has the smaller input output devices. So, those are push buttons and all lights for also debugging this system. It has a buzzer. It has an IMU and GPS antenna for monitoring its position and its state so that it can detect if it is falling or if it is misaligned. Uh it has the weather sensors, mainly a temperature and humidity sensor and also a water sensor. It also has passive infrared devices to control uh to see whether there are pedestrians on the crosswalk in addition to the camera. So, uh the third circuit is the power management circuit. It can take supply from the uh mains at 230 V AC. Then it will be converted to 19 V DC and then further to 12 V DC to be used to charge um three 2,000 mA hours batteries, which will then provide uh voltage through uh another uh converter uh to 5 V to the Raspberry Pi and the rest of the circuit. And also at 12 V to the temperature controllers that will power the fan and heating pads. Fan and the heating pads. The heating pads are mainly present to in case of snow accumulation on the top of the roof so that they can melt it away. So, the 3D model which you can see it printed right here. These are the it's schematics and dimensions. Yeah, it consists mainly of the base and the walls which hold the components together. It consists of the cover, the pedestrian detection which is done in this part. The stand which is this with where this can attach on top and then this can clamp onto the traffic light pole. And also it has small holders on the inside to hold the various small components. So, here are some rough tests from above the AUA crosswalk and we can see that the device successfully counts the number of cars and pedestrians in each zone. Um >> [snorts] >> So, yeah. In conclusion, the device is able to detect cars and people in various zones of the road. It can make the decisions based on that data. It can communicate with other devices and is powered through the main supply with emergency reserve. It has temperature control for extreme heat and cold situations and it has a case that holds all of the components together and can be mounted on a pole. So, for future work first is to do more testing because very rough testing was done. And the second thing is to activate all of the included components because some of the components which I haven't talked about are not activated. Mainly, it is a lidar which is sitting right here and a millimeter wave radar which is sitting right here for pedestrian detection. Uh the third thing is to add a specialization for the redundant camera because as we can see here, we have two cameras and only one is needed to monitor the roadway ahead. So, the second one can be used for additional functions such as facial recognition or number plate recognition. Um next, uh it is needed to reprint uh and redesign a more durable case. Uh to replace the sequential camera capture with a parallel one. The current uh camera module allows only capturing from one camera at a time which introduces additional delays in the processing while a parallel one can means that uh data from two cameras can come at once and can be processed in the Raspberry Pi at the same time. Um the next thing is to combine the whole uh or as much as possible of the circuit into a PCB so that it can be much more compact than this. Um and to replace the Raspberry Pi board with a Jetson board because it is more specialized for uh this type of stuff and uh to develop lower level codes instead of third-party modules which I have used a lot of right now. And to to add detection for emergency vehicles, public transport, and other high priority traffic. That's it. THANK YOU. >> [applause] >> A LOT OF FUTURE WORK, SO I GUESS YOU ARE STAYING for masters. >> From them. >> Not this year. >> Uh and a very rich project. I think they this bought the second bomb detecting one after the the war things robot. Uh but very uh uh well thought. A lot of work has been done. Uh but from the uh from the paper, I've seen that you have done your field test uh from behind a window. >> Yeah, which is not ideal. >> Uh which will make your thermal camera and IR sensors >> Yes. >> useless. >> That is true. And it is also >> outside environment, yeah. >> This is glass, which means this material has to be changed to uh I know. Yes. >> I have transparent glass wall. >> I have tested it from an open window at home the thermal camera. Yeah. >> Uh yeah, but we see that some people some persons they don't detect >> That is true also because this is from behind the window. This is from the second floor. Uh it is not the ideal conditions for the camera to detect. >> Uh I think the some sort of radars, short-range radars will help to detect uh fast-moving vehicles. >> Yes. We have the lidar here, which can help with that also. >> Overall, very well done. >> Thank you. >> Good job. >> I can >> Uh I think also it's very interesting project, but the confidence score here is not that high in some cases. >> Yes, I actually >> you use or >> I lowered the confidence to 30% here. Uh it is the uh YOLO V26 nano model. Uh >> And it's running on the board? >> It's running on the device, yes. While it is capturing, it is running the image processing also. >> And then uh Laura in and I assume tested because >> No, I didn't. I I didn't. >> Yeah. Uh when you said uh facial recognition, uh you have one camera for that. If you want to place it on the public areas or on the street, you need to also consider privacy of the people as well. So, uh why you need uh to detect the face? >> I'm just saying it can be used for that. It's not a primary. >> All right. >> In theory, which is it? >> So, in different countries, they see you similar systems for not only the traffic, but for like identifying specific people, the faces, or identifying their plate numbers or the vehicles, or let's say I don't know, for the FBI or stuff like that. They use that for So, that's not in our scope, but ideally, a high-quality camera with the specific software and the algorithm can be used for that purposes as well. So, it's a dual-purpose kind of application. >> You're wrong. >> You have no other color. >> So, it's a sponsor. >> Um a quick question, why Zigbee? >> Uh it is a communication module that was available. It has uh an satisfiable range and power, and it can easily connect because basically now the it's like um wireless serial protocol. We just have these two devices. You put them into USBs on each end, and it is ready communication. You don't need any additional um configuration. I mean, very minimal configuration. >> Okay, and it's going to work outdoors, right? >> Yes. >> So, you have the heaters on the roof for the snow. >> Yes. >> Why don't you make the roof not flat, I guess? >> Well, snow sticks. But, uh yeah. >> But, I mean, it's it's just very, very basic engineering. >> Yes, but also uh some of the devices that are inside might be uh might not operate at ideal conditions if the temperature goes below zero. So, it also provides additional heat for the devices on the inside. And it is on right now, so you can see circuit is closed. >> Great work. >> It can be >> For example, let's say the money is equal uh government decided to put it in the city. But they will say, "Okay, this will be lowered in price seven times." What what you will do? >> Sure. I mean, there are a lot of things that can be removed. This is like the Sorry. Uh >> If they say lower it by seven times, okay? >> They calculate it. The Raspberry Pi is lower than >> So, why is it not Oh, it's stand it's standing on the cable. No, it's beeping because it is tilting. Uh and uh to answer your question, basically, we can remove a lot of components from here. The communication devices, the thermal camera, the lidar, the second camera, the millimeter wave radar, the temperature controllers as much as you possibly want, and it will still have the basic functionality with basically even just one camera because the one at the bottom is a wide angle camera. You can see here how much it captures of the road. So, yeah. And I can also plug it into the power. So, you can see it does not blow up. >> So, the device is ready enough so that we can use this device to monitor our one of our labs in G labs. >> What do you mean as a CCTV? A CCTV? >> Yes. >> Well, you just need a camera and transmit it somewhere. >> Detection of the people also. >> Well, yes, but a lot of this is not needed. >> Right now, can we put this in one of our labs? >> Sure, you can. I'll need to modify >> for during the summer. >> Sell it. Don't give it to us. >> We paid for that. >> Seven times more. Yes, with slight modification of the code, you can. With slight modification of the code, you can. And you can have the LoRa device somewhere else at the university or with the Raspberry Pi connected to the Wi-Fi and uh transmitting to somewhere else or through the ethernet cable also. So, yeah. Now Yeah? >> I wanted to mention that although it's pricey, but almost the most expensive components are not from the budget of university. >> Ah, so you want to sell it. >> [laughter] >> Nice. Still very well done. >> Thank you. >> Good job. >> Thanks, man. >> Okay. >> Thank you. >> [applause] [music]