Fighting Urban Heat with Reality Mapping with Arkadiusz Szadkowski | ESRI

Duration: 15:31 2887 transcript words EN summary

Video summary

Arcadius Szadkowski from Esri opens his presentation by defining "reality mapping" as a technology that replicates the human ability to perceive and reconstruct reality using two eyes and a brain, but scales this capability through GIS software. He explains that while traditional GIS workflows often focus on aggregating vector layers downstream, reality mapping integrates high-quality imagery, satellite data, and remote sensing directly into the platform to create accurate 3D base maps. This approach eliminates confusion regarding what lies between lines and points, paving the way for an era of automated mapping where high-quality imagery allows for fast, high-quality insights that eventually replace manual methods. The core mission is to bring these advanced photogrammetric capabilities to millions of users by providing a foundational layer that decision-makers can trust to address critical urban challenges. The presentation highlights the severe impact of extreme heat on cities, citing a shocking statistic that more people died in European cities from heat last year than were shot in the US. To combat this, Szadkowski details a proof-of-concept project in Phoenix, Arizona, which utilized free satellite data like Landsat to identify long-term temperature rises and partnered with high-resolution aerial surveys using RGB, near-infrared, and thermal cameras. By processing this simultaneous data acquisition through reality mapping tools, the team generated foundational products including orthophotos, digital surface models, point clouds, meshes, and thermal mosaics. These outputs allowed for precise alignment of shadows and temperatures across different spectral bands, enabling analysts to distinguish between overheating asphalt and cooler vegetated areas with pixel-level accuracy. Building upon these high-resolution foundations, the system automatically extracted building footprints, segmented vegetation above ground level, and performed detailed shadow analyses to determine exactly how much shade specific locations receive throughout the day. Users can interact with a thermal 3D mesh to visualize temperature variations by clicking on different areas, while specialized tools allow for the isolation of hot pavement surfaces for digitization into vector feature layers. This data is then integrated into an enterprise GIS environment where it can be overlaid with census data, population density, and IoT sensor inputs to create a comprehensive digital twin. This evolution from a descriptive model to a fully functional digital twin enables scenario-based modeling and forecasting, empowering urban planners to simulate interventions like adding green canopies or adjusting building designs to mitigate heat islands effectively. In conclusion, Szadkowski emphasizes that the true solution to urban heat is not merely collecting more data but converting it into actionable impact through clarity and coordination. By investing in high-quality imagery and spectral data as a robust foundation, municipalities can move beyond simple mapping to rebuilding a strong base for diverse applications like urban planning and public health safety. The ultimate goal is to provide politicians and decision-makers with factual, trustworthy information that encourages them to take immediate action where it is needed most. He invites the audience to explore demo apps and OGC blog articles that demonstrate how cities can fight extreme heat using these open standards and independent software solutions, reinforcing that real change comes from acting on clear data rather than just visualizing colors on a map.

Read the full video transcript

Yeah. All right. Very good. My name is Arcadus for short ADC and I work with Esri and I would like to invite you today to my presentation about how how the extreme heat is become has become the huge problem for the cities and how we can help cities to understand and eliminate any any opinions and look into the brutal facts because this is very important to start action and uh and acting on that. Before that I need to start a story about what is reality mapping. Reality mapping capabilities at S3 are powered by RGS reality suit. But to explain you what is reality mapping. I would like to tell a story I told to my son. One year ago he went I was picking up from school. He was 7 years old and he was really angry. And I asked him why you're angry and I said today I had to stand in front of the entire class and explain what my parents are doing for living. Oh what did you tell them? I felt that you are sitting in front of the computer talk English and travel a lot because I don't understand what you are doing. So I told him hey you were born I couldn't explain to him a principles of the photoggramometry or imagery right so I I approached this from the perspective you were born with two eyes you were born with these two eyes for a reason and you were born with a very powerful computer your brain your eyes your your eyes acts act as a camera each eye is capturing a an image of the world and your brain computes this in the real time all so you can play so you can kick the ball and you understand reality. You are reconstructing reality in the real time. You are texturing this with the color. And what I'm doing, we are trying to replicate what nature gave you with eyes and the brain and put it into the GIS software. So everybody who works with our software can understand what he's looking at. And there are different there are different terms. There is a reality capture, reality mapping, reality modeling. We have chosen to use the reality mapping because we want to be closer to the maps. We are in map making business. And it all begins today with the sensors. There is multiple sensors capturing and observing the earth. And we are really betting on being hardware agnostic to consume satellite, aerial or drone imagery. So any user can can bring any type of the imagery. And those imagery are processed and we are producing products like tuto point clouds, meshes and um and digital surface models that are used later about and the whole concept is that we want to bring this base map 3D base map that is used on together with the other GIS layers and overlay all the analysis on top of it to eliminate any uh confusions about what's in between the lines and the points in the standard GIS workflow. So we look from the perspective of the GIS market has been always concentrated downstream. Users were aggregating layers, vector layers and building their maps. With reality mapping, we want to extend the portfolio to do a seamless integration with any of the imagery and um remote sense data all within our RGS platform. And why we do this? We do this because we really believe that from any imagery through reality mapping and digital surface model and short of photo a new era is emerging an era of automated mapping and a mapping that can provide very fast high quality insights that stereo old traditional stereo mapping and manual mapping will be replaced sooner or later with the GI mapping but it's all depends on the high quality imagery and then when I tell to my team always like we are on the mission to bring an imagery and uh and the photoggramometry to 10 million RGS pro users. Today very few of them is using this technology but we need to bring it and put it in the context that they understand and a lot of GIS organiz GIS users or people using GIS they think from the use case perspective what how can I use it for my city for my nation for my organization. So we identified that there is a huge problem with a lot of the cities and uh there are the the the city starts to overheat. We have a huge problem and there is a fact actually that I learned yesterday. According to the vauhao last year more people died because of the heat in cities in Europe than people were shot in US. That's a very brutal fact but it puts a little bit in the perspective. So we start looking into this okay what we can do about it and then we identified our one of our customers city of Phoenix as you know in Arizona it's a very hot in the summer we looked into the population data this is a growing city and this city uh of course it has a heat problem and overheating of the city so what we looked into is the first thing where we look is then the imagery data that freely open data which is for example LANCAT the longest operating satellite that has a multisspectral resolution and also has and also has the long temporal resolution because has been operating for so many many years. But the problem with the with the Halansat is that is 30 m resolution. So very low spatial resolution but Lancet allows us to identify through the thermal channel that there is really a problem with the heating and we actually looked into the uh many decades of the uh of the analysis and what you see is the temperature is raising over the decades. So the average temperature the city of Phoenix is on the rise. So then of course we brought this Lancet data into the into the GIS software and start analyzing all the channels and very quickly we and our customer using our software realized that okay this spectral resolution temp the heat can be mapped but I don't see a details. You can see this big blue um uh area. We will come back to this. I wonder if you can guess what it is right now. guess in your head and you will it will be presented in a moment. So to go through there we partner with some of our partners and done a proof of concept over the city of Phoenix with dual installation of the high resolution RGB camera with also with the near infrared channel and the echo mapper which is a thermal camera. It's been it has been done as an aerial survey over a part of the city and this uh and the concept here is to create a foundational data that has high spatial resolution and high spectral resolution. We have a six band and all was captured simultaneously in within one of flight and then we process this data with our reality reality mapping capability software. We produce all these products which is short of auto DSMs uh point clouds meshes and thermal automoses and then we I will get it in a moment uh to show you but first let me show you the the the examples. So we are right now okay yeah we are right now looking on the true order of photo of the free and 3.5 cm resolution over the city of Phoenix and yes as you understand the true order of auto is corrected by the with the leaning effect. So when we compare this with the satellite imagery, you see that tall buildings are leaning. That's a very important uh definition to understand that our tools are focusing on producing troto photo that is a pixel accurate automos with the position of the GIS data but also elevation data like digital surface models. So those are the first products to photo and DSM model that was used. Then of course we produced the 3D mesh model with the with the city of Phoenix just to show them the the capability of the 3D perspective. This is a super high resolution mesh with three and a half cm. And what you see is already here that city started recognizing and doing this canopies for the shade for some of the citizens. And uh what we see here al for example two different parking lots. One with the shading with the roofs and one with very very open in between. So okay and then the next product was the thermal mosaic because we have done the simultaneous data acquisition. The thermal autophoto was extremely well aligned with the RGB or true auto photo. The shadows and the shadows are exposed and visible exactly in the same position and that that delivers a very strong value for the analysis because when we do a GOI and feature analysis, we have a four actually six bands here. If we include the infrared uh we have all the bands aligned perfectly in the pixel precision. So what you see I'm swiping between the RGB and the thermal resolution and the shadows are line very well between the two or two automoses. Okay, of course we can because we are in the GIS application we can read each each value of the pixel either is a thermal or the RGB pixel. So these products we call them foundational products that are brought into the GIS but we want to the goal for us is to create a share understanding because for the um decision makers we need to make build the trust for them that they can trust that okay there is a problem we need to act and but where we need where we want to act first. So this is where power of GIS comes um very handy and to do that we introduce this digital twin curve to them. So the first foundation level at the at all your digital twin journey is always creating a digital models. You cannot have a digital model before you have a digital sorry you don't you cannot have a digital twin before you establish a digital model. So we we establish the we call it descriptive layer with the foundational and the cornerstone of the reality mapping. Then the next level of this evolution is where the users are able to collect some of the information. So you can put IoT sensors, you can extract features from it, you can do the analysis, but only only after you build these two layers, you are able to actually elevate and and grow to the fully true digital twins that are where you can run the scenario-based modeling or any simulations and forecasting. Your digital twin highly depends on the quality of your digital model and descriptive in information layer. So they agreed with us and we were we are on we are still on the journey on this because we just finished the proof of concept and this journey will be will continue but all of this would not be possible without the imagery and remote sense data processed uh as an input to to the workflow. So let's go back to the to the city of Phoenix and I will show you now some of the capabilities. So first is the full automatic extraction of the buildings because we have a troto and the DSM model we were able to map full automatically all the buildings with the GI algorithms. So of course we can bring an existing cadustra data if they exist if we trust that they are up to date but with the new imagery and high resolution imagery we can also map them automatically. Then we computed the NDVI uh based on the infrared uh true of photo and uh we have a very good quality information about the vegetation and using the DSM model in the combination with Truto and NDVI we are able to extract vegetation. So we are able to segment vegetation and please notice that vegetation was segmented only above the ground level. So we didn't catch the grass, we catch only the trees that are standing because the trees cast shadow and the trees also are very important for the fighting the urban heat. So this way we created a a vector layer fully automatically with a with a just few clicks. Uh the next analysis that can that was performed was like a time series of the shadow analysis for the user. So of course we when we have a accurate um um the mesh provides you additional context where you allowing you to easy interact with the data in the way that is not possible in the 2D. So like performing a simple visual shadow analysis but also you can compute it and present it as a feature layer that shows exactly how much shadow time total time per day you will have in a specific location. And here is a simple app also where we use the uh side rendering of the thermal imageries. By adjusting the temperature slider, the user is able to quickly cut the histogram and show and rem remove the coolest pixels or the hottest pixels and isolate for example a pavement. So by using the thermal auto or thermal autophoto, we were able to separate immediately all the pavement and the asphalt because that was overheating uh a lot in the city. And this can be automatically then digitized into the feature layer. So that's a site feature for the user to uh to have a have a road network or the pavement network as uh as a vector feature. And here we are back to the location where there are the canopies already created by the city and we immediately see that the temperature is lower in this area. And the last uh but not least is the thermal 3D mesh. So once we created the 3D mesh and the thermal automos, you can drape it on top of the mesh and because we have a simultaneous data capture and simultaneous data production, everything aligns very well. So the shadows, temperature of the shadow and the shadow created by the analysis will align very well. And of course in this case we are simplifying to the 1 m hexagonals and the user is able to interact with the mesh by clicking and uh and seeing where is the higher temperature and and and lower. So this takes me a little bit to the perspective. So we created a very high resolution spatial temporal and spectral snapshot of the real of the city all in case to create then and then brought it to the enterprise GIS system. So the user can now overlay their other existing GIS data like census data, where people live, where people work, how people travel and then we are now on the way to connect IoT sensors to have a real-time sensors about the temperature in the city and other open data all in case because we want to create a multiple twins on top of that foundation. Urban planning is just one of the use cases. But where we going with this is that when CT is investing in the high resolution and spectral data as a foundation and is able to connect it with the GIS data, it's not about anymore about building one twin. It's about rebuilding a strong foundation that many things can grow from it. So this takes me a little bit to the final conclusion. So we really right now on we are on the mission to promote thinking end to end solution investing in the high quality imagery and input data bringing it into the into the workflow like reality mapping build your content so increase your temporary resolution use it for the data analysis and the and the visualizations all in order to help the city municipalities and decision makers to act on their on their city. So the real change isn't in collecting more data. It's in converting it into the impact that leads to action. And we don't really fight heat with colors on the map here. We try to fight it with clarity and coordinations and the courage for the politicians and decision makers to actually trust based on the factual information and the data that that they need to act and where they need to act first. So thank you. That was my presentation and uh you can if you take the picture you can test your demo apps the the our demo apps with uh some of the small snapshot of the data and there is also a very good OGC blog article about how cities fight and beat extreme OGC is the open geo special consortium so independent from S3 or any other uh any other uh software or hardware providers so thank you for the opportunity thank you for your time.