Train Custom Deep Learning Models Without Coding using QGIS, Roboflow and Ultralytics

[music] >> Hello, in this video I'm going to walk you through the full workflow of creating, training, exporting, and running a custom deep learning model for geospatial object detection using QGIS, the Deepness plugin, Roboflow, and Ultralytics. In a previous video, I've already explained how to install the Deepness plugin and use it for pre-trained models. In this video, we're going to train a model to detect wind turbines. We'll use a 25 cm aerial photograph of an area in the Netherlands that has many wind turbines onshore and offshore. The data was downloaded from Beeldmateriaal and clipped to the area for training of the deep learning algorithm. In the Deepness panel, indicate that the input layer is your aerial photograph and set the processed area mask to entire layer. Scroll down to processing parameters and there change the resolution to 25 cm, which is the resolution of this aerial photograph. Change the tile size to 1280 pixels. Keep the tiles overlap at 15%. This tile overlap prevents objects from being cut off at the tile edges. Under training data export, browse to the folder where you want to save the result and make sure that export image styles is checked. Then click export training data. This will take a while depending on the size of your image. When it's finished, you'll get a pop-up which indicates how many tiles were generated. Now go to the folder where the tiles were generated and there you need to delete all the tiles that don't have windmills. This is a lot of work though there were many tiles created, but it cleans up the data set that you'll use later. >> [music] >> Now we go to the Roboflow website to create an annotated data set. Sign up. When you sign up by email, verify your email. Enter your full name. Accept the terms. Click continue. Before we can get started, name your workspace. I keep the default here. Click continue. Then indicate how you're going to use Roboflow and we will use it to train and improve data sets and models. Click continue. We can skip this screen to invite your team and create a workspace. In the pop-up, it shows what is included in the premium trial. You can close the dialogue. Now on the left side, go to projects. Click new project and give it a name. For example, wind turbines detection. Make sure the project type is object detection and click next. Now you can upload the data. You can drag and drop the files or select the folder. Then it starts processing the files >> [music] >> and they appear on the screen when done. Click save and continue and now they'll be uploaded. When the upload is completed, Roboflow will ask you how do you want to label your images? The auto label entire batch feature is nice, but in our case it will result in segments instead of bounding boxes that we need for detection. Therefore, select here label myself. Now click start annotating. Now the most time-consuming step starts. Instead of manually drawing the bounding boxes, we'll use the box prompting feature to speed up the process. Box prompting is a feature that lets you guide the auto labeling model by drawing a rough bounding box around the object you want to detect. Instead of relying on the model to find the object in the entire image, you give it a hint about where to look. Roboflow then uses that hint to generate a precise bounding box. Let's use the box prompt to create our first box. Draw it around a wind turbine and click save. This box is now used to train the box prompting model. Use the arrow button on the keyboard to go to the next tile. Now the box prompting model will try to detect the box around the windmill automatically. If it's done correctly, click approve predictions. We go to the next tile and it will repeat the process. The wind turbine is well detected and we approve. Now we're on a tile that I confused with a windmill and it has no windmills. Therefore, click the null tool to exclude this from the training data set. Let's go to the next one. It detects it well, so I approve. And then the next one, it detects two, but if I increase then the confidence level, it detects only one which I can approve. In this way, continue indicating the bounding boxes around your training data set. >> [music] >> Sometimes it detects too much. Then I need to shrink the bounding box manually to better fit the wind turbine. Save the object and go to the next one. So that was a lot of work. Now I add the images to the data set and I change here the method to split images between training, validation, and test data set using a default distribution. And then I click to add. Under data set, we can now see our annotated tiles. Click new data set version and there change resize to the size of the tiles that we had, which was 1280 by 1280. Click apply. Then click continue. We don't want any further augmentation, so click continue. And then we're ready to create the data set. Once it's done, click download data set. There make sure to choose YOLO version 5 PyTorch and download it as a zip to your computer. Now we move to Ultralytics to create our deep learning model based on the annotated data. Click get started. Click get started for free. And here you can sign up. If you use your company email, you get $25 credits for free. Any amount of free credits will do for this tutorial. Click create account. It will send you the verification code. Enter the code and click verify email. Here you can change the company name if you want or just click continue. Now choose your data region where your models and data will be stored and then you're all set. Click get started. It will now set up your dashboard with an example data set and project. In the Ultralytics dashboard, we can now upload a new data set. Click new data set. Upload the zip file that we created in Roboflow. Make sure that the task type is set to detect. Click create and upload. After uploading, we can see our annotated tiles. We can use these to train a new model. Click new model. As a base model, choose YOLO V5. Choose the YOLO V5 MU model. You can try other ones. Here we will use cloud training and not local training. This will cost us some credits, but our free trial has some free credits, so let's use them. We start training the model. During training, we can see in a graphical way how some key metrics evolve. These metrics give an indication of performance of the training. We can see here at the end of the training that the results of the metrics are >> [music] >> very promising. We can see more metrics if we go to train and then under charts, you can find there also the loss metrics. They also look quite okay here. We find this model acceptable, so go to the export tab and there choose the ONNX format. Click export. This is the format that the Deepness plugin in QGIS can use. Back in QGIS, I'm going to use the same image that we used for training to see how well our model performs. I zoom in to a certain part there and in the Deepness panel, I make sure that the input layer is that raster layer and that the processed area mask is just a visible part. I change the ONNX model type to detector and I load the model exported from Ultralytics. It gives us some information here which we keep. Make sure that under the detection parameters, you change the detection type to YOLO Ultralytics and then click run. Once it's done, it will give you the count of wind turbines in the visible part of the image. And when we zoom in, we can see how well it performed. Did quite well, but there are some wrong ones. It missed the windmill somewhere and also some objects are misclassified as windmills. But overall, it did a great job. Now, let's apply our deep learning wind turbine detection model to a new area. Here I've loaded an aerial photograph of a wind park near Afsluitdijk in the Netherlands. Make sure to change the input layer to the one that we see and use the entire layer. We use the same model, so we don't change anything there. And then the only thing I need to push is run. Once it's done, we see the pop-up that it counted 155 wind turbines. And let's visually check the result. It detected many, but some are missing. So, this gives us an idea of the performance of the model when applied to a new image. I hope this was a useful video. Please subscribe and see you next time.