Lab 10: Processing Oblique Imagery in Pix4D

Introduction

This lab utilizes the tool of image annotation which allows the user to remove unwanted pieces of an image. For this lab we used oblique imagery and 3D models that were created in Pix4D, and using those models and the image annotation tool we were then able to remove unwanted parts of the image leaving just the truck and light post.

At what phase in the processing is this performed?

This step needs to be performed after the initial processing.

What types of Image Annotation are there?

There are three types of image annotation: Mask, Global Mask, and Carve. 

• Mask annotation would be used for removing some obstacles that appear in a few of the images or to automatically remove the sky
• Global Mask annotation would be used for a piece of the image that is consistently in the lens at a fixed location. This could be a piece of the drone for example.
• Carve annotation can be used to remove large pieces of an image, such as the sky or a parking lot. 

Method 

This lab involved the processing of 2 sets of images, one was a pickup truck and the other was a light post in a field. We used these images to create a 3D point cloud model and then used image annotation to remove the background and only have the final object in question remaining.

Truck Data Set

For this data set we wanted to remove every aspect of this image except the truck, so the carve annotation made the most sense. This is because a majority of the image needed to be removed leaving only the truck so that the final product was focused on the image. This step needed to be done after the initial processing. About 14 images were doing with the preliminary carve annotation which can be seen in image 1, and a handful of images involved the full annotation, which can be seen in image 2.

Image 1: Beginning Carve Annotation

Image 2: Full Carve Annotation

Once this process was complete it was time to move on to the second stage of processing, creating the point cloud. The following images below (image 3 and image 4) show the final product after the point cloud was processed.  

Image 3: Front view of truck

Image 4: Side view of truck

Light Pole Data Set

Due to a fewer number of images the processing time for the initial processing of the light pole was significantly shorter. I noticed it was significantly more difficult to get the light pole properly annotated as it is a tall and narrow object, so fully removing the entire background proved to be a long process. From this data set I fully annotated 6 images and did partial annotation on 22 out of the 33 images. Figure 5 shows the partial annotation of the light pole and figure 6 shows the full annotation. 

Image 5: Partially annotated pole

Image 6: Fully annotated pole

After this step was completed then it was time to perform the next step of the processing which created the point cloud mesh. Image 7 below shows the final product once this processing was complete.

Image 7: Pole final product

Discussion 

Once the image has been processed with the point cloud the errors with the annotation become evident. This can clearly be seen with the truck as the annotation was done too close to the base of the truck which cut off the bottom part of the truck. The same issue can be seen with the light pole. There is a balancing act between getting close to the object but not so close that the object itself gets annotated. When annotating it is also important to ensure all the angles are covered. As the vehicle flies along the angle on the image changes leading to different backgrounds needing to be annotated. By covering a larger number of images with the annotation it will ensure a better final product because all of the potential backgrounds would have been annotated. 

Conclusion

In order to created an undisturbed 3 dimensional image of a single object it is important to use image annotation. By removing the background of the image the final product will only hold the desired object in the image.