Mask R-CNN for Self Driving Cars

A complete guide for solving instance segmentation problems

By 2040, 95% of new vehicles sold will be fully autonomous.

You might also like

Best AI and Machine Learning Applications in the Banking Sector | by Sasha Andrieiev | Sep, 2021

Transfer Learning — Introduction !! Part -2 | by RAVI SHEKHAR TIWARI | Sep, 2021

Why Information For All Future Breakthrough Innovation Already Exists | by Anton Krutz | Sep, 2021

Two reasons why this is revolutionary:

1. It will save 1.25 million lives every year from traffic accidents
2. It will give you the equivalence of 3 extra years in a lifetime, currently spent in transit

To identify each object instance of each pixel for every known object within an image. This problem is known as instance segmentation. I will be focussing mostly on the applications of mask r-CNN for self-driving cars in this blog. I have used google colab for the implementation.

1. Content-based image retrieval
2. Self Driving Cars
3. Computer vision
4. Video surveillance
5. Traffic control systems
6. Object detection
7. Medical imaging

1. It is simple, flexible, and general framework for object instance segmentation.
2. It efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance.
3. Mask R-CNN is simple to train and adds only a small overhead to earlier state of art Faster R-CNN, running at 5 fps.
4. Mask R-CNN is easy to generalize to other tasks like allowing us to estimate human poses in the same framework, detection landmarks in satellite imagery, segmenting medical images etc.

1. google colab
2. numpy
3. scikit-image
4. matplotlib

I have used the MS-COCO data-set as mentioned in the official mask r-CNN GitHub repository. The link to the data-set can be found here.

This is a typical instance segmentation problem. I have used Mask-R-CNN in this project.

• Image Classification: Classify the main object category within an image.
• Object Detection: Identify the object category and locate the position using a bounding box for every known object within an image.
• Semantic Segmentation: Identify the object category of each pixel for every known object within an image. Labels are class-aware.
• Instance Segmentation: Identify each object instance of each pixel for every known object within an image. Labels are instance-aware.

Let’s start coding it. Feel free to check out the corresponding notebook here.

I started with cloning and setting up the mask r-CNN from the official repository.

Next I imported the required libraries and dependencies.

Then I imported mask r-cnn and coco dataset from the repository. Also i made a directory to save logs and trained models.

After that, I downloaded coco pre-trained weights from the mask r-CNN repository.

I continued with making the list of class names as specified in the ms-coco dataset.

The next step was to take an image and convert it into an URL to run instance segmentation on. For that I used postimages. Check it out here.

Finally, let’s run instance segmentation on the input image and display the output image.

Mask R-CNN is considered the state of art for instance segmentation problems. It has been proven to be useful across industries and domains like medical imaging, satellite image segmentation, segmenting objects in computer vision problems etc.

The corresponding notebook can be found here.

Happy reading, happy learning and happy coding!

And don’t forget to 👏 if you enjoyed this article 🙂.