An estimated 5 to 25 million tons of plastic are thrown in our oceans every year. Although we know that those devastating trashes for biodiversity and environment mainly come from rivers and beaches, it remains really tough to catch them all, especially in some poor and neglected countries.
From this observation, we decided to consider a solution using deep learning to detect rubbish with a camera.
Is it possible to generate a trash detector which would be the first step to clean up our rivers?
Such an issue is the kind we want to address at Picsell.ia.
The code for this experiment is available here.
The TACO trashes Dataset available on the Dataset Hub seemed to be really suited to begin our project. This Dataset contains 1500 pictures of everyday life trashes that we tried to annotate accurately.
As you can see the Dataset isn’t really well balanced and doesn’t have a lot of objects annotated but that’s a first try, we will allow you to contribute to this Dataset in the next few week !
We fine tuned a Faster RCNN model pre-trained on COCO (model already available on the Model Hub) and within 2 hours, our “trash detector” model was ready to be tested on the playground with our own data.
Let’s have a look on the playground :
Here, the bottles are well detected on this picture, but to be honest, our model is not as good on every images.
The reasons are of course the lack of training data, and the fact that we didn’t optimized our network so far. This leaves a lot of room for improvements.
Although this project has proved itself, it can hardly be used for the moment. Our final goal is to embed our model on an edge device (eg. NVIDIA Jetson..)and run it in real time. But for that the model has to be way more accurate than it is now and also suited for near real-time inference.
But how to improve the model ?
This article is the first part of a series, here we have just made a ‘prototype’ of our algorithm but the next parts will be dedicated to :
- Better algorithm choice and optimization
- The influence of training data
- The deployment on edge device
The different ideas we would like to implement are the following ones :
- The dataset : What if we had at our disposal a TACO trashes dataset where all trashes were floating on the water ? All of our annotations are in fact very precise segmentation of the objects (we extracted the bounding-boxes in this article for prototyping purpose) but we can easily perform data augmentation and simply cut and paste the objects on another backgrounds (like sea water) or perform some advanced techniques like Neural Style Transfer to make it look more like ‘real’ data. We will also increase the volume of training data along the way.
- The model : Some models are more accurate but also a lot heavier than others so we will try a bunch of them and try to find a compromise so we can achieve our goal and compare the results.
Finally, if we succeed in setting this trashes detector up, the next goal will be to find a way to pull the trashes detected out of the water. That is now out of our field, but we are sure that you will find solutions such as the first floating devices that already exist.
1. AI for CFD: Intro (part 1)
2. Using Artificial Intelligence to detect COVID-19
3. Real vs Fake Tweet Detection using a BERT Transformer Model in few lines of code
4. Machine Learning System Design
This is a long run project that will need a lot of iterations but after that’s what we want to facilitate at Picsell.ia !
Do not hesitate to come along and help us built the biggest Open Computer Vision Hub and share this with your relatives, also please come and ask for help if you need it, it’s always a pleasure to guide you 😉
Credit: BecomingHuman By: Thibaut Lucas
