Semantic Segmentation for Pneumothorax Detection & Segmentation | by smit kumbhani | Aug, 2020

Semantic segmentation — classifies all the pixels of an image into meaningful classes of objects. These classes are “semantically interpretable” and correspond to real-world categories. For instance, you could isolate all the pixels associated with a cat and color them green. This is also known as dense prediction because it predicts the meaning of each pixel.

Instance segmentation — identifies each instance of each object in an image. It differs from semantic segmentation in that it doesn’t categorize every pixel. If there are three cars in an image, semantic segmentation classifies all the cars as one instance, while instance segmentation identifies each individual car.

1. Machine Learning Concepts Every Data Scientist Should Know

2. AI for CFD: byteLAKE’s approach (part3)

3. AI Fail: To Popularize and Scale Chatbots, We Need Better Data

4. Top 5 Jupyter Widgets to boost your productivity!

Application of Image Segmentation:

1 Medical Imagine

2 Computer Guided Surgery

3 Video surveillance

4 Recognition Tasks

5 Self-Driving Car

6 Industrial Machine Vision for product assembly and inspection

This Problem basically from the Healthcare domain. imagine suddenly
gasping for air, helplessly breathless for no apparent reason. Could it be a
collapsed lung? In the future, we are going to predict this answer.

Pneumothorax can be caused by chest injury, damage from underlying
lung disease, or most horrifying — it may occur for no obvious reason at all.
On some occasions, a collapsed lung can be a life-threatening event.

Pneumothorax is usually diagnosed by a radiologist on a chest x-ray
images ,but sometimes it could be difficult to confirm.

Pneumothorax is visually diagnosed by radiologist, and even for a
professional with years of experience; it is difficult to confirm.

So Our Goal is to Detect and Segment those Pneumothorax affected area with a help of Semantic Segmentation methods,so that we can help the radiologist by giving the results with higher precision.

Solution:

An AI algorithm to detect Pneumothorax would be useful to Solve this problem,

we will try to solve this problem in two Phase:

1 Pneumothorax Classification:

2 Pneumothorax Segmentation

So, in first phase we will develope a classification model to classify Pneumothorax and in Second Phase we will build a model for segmentaiton task on given image

How prediction pipeline will work?

if Classification model detects the Pneumothorax in input X-Ray image then our Prediction Pipeline will pass that X-Ray image to Segmentation Model to segment the Pneumothorax in That X-ray image so that Radiology Expert can easily Analyze and Diagnose this Problem

Before Going to Understand the Classification & Segmentation Architectures, let’s Have a glance at Dataset,

here we are using dataset which has been modified from the kaggle competition’s dataset.

so that we can easily get our input and target mask in (.png format).

Before start the Explaination Deep Learning Architecture, i assume that you have guys have a good Understanding of Basic Convolutional Operations like,

Conv2d, UpSampling , Dense Layers, Upconv2d Layer, Conv2dTranspose Layer , Softmax, Relu, BatchNormalisation all Basic stuff of Deep Learning with keras and most important Residual Block (ResNet & DenseNet).

As you know that we have divided this problem into 2 part , So let’s have a look at part-1,

Part 1 : Pneumothorax Classification:

Basically here to solve this Classification problem , we have used DenseNet121 Architecture.

DenseNet:

DenseNet is One of the new Architecute for image classification & Object Recognition , it is quite similar to ResNet Architecture with some fundamental differences, ResNet uses an additive method (+) that merges the previous layer (identity) with the future layer, whereas DenseNet concatenates (.) the output of the previous layer with the future layer.

Advantages of Densenet:

1. They alleviate the vanishing-gradient problem.
2. Strengthen feature propagation.
3. Encourage feature reuse.
4. Reduce the number of parameters.

here for our problem i used the DensNet-121 architecture Using this package,

from tensorflow.keras.applications.densenet import DenseNet121

But there is one more important thing that i have used to get state of the art results is, for Transfer Learning instead of using pre-trained weights of imagenet , i have used the weights of ChestXpert — DenseNet , because this ChestXpert model was trained on Medical X-Ray images to classify around 14 Disease which was related to Lungs.

and Pneumothorax was one of them so i directly loaded the weights of ChestXpert model for our DenseNet-121 but only for all convolutional blocks , and for dense layer i have initialized the weights using keras..

Below is the keras code to implement the above model,

from tensorflow.keras.applications.densenet import DenseNet121
from tensorflow.keras.layers import Dense, Input
from tensorflow.keras.models import Model, load_modeldef get_chexnet_model():
input_shape = (256, 256, 3)
img_input = Input(shape=input_shape)
base_weights = 'imagenet'  #create the base pre-trained model
base_model = DenseNet121(include_top=False,
input_tensor=img_input,
input_shape=input_shape,
weights=base_weights,
pooling='avg')  x = base_model.output
# add a logistic layer -- let's say we have 14 classes
predictions = Dense(14,activation='sigmoid',name='predictions')(x)  # this is the model we will use
model = Model(inputs=img_input,
outputs=predictions,)  # load chexnet weights
model.load_weights('/content/drive/My Drive/Case-Study    1/best_weights.h5')  # return model
return base_model, modeltf.random.set_seed(1234)
base, model = get_chexnet_model()
x = Dense(1024, activation='relu', kernel_initializer='he_normal')(model.layers[-2].output)
x = Dense(2, activation='softmax', kernel_initializer='he_normal')(x)final_model =  Model(model.input, x)
final_model.summary()

and you if you guys want to Understand the DenseNet Architechure then Must read this Paper on DenseNet architecture

Training:

To Train this model we used

final_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=[tf.keras.metrics.AUC(), 'accuracy'])final_model.fit_generator(generator=Train_pipeline, epochs=30, validation_data=Test_pipeline)

and we achieved 90%Accuracy on trainset and 87% Accuracy on validation set….

So now after completing the part 1 ,Let’s have a look at Part -2

Part 2: Pneumothorax Segmentation:

Now we will discuss most important part of this blog ,

For this task we have implemented the Architecture called ” UNet++ : nested UNet architecture” , it is the advanced or can say extended version of UNet.

to Understand this architecture must have basic idea about how UNet work for Semantic segmentation , you can read this to understand the UNet Architecture.

So i hope now you will have a better understandings of UNet.

UNet++ : Nested UNet architecture for Medical Image Segmentaion:

So as you can see in this diagram so we will find some similarites between UNet and UNet++

Because Like Unet , UNet++ also follows the same encoder-decoder approch to generate the semantic segmentation

But here i have mentioned some points which differs UNet++ from UNet:

1. Convolution layer on skip pathways which bridges the semantic gap between encoder-decoder
2. Dense skip connections at skip pahways which improves the Gradient flow and prevents from gradient vanishing problem
3. Deep Supervision which improves the model pruning

that’s it , But if you want to go more deeper to understand How this things work in UNet++ then you from here you will get the better idea

Below keras code will help you to define above model (UNet++):

def convolution_block(x,filters,
size,strides(1,1), 
padding='same',activation=True):
x = BatchNormalization()(x)
if activation == True:
x = LeakyReLU(alpha=0.1)(x)
return x
def residual_block(blockInput, num_filters=16):
x = LeakyReLU(alpha=0.1)(blockInput)
x = BatchNormalization()(x)
blockInput = BatchNormalization()(blockInput)
x = convolution_block(x, num_filters, (3,3) )
x = convolution_block(x, num_filters, (3,3), activation=False)
x = Add()([x, blockInput])
return x