Deep learning for improved breast cancer monitoring using a portable ultrasound scanner
About 1 in 8 U.S. women will develop invasive breast cancer during their lifetimes. It’s the second leading cause of cancer death for women in the U.S. Ultrasound imaging is a noninvasive medical imaging technique used for breast cancer screening. At Insight, I worked on a consulting project with a local start-up company that developed an automated, portable, and wearable ultrasound imaging platform allowing users to perform self-monitoring of breast health. The goal of the project is to automatically detect malignant lesions in ultrasound images. In this blog, I’ll describe how this project uses segmentation to detect lesions in an image, and classification to detect whether those lesions are benign or malignant.
Data pre-processing
For the segmentation task, radiologists drew contours around the lesions in 240 pairs of raw images. The regions within the contour were highlighted in the mask images (see image above). For the classification task, I had roughly balanced data for benign and malignant cases. Before using the raw data for the training, they were pre-processed as shown in the following figure.
The raw images were filtered to suppress noise, then the contrast in the images was enhanced. Since the dataset was small, augmentation was performed on the images, such as horizontally flipping the image, rotation and other deformations. Finally, the images were cropped to the same size.
Segmentation
Since a few patients had multiple images in the dataset, the data were separated, by patient, into three parts: training (80%), validation (10%), and testing (10%).
The model was a modified U-Net and trained on GPU hosted by Amazon Web Services (AWS) EC2 instances. The loss function used is illustrated in the figure below, with “A” representing the ground truth (manually labeled mask) and “B” representing the model generated mask. If the mask from the model is completely off, then the intersection/union ratio is 0. If they are completely overlapping, then the ratio is 1.
The resulting image (below) shows the ground truth in blue, with the red contour generated by the model. The box plot below shows a summary of the testing results. The average intersection/union ratio is 0.74, which means that, on average, there’s 74% overlap between the target mask and our prediction output. The baseline ratio is 0.06, and is calculated by assuming the model is just randomly guessing. In this case, the intersection is half the size of the average mask size, and the union is half the size of the whole image.
Classification
When a radiologist reads an ultrasound image, he first identifies the lesion region in the image, then looks at both the lesion region and surrounding regions for different types of features in order to make a decision. Here, we built a model to mimic this process.
The architecture of the model for classification is shown in the following figure. Since there was no mask available in the provided classification dataset, the masks were generated using the model in the previous segmentation step. Raw images were passed through convolutional layers so the model could learn a set of filters to extract features. The mask provided the region of interest and, after, it passed through convolutional layers with the same number of filters as the image branch. For each filter in the image branch, there was a corresponding filter in the mask branch. Some filters in the image branch were important in extracting features from the lesion region, while others were important in extracting features from the background region of the image. The filters in the mask branch can be trained to weight these two regions differently for features learned in the image branch. These two pieces of information were combined and fed into convolutional layers, then fully connected layers, to generate results.
In order to see what the model learned after training, we fed it with a fixed mask and a blank image with white noise, as shown in the following figure. To generate an image that represented the malignant features learned, we let the model modify the blank image so that the probability of malignancy is maximized (to almost 1).
The following figure shows two examples of the generated images from the model. From these images, we can observe a few things:
- Both the lesion region and surrounding region contribute to the classification.
- The lesion regions are more important, since most of the features are concentrated inside these regions.
- We can see that the model looked for different features for these two regions. This shows that the model indeed learned where and what to look for in the images.
The results of the testing dataset are shown in the following confusion matrix. The model is optimized for recall in order to reduce the false negative. The testing accuracy of the model is 0.79 and the recall is 0.85 on test data.
Discussion
In this project, I used deep learning techniques to automatically detect lesion regions and classify the lesion, which can have both cost and time-saving benefits. Currently, patients must make appointments with ultrasound technicians and physicians, which is both time-consuming and costly. In order to reduce cost, combining the device and the mode can reduce the manual process involved. An estimation of the reduction in the manual process can provide a quantified deliverable for this project.
The equation used to estimate the manual process is shown in the following:
The positive class rate is the prevalence of breast cancer, which is roughly 12%. The recall can be set to 1 by adjusting the threshold of the model. Since the class rate in the training is different than the prevalence rate, the precision from the model is adjusted. The estimated percentage is about 26%, which means that combining the device and model can reduce 74% of the cost for users.
Conclusions
The deep learning model developed in this project can automatically detect lesions in the ultrasound images. By combining the model with the portable scanner, it can produce repeatable images and allow users to monitor their health changes over time, based on their own baseline, for the right diagnosis at the right time.
Qin Miao was an Insight Health Data Science Fellow in 2019. He holds a PhD in bioengineering. At Insight, he built deep learning models that can automatically detect lesions in ultrasound images.
Are you interested in working on high-impact projects and transitioning to a career in data? Sign up to learn more about the Insight Fellows programs and start your application today.
