Front Cover
Deep Learning for Medical Image Analysis
Copyright
Contents
Contributors
Foreword
1 Deep learning theories and architectures
1 An introduction to neural networks and deep learning
1.1 Introduction
1.2 Feed-forward neural networks
1.2.1 Perceptron
1.2.2 Multi-layer perceptron
1.2.3 Learning in feed-forward neural networks
1.3 Convolutional neural networks
1.3.1 Convolution and pooling layer
1.3.2 Computing gradients
1.3.3 Deep convolutional neural networks
1.3.3.1 Skip connection
1.3.3.2 Inception module
1.3.3.3 Attention
1.4 Recurrent neural networks
1.4.1 Recurrent cell
1.4.2 Vanishing gradient problem
1.5 Deep generative models
1.5.1 Restricted Boltzmann machine
1.5.2 Deep belief network
1.5.3 Deep Boltzmann machine
1.5.4 Variational autoencoder
1.5.4.1 Autoencoder
1.5.4.2 Variational autoencoder
1.5.5 Generative adversarial network
1.6 Tricks for better learning
1.6.1 Parameter initialization in autoencoder
1.6.2 Activation functions
1.6.3 Optimizers
1.6.4 Regularizations
1.6.5 Normalizations
1.7 Open-source tools for deep learning
References
2 Deep reinforcement learning in medical imaging
2.1 Introduction
2.2 Basics of reinforcement learning
2.2.1 Markov decision process
2.2.2 Model-free methods
2.2.2.1 Policy gradient methods
2.2.2.2 Value-based methods
2.2.2.3 Actor-critic methods
2.2.3 Model-based methods
2.2.3.1 Value function
2.2.3.2 Policy search
2.3 DRL in medical imaging
2.3.1 DRL for parametric medical image analysis
2.3.1.1 Formulation
2.3.1.2 Landmark detection
2.3.1.3 Image registration
2.3.1.4 Object/lesion localization and detection
2.3.1.5 View plane localization
2.3.1.6 Plaque tracking
2.3.1.7 Vessel centerline extraction
2.3.2 Solving optimization using DRL
2.3.2.1 Image classification
2.3.2.2 Image segmentation
2.3.2.3 Image acquisition and reconstruction
2.3.2.4 Radiotherapy planning
2.3.2.5 Video summarization
2.4 Future perspectives
2.4.1 Challenges ahead
2.4.2 The latest DRL advances
2.5 Conclusions
References
3 CapsNet for medical image segmentation
3.1 Convolutional neural networks: limitations
3.2 Capsule network: fundamental
3.3 Capsule network: related work
3.4 CapsNets in medical image segmentation
3.4.1 2D-SegCaps
3.4.2 3D-SegCaps
3.4.3 3D-UCaps
3.4.4 SS-3DCapsNet
3.4.5 Comparison
3.5 Discussion
Acknowledgments
References
4 Transformer for medical image analysis
4.1 Introduction
4.2 Medical image segmentation
4.2.1 Organ-specific segmentation
4.2.1.1 2D segmentation
4.2.1.2 3D medical segmentation
4.2.2 Multi-organ segmentation
4.2.2.1 Pure transformers
4.2.2.2 Hybrid architectures
4.2.2.2.1 Single-scale architectures
4.2.2.2.2 Multi-scalearchitectures
4.3 Medical image classification
4.3.1 COVID-19 diagnosis
4.3.1.1 Black-box models
4.3.1.2 Interpretable models
4.3.2 Tumor classification
4.3.3 Retinal disease classification
4.4 Medical image detection
4.5 Medical image reconstruction
4.5.1 Medical image enhancement
4.5.1.1 LDCT enhancement
4.5.1.2 LDPET enhancement
4.5.2 Medical image restoration
4.5.2.1 Under-sampled MRI reconstruction
4.5.2.2 Sparse-view CT reconstruction
4.5.2.3 Endoscopic video reconstruction
4.6 Medical image synthesis
4.6.1 Intra-modality approaches
4.6.1.1 Supervised methods
4.6.1.2 Semi-supervised methods
4.6.1.3 Unsupervised methods
4.6.2 Inter-modality approaches
4.7 Discussion and conclusion
References
2 Deep learning methods
5 An overview of disentangled representation learning for MR image harmonization
5.1 Introduction
5.1.1 Domain shift
5.1.2 Image-to-image translation and harmonization
5.2 IIT and disentangled representation learning
5.2.1 Supervised IIT and disentangling
5.2.2 Unsupervised IIT and disentangling
5.3 Unsupervised harmonization with supervised IIT
5.3.1 The disentangling framework of CALAMITI
5.3.2 Network architecture
5.3.3 Domain adaptation
5.3.4 Experiments and results
5.4 Conclusions
Acknowledgments
References
6 Hyper-graph learning and its applications for medical image analysis
6.1 Introduction
6.2 Preliminary of hyper-graph
6.3 Hyper-graph neural networks
6.3.1 Hyper-graph structure generation
6.3.2 General hyper-graph neural networks
6.3.3 Dynamic hyper-graph neural networks
6.3.4 Hyper-graph learning toolbox
6.4 Hyper-graph learning for medical image analysis
6.5 Application 1: hyper-graph learning for COVID-19 identification using CT images
6.5.1 Method
6.5.2 Experiments
6.6 Application 2: hyper-graph learning for survival prediction on whole slides histopathological images
6.6.1 Ranking-based survival prediction on histopathological whole-slide images
6.6.1.1 Method
6.6.1.2 Experiments
6.6.2 Big hyper-graph factorization neural network for survival prediction from whole slide image
6.7 Conclusions
References
7 Unsupervised domain adaptation for medical image analysis
7.1 Introduction
7.2 Image space alignment
7.2.1 MI2GAN
7.2.1.1 X-shape dual auto-encoders
7.2.1.2 Mutual information discriminator
7.2.1.3 Objective
7.2.2 Implementation details
7.2.2.1 Network architecture
7.2.2.2 Optimization process
7.2.3 Experiments
7.2.3.1 Data sets
7.2.3.2 Ablation study
7.2.3.3 Comparison to state of the art
7.3 Feature space alignment
7.3.1 Uncertainty-aware feature space domain adaptation
7.3.1.1 Adversarial learning block for feature space alignment
7.3.1.2 Uncertainty estimation and segmentation module
7.3.1.3 Uncertainty-aware cross-entropy loss
7.3.1.4 Uncertainty-aware self-training
7.3.1.5 Overall objective
7.4 Experiments
7.4.1 Exploration on uncertainty estimation
7.4.2 Comparison with existing UDA frameworks
7.5 Output space alignment
7.5.1 Robust cross-denoising network
7.5.1.1 Robust cross-denoising learning
7.5.1.2 Overall training objective
7.5.2 Experiments
7.5.2.1 Comparative study
7.6 Conclusion
References
3 Medical image reconstruction and synthesis
8 Medical image synthesis and reconstruction using generative adversarial networks
8.1 Introduction
8.2 Types of GAN
8.2.1 GAN
8.2.2 Conditional GAN
8.2.3 AmbientGAN
8.2.4 Least squares GAN and Wasserstein GAN
8.2.5 Cycle-consistent GAN
8.2.6 Optimal transport driven CycleGAN
8.2.7 StarGAN
8.2.8 Collaborative GAN
8.3 Applications of GAN for medical imaging
8.3.1 Multi-contrast MR image synthesis using cGAN
8.3.2 MRI reconstruction without fully-sampled data using AmbientGAN
8.3.3 Low dose CT denoising using CycleGAN
8.3.4 MRI reconstruction without paired data using OT-CycleGAN
8.3.5 MR contrast imputation using CollaGAN
8.4 Summary
References
9 Deep learning for medical image reconstruction
9.1 Introduction
9.2 Deep learning for MRI reconstruction
9.2.1 Introduction
9.2.2 Basic of MR reconstruction
9.2.3 Deep learning MRI reconstruction with supervised learning
9.2.3.1 Purely data-driven methods
9.2.3.2 Unrolling-based methods
9.2.4 Deep learning MRI reconstruction with unsupervised learning
9.2.5 Outlook
9.2.6 Conclusion
9.3 Deep learning for CT reconstruction
9.3.1 Image domain post-processing
9.3.2 Hybrid domain-based processing
9.3.3 Iterative reconstruction via deep learning
9.3.4 Direct reconstruction via deep learning
9.3.5 Conclusion
9.4 Deep learning for PET reconstruction
9.4.1 Introduction
9.4.2 Conventional PET reconstruction
9.4.3 Deep learning-based algorithms in PET imaging
9.4.4 Conclusion
9.5 Discussion and conclusion
References
4 Medical image segmentation, registration, and applications
10 Dynamic inference using neural architecture search in medical image segmentation
10.1 Introduction
10.2 Related works
10.2.1 Efficient ConvNet models for medical imaging
10.2.2 Domain adaptation
10.2.3 Neural architecture search
10.3 Data oriented medical image segmentation
10.3.1 Super-net design and training
10.3.2 Data adaptation with super-net
10.4 Experiments
10.5 Ablation study
10.5.1 Validation with single path or multiple paths
10.5.2 Guided search and random search
10.5.3 Training with single path or multiple paths
10.6 Additional experiments
10.7 Discussions
References
11 Multi-modality cardiac image analysis with deep learning
11.1 Introduction
11.2 Multi-sequence cardiac MRI based myocardial and pathology segmentation
11.2.1 Introduction
11.2.2 Methodology summary for challenge events
11.2.2.1 MS-CMRSeg challenge: cardiac segmentation on late gadolinium enhancement MRI
11.2.2.2 MyoPS: myocardial pathology segmentation from multi-sequence cardiac MRI
11.2.3 Data and results
11.2.3.1 Data
11.2.3.2 Evaluation metrics
11.2.3.3 Results from MS-CMRSeg challenge event
11.2.3.4 Results from MyoPS challenge event
11.2.4 Discussion and conclusion
11.3 LGE MRI based left atrial scar segmentation and quantification
11.3.1 Introduction
11.3.2 Method
11.3.2.1 LearnGC: atrial scar segmentation via potential learning in the graph-cut framework
11.3.2.2 AtrialJSQnet: a new framework for joint segmentation and quantification of left atrium and scars incorporating spat...
11.3.3 Data and results
11.3.3.1 Data acquisition
11.3.3.2 Gold standard and evaluation
11.3.3.3 Performance of the proposed method
11.3.4 Conclusion and future work
11.4 Domain adaptation for cross-modality cardiac image segmentation
11.4.1 Introduction
11.4.2 Method
11.4.2.1 DDFSeg: disentangle domain features for domain adaptation and segmentation
11.4.2.2 CFDNet: characteristic function distance for unsupervised domain adaptation
11.4.2.3 VarDA: domain adaptation via variational approximation
11.4.3 Data and results
11.4.3.1 Data
11.4.3.2 Comparison study for DDFSeg
11.4.3.3 Comparison study for CFDNet
11.4.3.4 Comparison study for VarDA
11.4.4 Conclusion
References
12 Deep learning-based medical image registration
12.1 Introduction
12.2 Deep learning-based medical image registration methods
12.2.1 Deep learning-based medical image registration: supervised learning
12.2.2 Deep learning-based medical image registration: unsupervised learning
12.2.3 Deep learning-based medical image registration: weakly-supervised learning
12.2.4 Deep learning-based registration: smoothness, consistency and other properties
12.3 Deep learning-based registration with semantic information
12.4 Concluding remarks
References
13 Data-driven learning strategies for biomarker detection and outcome prediction in Autism from task-based fMRI
13.1 Introduction
13.2 BrainGNN
13.2.1 Notation
13.2.2 Architecture overview
13.2.3 ROI-aware graph convolutional layer
13.2.4 ROI-topK pooling layer
13.2.5 Readout layer
13.2.6 Putting layers together
13.2.7 Loss functions
13.2.8 Experiments and results
13.2.9 Brain-GNN implication for dynamic brain states
13.3 LSTM-based recurrent neural networks for prediction in ASD
13.3.1 Basic LSTM architecture for task-based fMRI
13.3.2 Strategies for learning from small data sets
13.3.3 Prediction of treatment outcome
13.4 Causality and effective connectivity in ASD
13.4.1 Dynamic causal modeling
13.4.2 The effective connectome
13.4.3 Overcoming long time series and noise with multiple shooting model driven learning (MS-MDL)
13.4.4 Adjoint state method
13.4.5 Multiple-shooting adjoint state method (MSA)
13.4.6 Validation of MSA on toy examples
13.4.7 Application to large-scale systems
13.4.8 Apply MDL to identify ASD from fMRI data
13.4.9 Improved fitting with ACA and AdaBelief
13.4.10 Estimation of effective connectome and functional connectome
13.4.11 Classification results for task fMRI
13.5 Conclusion
References
14 Deep learning in functional brain mapping and associated applications
14.1 Introduction
14.2 Deep learning models for mapping functional brain networks
14.2.1 Convolutional auto-encoder (CAE)
14.2.2 Recurrent neural network (RNN)
14.2.3 Deep belief network (DBN)
14.2.4 Variational auto-encoder (VAE)
14.2.5 Generative adversarial net (GAN)
14.3 Spatio-temporal models of fMRI
14.3.1 Deep sparse recurrent auto-encoder (DSRAE)
14.3.2 Spatio-temporal attention auto-encoder (STAAE)
14.3.3 Multi-head guided attention graph neural networks (multi-head GAGNNs)
14.3.4 SCAAE and STCA
14.4 Neural architecture search (NAS) of deep learning models on fMRI
14.4.1 Hybrid spatio-temporal neural architecture search net (HS-NASNet)
14.4.2 Deep belief network with neural architecture search (NAS-DBN)
14.4.3 eNAS-DSRAE
14.4.4 ST-DARTS
14.5 Representing brain function as embedding
14.5.1 Hierarchical interpretable autoencoder (HIAE)
14.5.2 Temporally correlated autoencoder (TCAE)
14.5.3 Potential applications
14.6 Deep fusion of brain structure-function in brain disorders
14.6.1 Deep cross-model attention network (DCMAT)
14.6.2 Deep connectome
14.7 Conclusion
References
15 Detecting, localizing and classifying polyps from colonoscopy videos using deep learning
15.1 Introduction
15.2 Literature review
15.2.1 Polyp detection
15.2.2 Polyp localization and classification
15.2.3 Uncertainty and calibration
15.2.4 Commercial systems
15.3 Materials and methods
15.3.1 Data sets
15.3.1.1 Polyp detection
15.3.1.2 Polyp localization and classification (with uncertainty and calibration)
15.3.2 Methods
15.3.2.1 Detection of frames with water jet sprays and feces
15.3.2.2 Few-shot polyp detection
15.3.2.3 Localization and classification of polyps
15.3.2.4 Polyp classification uncertainty and calibration
15.4 Results and discussion
15.4.1 Polyp detection experiments
15.4.2 Polyp localization and classification experiments
15.4.3 Uncertainty estimation and calibration experiments
15.4.4 System running time
15.5 Conclusion
References
16 OCTA segmentation with limited training data using disentangled representation learning
16.1 Introduction
16.2 Related work
16.3 Method
16.3.1 Overview
16.3.2 Conditional variational auto-encoder
16.3.3 Anatomy-contrast disentanglement
16.3.4 Semi-supervised segmentation
16.3.5 Data sets and manual delineations
16.3.6 Foveal avascular zone segmentation
16.4 Discussion and conclusion
References
5 Others
17 Considerations in the assessment of machine learning algorithm performance for medical imaging
17.1 Introduction
17.1.1 Medical devices, software as a medical device and intended use
17.2 Data sets
17.2.1 General principles
17.2.2 Independence of training and test data sets
17.2.3 Reference standard
17.2.4 Image collection and fairness
17.2.5 Image and data quality
17.2.6 Discussion
17.3 Endpoints
17.3.1 Metrics
17.3.1.1 Segmentation
17.3.1.2 Classification (e.g., computer-aided diagnosis, or CADx, algorithms)
17.3.1.3 Detection (e.g., computer-aided detection, or CADe, algorithms)
17.3.1.4 Triage, prioritization and notification (e.g., computer-aided triage, or CADt, algorithms)
17.3.1.5 Image generation, noise reduction, reconstruction
17.3.1.6 Quantitative imaging tools
17.3.1.7 Discussion
17.4 Study design
17.4.1 Transportability
17.4.2 Assessment studies for ML algorithms in medical imaging
17.4.2.1 Standalone performance assessment
17.4.2.2 Clinical performance assessment
17.4.2.3 Prospective vs. retrospective studies
17.4.3 Discussion
17.5 Bias
17.5.1 Bias and precision
17.5.2 Bias and generalizability
17.5.3 Types and sources of bias in pre-deployment performance evaluation studies of ML algorithms in medical imaging
17.5.3.1 Case collection
17.5.3.2 Over-fitting, train-test contamination and data leakage
17.5.3.3 Reference standard
17.5.3.4 Study endpoints and metrics
17.5.4 Discussion
17.6 Limitations and future considerations
17.7 Conclusion
References
Index
Back Cover