Generative Adversarial Networks for Image-to-Image Translation

✍ Scribed by Arun Solanki (editor), Anand Nayyar (editor), Mohd Naved (editor)

• Publisher: Academic Press
• Year: 2021
• Tongue: English
• Leaves: 428
• Edition: 1
• Category: Library

✦ Synopsis

Generative Adversarial Networks (GAN) have started a revolution in Deep Learning, and today GAN is one of the most researched topics in Artificial Intelligence. Generative Adversarial Networks for Image-to-Image Translation provides a comprehensive overview of the GAN (Generative Adversarial Network) concept starting from the original GAN network to various GAN-based systems such as Deep Convolutional GANs (DCGANs), Conditional GANs (cGANs), StackGAN, Wasserstein GANs (WGAN), cyclical GANs, and many more. The book also provides readers with detailed real-world applications and common projects built using the GAN system with respective Python code. A typical GAN system consists of two neural networks, i.e., generator and discriminator. Both of these networks contest with each other, similar to game theory. The generator is responsible for generating quality images that should resemble ground truth, and the discriminator is accountable for identifying whether the generated image is a real image or a fake image generated by the generator. Being one of the unsupervised learning-based architectures, GAN is a preferred method in cases where labeled data is not available. GAN can generate high-quality images, images of human faces developed from several sketches, convert images from one domain to another, enhance images, combine an image with the style of another image, change the appearance of a human face image to show the effects in the progression of aging, generate images from text, and many more applications. GAN is helpful in generating output very close to the output generated by humans in a fraction of second, and it can efficiently produce high-quality music, speech, and images.

✦ Table of Contents

Front matter
Copyright
Contributors
Super-resolution-based GAN for image processing: Recent advances and future trends
Introduction
Train the discriminator
Train the generator
Organization of the chapter
Background study
SR-GAN model for image processing
Architecture of SR-GAN
Network architecture
Perceptual loss
Content loss
Adversarial loss
Case study
Case study 1: Application of EE-GAN to enhance object detection
Case study 2: Edge-enhanced GAN for remote sensing image
Case study 3: Application of SRGAN on video surveillance and forensic application
Case study 4: Super-resolution of video using SRGAN
Open issues and challenges
Conclusion and future scope
References
GAN models in natural language processing and image translation
Introduction
Variational auto encoders
Drawback of VAE
Brief introduction to GAN
Basic GAN model classification based on learning
Unsupervised learning
Vanilla GAN
WGAN
WGAN-GP
Info GAN
BEGAN
Unsupervised sequential GAN
Parallel GAN
Cycle GAN
Semisupervised learning
Semi GAN
Supervised learning
CGAN
BiGAN
ACGAN
Supervised seq-GAN
Comparison of GAN models
Pros and cons of the GAN models
GANs in natural language processing
Application of GANs in natural language processing
Generation of semantically similar human-understandable summaries using SeqGAN with policy gradient
Semantic similarity discriminator
Generation of quality language descriptions and ranking using RankGAN
Dialogue generation using reinforce GAN
Text style transfer using UGAN
Tibetan question-answer corpus generation using Qu-GAN
Generation of the sentence with lexical constraints using BFGAN
Short-spoken language intent classification with cSeq-GAN
Recognition of Chinese characters using TH-GAN
NLP datasets
GANs in image generation and translation
Applications of GANs in image generation and translation
Ensemble learning GANs in face forensics
Spherical image generation from the 2D sketch using SGANs
Generation of radar images using TsGAN
Generation of CT from MRI using MCRCGAN
Generation of scenes from text using text-to-image GAN
Gastritis image generation using PG-GAN
Image-to-image translation using quality-aware GAN
Generation of images from ancient text using encoder-based GAN
Generation of footprint images from satellite images using IGAN
Underwater image enhancement using a multiscale dense generative adversarial network
Image datasets
Evaluation metrics
Precision
Recall
F1 score
Accuracy
Fréchet inception distance
Inception score
IoU score
Sensitivity
Specificity
BELU score
ROUGE score
Tools and languages used for GAN research
Python
R programming
MatLab
Julia
Open challenges for future research
Conclusion
References
Generative adversarial networks and their variants
Introduction of generative adversarial network (GAN)
Generative model (GM)
Discriminator model (DM)
Related work
Deep-learning methods
Convolutional neural network
Recurrent neural network (RNN)
Deep belief network (DBN)
Long short-term memory
Variants of GAN
Vari GAN
TGAN
Laplacian pyramid of generative adversarial network (LAPGAN)
Video generative adversarial network (VGAN)
Superresolution GAN (SRGAN)
Face conditional generative adversarial network (FCGAN)
Applications of GAN
Conclusion
References
Comparative analysis of filtering methods in fuzzy C-means: Environment for DICOM image segmentation
Introduction
Organization of chapter
Related works
Methodology
Proposed algorithm
Evaluation metrics
Morphological operations
2D median filter
Imguided filter
Imfilter
Wiener 2 filtering
Gaussian filter
Research design
Experimental analysis
Performance analysis
Results and discussion
Conclusion
References
A review of the techniques of images using GAN
Introduction to GANs
Need for GANs
GAN architectures
Fully connected GANs
Conditional GANs
Adversarial autoencoders
Deep convolution GANs
StackGANs
CycleGANs
Wasserstein GANs
Discussion on research gaps
GAN applications
Conclusion
References
A review of techniques to detect the GAN-generated fake images
Introduction
DeepFake
DeepFake challenges
GAN-based techniques for generating DeepFake
Image-to-image translation
StarGAN: Unified generative adversarial networks for multidomain image-to-image translation
Toward multimodal image-to-image translation
U-GAT-IT: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-i ...
Image-to-image translation with conditional adversarial networks
Multichannel attention selection GAN with cascaded semantic guidance for cross-view image translation
Cross-view image synthesis using geometry-guided CGANs
Cross-view image synthesis using CGANs
WarpGAN: Automatic caricature generation
CariGANs: Unpaired photo-to-caricature translation
Unpaired photo-to-caricature translation on faces in the wild
Text-to-image synthesis
Generative adversarial text-to-image synthesis
StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks
MC-GAN: Multiconditional generative adversarial network for image synthesis
MirrorGAN: Learning text-to-image generation by redescription
StackGAN++: Realistic image synthesis with stacked generative adversarial networks
Conditional image generation and manipulation for user-specified content
Controllable text-to-image generation
DM-GAN: Dynamic memory generative adversarial networks for text-to-image synthesis
Object-driven text-to-image synthesis via adversarial training
AttnGAN: Fine-grained text-to-image generation with attentional generative adversarial networks
Cycle text-to-image GAN with BERT
Dualattn-GAN: Text-to-image synthesis with dual attentional generative adversarial network
Artificial intelligence-based methods to detect DeepFakes
Can forensic detectors identify GAN-generated images?
Detection of deep network-generated images using disparities in color components
Detecting and simulating artifacts in GAN fake images
Detecting GAN-generated fake images using cooccurrence matrices
Detecting GAN-generated imagery using color cues
Attributing fake images to GANs: Analyzing fingerprints in generated images
FakeSpotter: A simple baseline for spotting AI-synthesized fake faces
Incremental learning for the detection and classification of GAN-generated images
Unmasking DeepFakes with simple features
DeepFake detection by analyzing convolutional traces
Face X-ray for more general face forgery detection
DeepFake image detection based on pairwise learning
Comparative study of artificial intelligence-based techniques to detect the face manipulation in GAN-generated fake ...
Techniques for detecting the construction of a new face
Techniques for detecting the swapping of the facial identity
Techniques for detecting the manipulated of facial features
Techniques for detecting the manipulated facial expressions
Legal and ethical considerations
Conclusion and future scope
References
Synthesis of respiratory signals using conditional generative adversarial networks from scalogram representation
Introduction
Related work
GAN for signal synthesis
Simple GAN
Conditional generative adversarial networks
Conditional GAN for respiratory sound synthesis
System model
Time-scale representation using CWT
Generator and discriminator network architecture of cGAN
Algorithm
Steps
Results and discussion
Dataset
Data augmentation using conditional GAN
Samples of generated scalogram images for different classes
Synthesis of respiratory sounds using inverse CWT
Performance results
Analysis
Conclusion and future scope
References
Visual similarity-based fashion recommendation system
Introduction
Related works
Vanilla GAN
InfoGAN
CNN-based architectures
Fashion recommendation system
Deep network architectures
Proposed network
State-of-the-art CNNs
Experiments and results
Experimental setup
Comparative results
Web interface for visual inspection
Conclusion and future works
References
Deep learning-based vegetation index estimation
Introduction
Related work
Vegetation index: Formulations and applications
Deep learning-based approaches
Proposed approach
Cycle generative adversarial networks
Residual learning model (ResNet)
Proposed architecture
Loss functions
Least-square GAN's loss
Results and discussions
Datasets for training and testing
Data augmentation
Evaluation metrics
Experimental results
Conclusions
References
Image generation using generative adversarial networks
Introduction to deep learning
Generative deep learning
Variational autoencoder
Introduction to GAN
Nash equilibrium
GAN and Nash equilibrium
Nash equilibrium proof
Training problems
VAE-GAN
Applications
Image-to-image translation using {c, cycle}-GAN
Face generation using StarGAN
Photo-realistic images using SRGAN and Art2Real
Image animation and scene generation using monkey net, first-order motion, and StackGAN
Future of GANs
References
Generative adversarial networks for histopathology staining
Introduction
Generative adversarial networks
Improvements to vanilla GAN
Deep convolutional GANs
Variations in optimization functions
Image-quality metrics
The image-to-image translational problem
Histology and medical imaging
Histology as different feature spaces
Network architecture and dataset
ANHIR dataset
Dataset preparation
Network architectures
Results and discussions
Conclusions
Appendix: Network architectures
References
Analysis of false data detection rate in generative adversarial networks using recurrent neural network
Introduction
Contributions
Related works
Methods
GAN-RNN architecture
Optimization of GAN using RNN
Performance evaluation
Dataset collection
Performance metrics
Discussions
Conclusions
References
WGGAN: A wavelet-guided generative adversarial network for thermal image translation
Introduction
Related work
Infrared image translation
GANs in image translation
Wavelet-guided generative adversarial network
Overall architecture
Wavelet-guided variational autoencoder
Reparameterization in latent space
Discrete wavelet transformation for pooling
Objective functions in adversarial training
Experiments
Data description
Evaluation methods
Baselines
Experimental setup
Translation results
Qualitative analysis
Quantitative analysis
Conclusion
References
Generative adversarial network for video analytics
Introduction
Building blocks of GAN
Training process
Objective functions
GAN variations for video analytics
GAN variations for video generation and prediction
GAN variations for video recognition
GAN variations for video summarization
PoseGAN
Discussion
Advantages of GAN
Disadvantages of GAN
Conclusion
References
Multimodal reconstruction of retinal images over unpaired datasets using cyclical generative adversarial networks
Introduction
Related research
Multimodal reconstruction of retinal images
Cyclical GAN methodology
Paired SSIM methodology
Network architectures
Experiments and results
Datasets
Qualitative evaluation of the reconstruction
Quantitative evaluation of the reconstruction
Ablation analysis of the generated images
Structural coherence of the generated images
Discussion and conclusions
References
Generative adversarial network for video anomaly detection
Introduction
Anomaly detection for surveillance videos
A broader view of generative adversarial network for anomaly detection in videos
Literature review
The basic structure of generative adversarial network
The literature of video anomaly detection based on generative adversarial network
Cross-channel generative adversarial networks
Future frame prediction based on generative adversarial network
Cross-channel adversarial discriminators
Training a generative adversarial network
Using generative adversarial network based on the image-to-image translation
Unsupervised learning of generative adversarial network for video anomaly detection
System overview
Feature collection
Spatiotemporal translation model
Anomaly detection
Experimental results
Dataset
UCSD dataset
UMN dataset
CHUK Avenue dataset
Implementation details
Evaluation criteria
Receiver operating characteristic (ROC)
Area under curve (AUC)
Equal error rate (EER)
Frame-level and pixel-level evaluations for anomaly detection
Pixel accuracy
Structural similarity index (SSIM)
Performance of DSTN
The comparison of generative adversarial network with an autoencoder
Advantages and limitations of generative adversarial network for video anomaly detection
Summary
References
Index