Machine Learning for Future Fiber-Optic Communication Systems

✍ Scribed by Alan Pak Tao Lau, Faisal Nadeem Khan

• Publisher: Academic Press
• Year: 2022
• Tongue: English
• Leaves: 404
• Category: Library

✦ Synopsis

Machine Learning for Future Fiber-Optic Communication Systems provides a comprehensive and in-depth treatment of machine learning concepts and techniques applied to key areas within optical communications and networking, reflecting the state-of-the-art research and industrial practices. The book gives knowledge and insights into the role machine learning-based mechanisms will soon play in the future realization of intelligent optical network infrastructures that can manage and monitor themselves, diagnose and resolve problems, and provide intelligent and efficient services to the end users.

With up-to-date coverage and extensive treatment of various important topics related to machine learning for fiber-optic communication systems, this book is an invaluable reference for photonics researchers and engineers. It is also a very suitable text for graduate students interested in ML-based signal processing and networking.

✦ Table of Contents

Front Cover
Machine Learning for Future Fiber-Optic Communication Systems
Copyright
Contents
Contributors
Preface
Acknowledgments
1 Introduction to machine learning techniques: An optical communication's perspective
1.1 Introduction
1.2 Supervised learning
1.2.1 Artificial neural networks (ANNs)
1.2.2 Choice of activation functions
1.2.3 Choice of loss functions
1.2.4 Support vector machines (SVMs)
1.2.5 K-nearest neighbors (KNN)
1.3 Unsupervised learning
1.3.1 K-means clustering
1.3.2 Expectation-maximization (EM) algorithm
1.3.3 Principal component analysis (PCA)
1.3.4 Independent component analysis (ICA)
1.4 Reinforcement learning (RL)
1.5 Deep learning techniques
1.5.1 Deep learning vs. conventional machine learning
1.5.2 Deep neural networks (DNNs)
1.5.3 Convolutional neural networks (CNNs)
1.5.4 Recurrent neural networks (RNNs)
1.5.5 Generative adversarial networks (GANs)
1.6 Future role of ML in optical communications
1.7 Online resources for ML algorithms
1.8 Conclusions
1.A
References
2 Machine learning for long-haul optical systems
2.1 Introduction
2.2 Application of machine learning in perturbation-based nonlinearity compensation
2.2.1 Wide & deep neural network
2.2.2 Data collection and pre-processing
2.2.3 Training results
2.2.4 Results and discussion
2.3 Application of machine learning in digital backpropagation
2.3.1 Physics-based machine-learning models
2.3.2 Single-polarization systems
2.3.3 Dual-polarization systems
2.3.4 Subband processing via filter banks
2.3.5 Training and application examples
2.4 Outlook of machine learning in long-haul systems
References
3 Machine learning for short reach optical fiber systems
3.1 Introduction to optical systems for short reach
3.2 Deep learning approaches for digital signal processing
3.3 Optical IM/DD systems based on deep learning
3.3.1 ANN receiver
3.3.1.1 PAM transmission
3.3.1.2 Sliding window FFNN processing
3.3.2 Auto-encoders
3.3.2.1 Auto-encoder design based on a feed-forward neural network
3.3.2.2 Auto-encoder design based on a recurrent neural network
3.3.3 Performance
3.3.4 Distance-agnostic transceiver
3.4 Implementation on a transmission link
3.4.1 Conventional PAM transmission with ANN-based receiver
3.4.2 Auto-encoder implementation
3.5 Outlook
References
4 Machine learning techniques for passive optical networks
4.1 Background
4.2 The validation of NN effectiveness
4.3 NN for nonlinear equalization
4.4 End to end deep learning for optimal equalization
4.5 FPGA implementation of NN equalizer
4.6 Conclusions and perspectives
References
5 End-to-end learning for fiber-optic communication systems
5.1 Introduction
5.2 End-to-end learning
5.3 End-to-end learning for fiber-optic communication systems
5.3.1 Direct detection
5.3.2 Coherent systems
5.3.2.1 Nonlinear phase noise channel
5.3.2.2 Perturbation models (NLIN and GN)
5.3.2.3 Split-step Fourier method (SSFM)
5.4 Gradient-free end-to-end learning
5.5 Conclusion
Acknowledgments
References
6 Deep learning techniques for optical monitoring
6.1 Introduction
6.2 Building blocks of deep learning-based optical monitors
6.2.1 Digital coherent reception as a data-acquisition method
6.2.2 Deep learning and representation learning
6.2.3 Combination of digital coherent reception and deep learning
6.3 Deep learning-based optical monitors
6.3.1 Training mode of DL-based optical monitors
6.3.2 Advanced topics for the training mode of DL-based optical monitors
6.3.2.1 Data augmentation based on domain knowledge of optical communication
6.3.2.1.1 Data augmentation on polarization state
6.3.2.1.2 Data augmentation on the frequency offset
6.3.2.2 Transfer learning for adaptation of DNNs
6.3.2.3 Federated learning for collaborative DNN training over multiple operators
6.3.3 Inference mode of DL-based optical monitors
6.3.4 Advanced topics for inference modes of DL-based optical monitors
6.3.4.1 Cloud-based vs. edge-based implementations
6.3.4.1.1 Cloud-based implementation of inference mode
6.3.4.1.2 Edge-based implementation of inference mode
6.3.4.2 Estimating the model uncertainty in inference mode
6.4 Tips for designing DNNs for DL-based optical monitoring
6.4.1 Shallow vs. deep network
6.4.2 DNN architecture for optical monitoring
6.4.2.1 Fully-connected DNNs
6.4.2.2 Convolutional neural networks
6.4.2.3 DNN architecture for the optical monitoring
6.5 Experimental verifications
6.5.1 Experimental setup for data collection
6.5.2 Neural network architecture for OSNR estimation task
6.5.2.1 DNN used in this experiment
DNN #1 (FC-DNN):
DNN #2 (CNN-1):
6.5.2.2 Results and discussion
6.5.3 Detailed experimental evaluation of CNN-based OSNR estimators
6.5.3.1 DNN used in this section
DNN #3 (CNN-2):
6.5.3.2 Results and discussion
6.5.4 Versatile monitoring using DNN
6.5.4.1 DNN architecture used in this experiment
DNN #4 (CNN-3):
6.5.4.2 Results and discussion
6.5.5 Data augmentation based on domain knowledge of optical transceivers
6.5.5.1 DNN used in this section
DNN #5 (CNN-4):
6.5.5.2 Results and discussion
6.5.6 Estimating uncertainty by dropout at inference
6.5.6.1 DNN used in this experiment
DNN #6 (CNN-5):
6.5.6.2 Results and discussion
6.6 Future direction of data-analytic-based optical monitoring
6.7 Summary
Acknowledgment
References
7 Machine Learning methods for Quality-of-Transmission estimation
7.1 Introduction
7.2 Classification and regression models for QoT estimation
7.2.1 Classification approaches for QoT estimation
7.2.1.1 Performance evaluation metrics - ML classification
7.2.1.2 Illustrative description of a classifier for QoT estimation
7.2.2 Regression approaches for QoT estimation
7.2.2.1 Regression models for QoT estimation
7.3 Active and transfer learning approaches for QoT estimation
7.3.1 Active learning
7.3.1.1 Gaussian Processes for QoT estimation
7.3.2 Transfer learning
7.3.2.1 Domain adaptation techniques
7.3.3 When to apply AL/DA during network lifecycle
7.4 On the integration of ML in optimization tools
7.4.1 RMSA integrating ML-based QoT estimation in EONs
7.4.1.1 Integrated network planning framework
7.5 Illustrative numerical results
7.5.1 Data generation
7.5.2 Classification
7.5.3 Regression
7.5.4 Active learning and transfer learning
7.6 Future research directions and challenges
7.7 Conclusion
References
8 Machine Learning for optical spectrum analysis
8.1 Introduction
8.1.1 Failure detection and localization
8.1.2 Optical spectrum
8.1.3 Failures affecting the optical spectrum
8.2 Feature-based spectrum monitoring
8.2.1 Motivation and objectives
8.2.2 OSA for soft-failure detection and identification
8.2.2.1 Soft-failure detection, identification, and localization
8.2.2.2 Options for classification using FeX
Multi-classifier approach
Single-classifier approach
Feature transformation for single-classifier approach
8.2.3 Soft-failure localization
8.2.4 Illustrative results
8.2.4.1 VPI set-up for data collection
8.2.4.2 ML-based classification comparison
8.2.4.3 Benefits of using a single OSA
8.2.4.4 Benefits of feature transformation for classification
8.2.4.5 Failure localization
8.2.5 Conclusions
8.3 Residual-based spectrum monitoring
8.3.1 Residual-based approach for optical spectrum analysis
8.3.2 Facilitating ML algorithm deployment using residual signals
8.3.3 Illustrative results
8.3.3.1 Comparison of residual-based and feature-based approaches
8.3.3.2 The efficiency of residual adaptation mechanism
8.3.4 Conclusions
8.4 Monitoring of filterless optical networks
8.4.1 Motivation of optical monitoring in FONs
8.4.2 Signal identification and classification
8.4.3 Optical signal tracking
8.4.3.1 Feature-based tracking
Individual feature
Super features
8.4.3.2 Residual-based tracking
8.4.4 Illustrative results
8.4.4.1 PAM4 scenario
8.4.4.2 QPSK scenario
8.4.5 Conclusions
8.5 Concluding remarks and future work
List of acronyms
References
9 Machine learning and data science for low-margin optical networks
9.1 The shape of networks to come
9.2 Current QoT margin taxonomy and design
9.3 Generalization of optical network margins
9.3.1 Optimal spectral efficiency
9.3.2 Field margins
9.3.3 Uncertainty margins
9.3.4 Unallocated and implementation margins
9.3.5 Protection margins
9.3.6 Total spectral efficiency margin and QoT margin equivalency
9.4 Large scale assessment of margins and their time variations in a deployed network
9.4.1 Assessing the quality of transmission
9.4.2 Description of the dataset
9.4.3 Example of SNR variations in time
9.4.4 Distributions of the minimal, maximal and median margins for all connections in the dataset
9.4.5 System margins and long term performance variations
9.4.6 Distribution of long term SNR variations
9.5 Trade-off between capacity and availability
9.5.1 Method and definitions
9.5.2 Case study: impact of a rate upgrade on availability for ``channel 112''
9.5.3 Setting margins based on availability estimations
9.5.4 Margins and network throughput
9.5.5 Capacity-availability trade-off and margin sweet spot
9.6 Data-driven rate adaptation for automated network upgrades
9.6.1 Memoryless, dynamic rate adaptation (M-D)
9.6.2 Causal dynamic rate (C-D) rate adaptation
9.7 Machine learning for low-margin optical networks
9.7.1 Solutions to emulate optical networks
9.7.2 Pitfalls when assessing the benefits of machine learning in optical network applications
9.7.3 Optimally using ML to solve open optical network problems
9.7.3.1 The quest for the best rate adaptation mechanism
9.7.3.2 The quest for the best QoT estimation method
9.7.3.3 The quest for the best QoT optimization
9.7.3.4 The quest for an holistic network optimization of margins
9.8 Conclusion
References
10 Machine learning for network security management, attacks, and intrusions detection
10.1 Physical layer security management
10.1.1 Physical-layer attacks
10.1.2 Attack management framework
10.2 Machine learning techniques for security diagnostics
10.2.1 Experimental setup and data collection
10.2.2 ML models and their configuration
10.2.2.1 Artificial Neural Network (ANN)
10.2.2.2 Density-Based Spatial Clustering of Applications with Noise (DBSCAN)
10.2.2.3 One-Class Support Vector Machine (OCSVM)
10.3 Accuracy of ML models in threat detection
10.3.1 ANN accuracy
10.3.2 DBSCAN and OCSVM accuracy
10.3.3 Window-based Attack Detection (WAD)
10.4 Runtime complexity of ML models
10.5 Interpretability of ML models
10.6 Open challenges
10.7 Conclusion
Acknowledgments
References
11 Machine learning for design and optimization of photonic devices
11.1 Introduction
11.2 Deep neural network (DNN) models
11.3 Nanophotonic power splitter
11.3.1 Device structure
11.3.2 Simulation and DNN modeling procedures
11.3.3 Deep learning for forward modeling to predict optical response
11.3.4 Deep learning for inverse modeling to construct device topology
11.3.5 Deep learning for generative modeling to produce device topology candidates
11.3.6 Nanophotonic power splitter experiment
11.3.7 Comparison with other optimization methods
11.4 Metasurfaces and plasmonics
11.4.1 Deep learning for forward modeling
11.4.2 Deep learning for inverse modeling
11.4.3 Deep learning for generative modeling
11.5 Other types of optical devices
11.5.1 Deep learning for forward modeling
11.5.2 Deep learning for generative modeling
11.6 Discussion
11.7 Conclusion
References
Index
Back Cover