Systems Engineering and Artificial Intelligence

✍ Scribed by William F. Lawless (editor), Ranjeev Mittu (editor), Donald A. Sofge (editor), Thomas Shortell (editor), Thomas A. McDermott (editor)

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 566
• Edition: 1st ed. 2021
• Category: Library

✦ Synopsis

This book provides a broad overview of the benefits from a Systems Engineering design philosophy in architecting complex systems composed of artificial intelligence (AI), machine learning (ML) and humans situated in chaotic environments.   The major topics include emergence, verification and validation of systems using AI/ML and human systems integration to develop robust and effective human-machine teams―where the machines may have varying degrees of autonomy due to the sophistication of their embedded AI/ML.  The chapters not only describe what has been learned, but also raise questions that must be answered to further advance the general Science of Autonomy. 

The science of how humans and machines operate as a team requires insights from, among others, disciplines such as the social sciences, national and international jurisprudence, ethics and policy, and sociology and psychology. The social sciences inform how context is constructed, how trust is affected when humans and machines depend upon each other and how human-machine teams need a shared language of explanation. National and international jurisprudence determine legal responsibilities of non-trivial human-machine failures, ethical standards shape global policy, and sociology provides a basis for understanding team norms across cultures. Insights from psychology may help us to understand the negative impact on humans if AI/ML based machines begin to outperform their human teammates and consequently diminish their value or importance. This book invites professionals and the curious alike to witness a new frontier open as the Science of Autonomy emerges.

✦ Table of Contents

Preface
Contents
1 Introduction to “Systems Engineering and Artificial Intelligence” and the Chapters
1.1 Introduction. The Disruptive Nature of AI
1.1.1 Justifying Speedy Decisions
1.1.2 Systems Engineering (SE)
1.1.3 Common Ground: AI, Interdependence, and SE
1.1.4 Social Science
1.1.5 The Science of Human Teams
1.1.6 Human–Machine Teams
1.2 Introduction to the Chapters
1.3 Summary
References
2 Recognizing Artificial Intelligence: The Key to Unlocking Human AI Teams
2.1 Introduction
2.1.1 Motivation and Goals
2.1.2 Types of Human-AI Collaboration
2.1.3 Ground Rules
2.2 System Engineering
2.2.1 Design and Embodiment
2.2.2 Generative Language Models
2.2.3 System Architecture
2.2.4 Agile Development
2.3 Applications
2.3.1 Ideation Discussions
2.3.2 Collaborative Writing
2.4 Innovative Brainstorm Workshop
2.4.1 Protocol
2.4.2 Analysis
2.4.3 Preliminary Results
2.5 Related Work
2.6 Future Applications
2.7 Conclusion
References
3 Artificial Intelligence and Future of Systems Engineering
3.1 Introduction
3.2 SERC AI4SE and SE4AI Roadmap
3.3 Digital Engineering
3.4 AI/ML Technology Evolution
3.5 Augmented Engineering
3.6 Workforce and Culture
3.7 Summary—The AI imperative for Systems Engineering
References
4 Effective Human–Artificial Intelligence Teaming
4.1 Introduction
4.2 Synthetic Teammates
4.3 HAT Findings and Their Implications for Human Teams
4.4 Conclusions and Future Work
References
5 Toward System Theoretical Foundations for Human–Autonomy Teams
5.1 Introduction
5.2 Organizational Structure and Role/Function Allocation
5.3 Working Together on Tasks
5.4 Teaming Over Longer Durations
5.5 Formally Modeling and Composing Complex Human–Machine Systems
5.6 Conclusions and Future Directions
References
6 Systems Engineering for Artificial Intelligence-based Systems: A Review in Time
6.1 Perspectives on AI and Systems Engineering
6.2 The Dynamics of This Space
6.2.1 Evolving an SE Framework: Ontologies of AI/ML—Dealing with the Breadth of the Fields
6.2.2 Systems Engineering as a Moving Target
6.2.3 The First to Market Motivation
6.2.4 Technical Debt
6.2.5 Summary
6.3 Stepping Through Some Systems Engineering Issues
6.3.1 Capability Maturity Model Integration [CMMI] and SE for R&D
6.3.2 Requirements Engineering
6.3.3 Software Engineering for AI/ML Systems
6.3.4 Test and Evaluation
6.4 Sampling of Technical Issues and Challenges
6.4.1 Emergence and Emergent Behavior
6.4.2 Safety in AI/ML
6.4.3 The Issue of Explanation/Explainability
6.5 Summary
References
7 Human-Autonomy Teaming for the Tactical Edge: The Importance of Humans in Artificial Intelligence Research and Development
7.1 Introduction
7.2 The Fundamental Nature of Human-Autonomy Teaming
7.2.1 Complementarity of Human and AI Characteristics
7.2.2 Tracking the Important Roles of the Human Across AI History
7.3 Artificial Intelligence for Human-Autonomy Teams
7.3.1 Quantifying Soldier Understanding for AI
7.3.2 Soldier-Guided AI Adaptations
7.3.3 Characterizing Soldier-Autonomy Performance
7.4 Conclusions
References
8 Re-orienting Toward the Science of the Artificial: Engineering AI Systems
8.1 Introduction
8.2 AI Software Engineering
8.3 AI-enabled Complex Systems-of-Systems and Emergent Behaviors
8.4 The Importance of Interoperability
8.5 The Role of Uncertainty in ML
8.6 The Challenge of Data and ML: An NLP Example
8.6.1 System Architecture
8.6.2 Results
8.6.3 Discussion
8.7 Design Science: Toward the Science of AI System Engineering
8.8 Conclusion
References
9 The Department of Navy’s Digital Transformation with the Digital System Architecture, Strangler Patterns, Machine Learning, and Autonomous Human–Machine Teaming
9.1 Introduction
9.2 Autonomous Human–Machine Teaming Lifecycle Difficulties
9.3 Unique Challenges Facing the Department of Navy and Autonomous Human–Machine Teaming
9.3.1 Department of Navy Non-technical Challenges
9.3.2 Department of Navy Technical Challenges
9.4 Attacking the Technical Debt and Inflation to Enable AHMT Solutions
9.4.1 AHMT Solutions and New Target Platforms
9.4.2 AHMT Solutions and Legacy Target Platforms
9.5 Conclusion and Path Forward
References
10 Digital Twin Industrial Immune System: AI-driven Cybersecurity for Critical Infrastructures
10.1 Introduction
10.1.1 Overview
10.1.2 Cybersecurity Technology Gaps for Advanced Detection, Protection and Monitoring Solutions
10.1.3 Digital Ghost: A Next-Generation Response to Close Critical Energy Infrastructure Gaps
10.2 People, Process and Technology Applicability Gap Analysis
10.2.1 Attack Detection
10.2.2 Attack Localization
10.2.3 Attack Neutralization
10.2.4 Man Versus Machine Anomaly Forecasting and Detection
10.3 Digital Ghost Research Findings and Future Research
10.3.1 Invariant Learning
10.3.2 Autonomous Defense: Critical Sensors Identification and Trust
10.3.3 Humble AI
10.3.4 Explainable AI (XAI)
10.4 Conclusion
References
11 A Fractional Brownian Motion Approach to Psychological and Team Diffusion Problems
11.1 Introduction
11.2 Random Walk
11.2.1 Wiener Process from the Fair Simple Random Walk
11.2.2 Wiener Process (standard Brownian Motion) Defined
11.2.3 Simulation of the Wiener Process via G0,1n
11.2.4 Continuity of Sample Paths
11.2.5 Non-differentiability of Wiener Process Sample Paths
11.3 Brownian Motion
11.3.1 Simulation of Brownian Motion
11.4 Stopping Times and Absorbing Boundaries
11.4.1 Two Absorbing Boundaries—The Situation for Ratcliff Drift Diffusion
11.5 Fractional Brownian Motion
11.5.1 Covariance of Brownian Motion
11.5.2 Definition of the Fractional Wiener Process
11.5.3 Existence and Properties of the Fractional Wiener Process
11.5.4 Ratcliff Diffusion Revisited
11.6 Determining H, a Problem in AI
11.6.1 Our Hybrid Approach
11.7 Team Science and Future Work
References
12 Human–Machine Understanding: The Utility of Causal Models and Counterfactuals
12.1 Introduction
12.2 Information-Theoretic Framework for SCM Construction
12.3 Assessing and Correcting for Bias in Information-Theoretic SCM Construction
12.4 Construction of SCM for Counterfactuals
12.5 Notes on Related Work
12.6 Summary
References
13 An Executive for Autonomous Systems, Inspired by Fear Memory Extinction
13.1 The Problem
13.2 Moondoodya, a Novel Electronic Warfare System
13.3 PTSD Fear Extinction
13.4 A Mathematical Approach to Executive Abstraction
13.5 ‘Effect First’ Modelling
13.6 A Closure Embedding Strategy
13.7 The Tookoonooka Vortex Collaborative
13.8 Conclusions
References
14 Contextual Evaluation of Human–Machine Team Effectiveness
14.1 Introduction
14.2 Related Works
14.3 Background
14.3.1 Interference
14.3.2 Inverse Reinforcement Learning (IRL)
14.3.3 Preferential Trajectory-Based IRL (PT-IRL)
14.4 Approach
14.4.1 Experimental Setup
14.4.2 Training Classifier
14.4.3 Human and Human–Machine Teams
14.4.4 Evaluation of Human–Machine Team Effectiveness
14.5 Conclusion and Future Work
References
15 Humanity in the Era of Autonomous Human–machine Teams
15.1 Introduction: AHMTs in the Form of the Trio
15.1.1 The Trio: Data, the Internet, and Algorithms
15.1.2 AHMTs Manifested by the Trio
15.1.3 Scitovsky’s Caveat
15.2 Human–Machine Teams
15.2.1 Shelley Model: Frankenstein and His Creature
15.2.2 Lovelock Model: GAIA and Novacene
15.2.3 Margulis Model: Symbiogenesis and Super Cooperators
15.2.4 Polanyi Model: Tension Between Habitation and Improvement
15.2.5 Laloux Model: Soulful Organizations
15.3 Meaning of the Trios for Humanity
15.3.1 Co-evolutions of Humans and Machines
15.3.2 Individuality
15.3.3 Democratization of Individuality
15.4 Meaning of the Trio for the Humanities
15.4.1 Distant Reading
15.4.2 Extended Reading
15.4.3 Participatory Reading
15.5 Concluding Remarks
References
16 Transforming the System of Military Medical Research: An Institutional History of the Department of Defense’s (DoD) First Electronic Institutional Review Board Enterprise IT System
16.1 Introduction. A Tale of Two Histories
16.1.1 Goal 1: The eIRB Transformed the MEDCENs
16.1.2 Goal 2: The Initial Meeting on Collaboration
16.1.3 Our Two Goals Merged into One
16.2 The Next Steps in the Transformation from a Paper to Electronic System
16.3 Boundary Maintenance
16.4 Future Steps to Determine Impacts. Preliminary Results in 2010
16.5 Summary
16.6 Postscript
References
17 Collaborative Communication and Intelligent Interruption Systems
17.1 Introduction
17.2 Interruptions in Multi-user Multitasking Interactions
17.2.1 Low Cognitive Interruption Timings
17.2.2 High Cognitive Interruption Timings
17.3 Methods
17.3.1 Data Collection
17.3.2 Conditions
17.4 Results and Discussion
17.4.1 Team Performance Analyses
17.4.2 Individual Subjective Analyses
17.4.3 Individual Interruption Task Measures
17.5 Discussion
17.6 Conclusion
References
18 Shifting Paradigms in Verification and Validation of AI-Enabled Systems: A Systems-Theoretic Perspective
18.1 Introduction
18.2 A Need for a Paradigm Shift in V&V
18.3 A Systems-Theoretic Interpretation of Intelligence
18.4 Challenges to the V&V of AI-Enabled Systems
18.4.1 Differential Learning in V&V Versus Operational Environment
18.4.2 Endogenous Evolution of Systems
18.4.3 Verification of Learning to Learn
18.4.4 Encapsulation of Intelligent Properties
18.5 Conclusion
References
19 Toward Safe Decision-Making via Uncertainty Quantification in Machine Learning
19.1 Introduction
19.2 Decision-Making and Machine Learning
19.2.1 Summary of a ML-augmented Decision-Making Process
19.2.2 Uncertainty Quantification as Part of the Decision-Making Process
19.3 Bayesian Inference
19.3.1 Bayesian Neural Networks
19.3.2 The Predictive Distribution
19.4 Making Decisions in the Presence of Uncertainty: Bayesian Decision Theory
19.5 A Case Study: Vehicle Classification from Acoustic Sensors
19.5.1 The Data Set
19.5.2 The Neural Network Architecture
19.5.3 Inference Approach
19.5.4 The Decision-Making Task: Avoiding Catastrophic Failure
19.5.5 Overall Results
19.5.6 Calibration of the Cost Function
19.6 Resource Requirements of Bayesian Inference
19.7 Conclusion
References
20 Engineering Context from the Ground Up
20.1 Introduction
20.2 Information Processing Architecture
20.3 Memory Import and Storage
20.4 Natural Language Processing
20.5 Confidence Aggregator
20.6 Perspective Transformation
20.7 Planning Unit
20.8 Communication Unit
20.9 Command Unit
20.10 Conclusions: Engineering Context
References
21 Meta-reasoning in Assembly Robots
21.1 Introduction and Background
21.1.1 Related Work
21.2 Illustrative Examples
21.3 Assembly as a Reasoning Problem
21.4 Failure Mode, Effect, and Repair Analysis
21.4.1 Skill-level Failures
21.4.2 Task-level Failures
21.4.3 Mission-level Failures
21.4.4 A Preliminary Taxonomy of Failure Modes
21.5 Assembly Plan Repair as a Meta-reasoning Problem
21.5.1 Meta-reasoning Architecture for Robots
21.5.2 Skill Failure Detection and Repair
21.5.3 Task Failures and Repairs
21.5.4 Toward Mission Repair
21.6 Discussion and Conclusions
References
22 From Informal Sketches to Systems Engineering Models Using AI Plan Recognition
22.1 Motivation
22.2 Related Work
22.2.1 Natural Sketching
22.2.2 Artificial Intelligence
22.3 Plan Recognition Approach
22.3.1 Approach Overview
22.3.2 Tolerance to Drawing Imperfections
22.4 Implementation
22.4.1 Automated Deterministic Planning
22.4.2 Modeling Sketches
22.5 Experiment
22.6 Conclusion
References
23 An Analogy of Sentence Mood and Use
23.1 Helpful Misalignment
23.2 Theorizing the Relation
23.3 Setters and Indicators
23.4 Schemes and Force
23.4.1 Practical Reasoning
23.4.2 Analogy
References
24 Effective Decision Rules for Systems of Public Engagement in Radioactive Waste Disposal: Evidence from the United States, the United Kingdom, and Japan
24.1 Introduction
24.2 Literature Review: Decision Rules that Encourage Public Engagement
24.3 Empirical Analysis of Public Engagement and Decision Rules
24.3.1 United States
24.3.2 United Kingdom: The “Participatory Turn” and Its Consequences
24.3.3 Japan: Public Interest in Participatory Approach to GDF Siting
24.4 Conclusion
References
25 Outside the Lines: Visualizing Influence Across Heterogeneous Contexts in PTSD
25.1 Introduction
25.2 Defining and Representing Context
25.2.1 Defining Narrative
25.2.2 Surrounding Literature
25.3 PTSD Example
25.4 Representations
25.4.1 Visual Grammar: Taxonomy
25.4.2 Visual Grammar: Dynamic Operations
25.4.3 Technical Foundations
25.5 Examples
25.5.1 Four Multi-disciplinary Models
25.5.2 Integration of Multiple Models: Model of Models
25.5.3 Zooming
25.5.4 Sandwich Layers
25.5.5 Signature Structures
25.5.6 Media
25.6 Discussion
25.6.1 Readability
25.6.2 Dynamism
25.6.3 Future Applications
25.7 Conclusion
References