Imaging and Sensing for Unmanned Aircraft Systems: Control and Performance (Control, Robotics and Sensors)

✍ Scribed by Vania V. Estrela (editor), Jude Hemanth (editor), Osamu Saotome (editor), George Nikolakopoulos (editor), Roberto Sabatini (editor)

• Publisher: The Institution of Engineering and Technology
• Year: 2020
• Tongue: English
• Leaves: 362
• Series: Control, Robotics and Sensors
• Category: Library

✦ Synopsis

This two-volume book set explores how sensors and computer vision technologies are used for the navigation, control, stability, reliability, guidance, fault detection, self-maintenance, strategic re-planning and reconfiguration of unmanned aircraft systems (UAS).

Volume 1 concentrates on UAS control and performance methodologies including Computer Vision and Data Storage, Integrated Optical Flow for Detection and Avoidance Systems, Navigation and Intelligence, Modeling and Simulation, Multisensor Data Fusion, Vision in Micro-Aerial Vehicles (MAVs), Computer Vision in UAV using ROS, Security Aspects of UAV and Robot Operating System, Vision in Indoor and Outdoor Drones, Sensors and Computer Vision, and Small UAV for Persistent Surveillance.

Volume 2 focuses on UAS deployment and applications including UAV-CPSs as a Testbed for New Technologies and a Primer to Industry 5.0, Human-Machine Interface Design, Open Source Software (OSS) and Hardware (OSH), Image Transmission in MIMO-OSTBC System, Image Database, Communications Requirements, Video Streaming, and Communications Links, Multispectral vs Hyperspectral Imaging, Aerial Imaging and Reconstruction of Infrastructures, Deep Learning as an Alternative to Super Resolution Imaging, and Quality of Experience (QoE) and Quality of Service (QoS).

✦ Table of Contents

Cover
Contents
About the editors
Preface
1 Introduction to advances in UAV avionics for imaging and sensing
1.1 Basic concepts
1.2 Navigation and intelligence
1.3 Communications
1.4 Sensors
1.5 Computational aspects: image/video processing, computer graphics, modelling, and visualisation
1.6 Security, health, and standards
1.7 Applications
1.8 Book organization
References
2 Computer vision and data storage in UAVs
2.1 Introduction
2.1.1 Requirements
2.1.2 Root file system
2.1.3 Data logging
2.1.4 Cloud support and virtualisation
2.2 The architecture of the cloud-based UAV cyber-physical system
2.3 UAV needs versus memory use
2.3.1 Limitations of OVP
2.3.2 General solutions and their viability analysis
2.4 UAV data logging
2.5 Types of data logging
2.5.1 Requirements and recommended solutions
2.5.2 Internal RAM with SD
2.5.3 External RAM with SD
2.5.4 External flash memory
2.6 Discussion and future trends
2.6.1 UAV-based data storage
2.6.2 UAV-based data processing
2.6.3 Distributed versus centralised control
2.6.4 Impact of big data in UAV-CPSs
2.6.4.1 Infrastructure readiness
2.6.4.2 Complexity
2.6.4.3 Privacy
2.6.4.4 Barriers to BD processing in UAV-CPSs
2.6.5 Challenges related to privacy and the protection of personal information
2.6.6 Organisational and cultural barriers
2.7 Conclusions
References
3 Integrated optical flow for situation awareness, detection and avoidance systems in UAV systems
3.1 Introduction
3.2 Computer vision
3.2.1 Optical Flow
3.2.1.1 Methods based on the brightness gradient
3.2.1.2 Feature extractor algorithm
3.3 Optical flow and remote sensing
3.3.1 Aerial Triangulation
3.4 Optical flow and situational awareness
3.4.1 Detect and avoidance system
3.4.1.1 Perception
3.4.1.2 Comprehension
3.4.1.3 Projection
3.5 Optical flow and navigation by images
3.5.1 Egomotion
3.6 Case study: INS using FPGA
3.6.1 Architectural proposals
3.6.1.1 Control unit (CU)
3.6.1.2 Generation of time
3.6.1.3 Feature points detector
3.6.1.4 OF calculation
3.6.1.5 Input and output component
3.6.2 Integration INS/GPS/OF using a Kalman filter
3.7 Future trends and discussion
3.7.1 3D optical flow
3.7.2 Multispectral and hyperspectral images
3.8 Conclusion
References
4 Introduction to navigation and intelligence for UAVs relying on computer vision
4.1 Introduction
4.2 Basic terminology
4.2.1 Visual servoing
4.2.2 Visual odometry
4.2.3 Terrain-referenced visual navigation
4.3 Future trends and discussion
4.4 Conclusions
References
5 Modelling and simulation of UAV systems
5.1 Need for modelling and simulation
5.1.1 Control systems design
5.1.2 Operator training
5.1.3 Sub-system development and testing
5.2 History and adoption
5.2.1 Early aviation
5.2.2 First computerised simulations
5.2.3 Entry of UAVs into service
5.2.4 Commercial and consumer drones
5.3 Modelling of UAV dynamics
5.3.1 Model representation methods
5.3.1.1 Differential equations
5.3.1.2 State-space representation
5.3.2 Common reference frames
5.3.2.1 Inertial frame of reference
5.3.2.2 Earth-centre frames of reference
5.3.2.3 Navigation frame of reference
5.3.2.4 Body frames of reference
5.3.3 Representation of state variables
5.3.3.1 Euler angles
5.3.3.2 Rotation matrices
5.3.3.3 Quaternions
5.3.4 Deriving the system equations of motion
5.3.4.1 Conservation of momentum
5.3.4.2 Euler–Lagrange method
5.3.4.3 Newton–Euler recursive method
5.3.5 Flight physics models
5.3.5.1 Fixed-wing flight
5.3.5.2 Multi-rotors and VTOL
5.4 Flight dynamics simulation
5.4.1 Integration of the equations of motion
5.4.1.1 Euler method
5.4.1.2 Runga–Kutta methods
5.5 Conclusion
References
6 Multisensor data fusion for vision-based UAV navigation and guidance
6.1 Introduction
6.2 Data-fusion algorithms
6.2.1 Extended Kalman filter
6.2.2 Unscented Kalman filter
6.2.3 Integration architectures
6.3 Fusion of visual sensors
References
7 Vision-based UAV pose estimation
7.1 Introduction
7.2 INS–GNSS drawbacks
7.2.1 Inertial navigation systems
7.2.2 Global navigation satellites systems
7.3 Visual navigation: A viable alternative
7.4 Visual navigation strategies
7.4.1 Photogrammetry: Extracting pose information from images
7.4.2 Template matching
7.4.3 Landmark recognition
7.4.3.1 Knowing the exact landmark
7.4.3.2 Identifying the landmarks' classes
7.4.4 Visual odometry
7.4.5 Combination of methods
7.5 Future developments on visual navigation systems
7.6 Conclusion
References
8 Vision in micro-aerial vehicles
8.1 Introduction
8.1.1 Fixed-wing MAVs
8.1.1.1 Longitudinal dynamics
8.1.1.2 Lateral dynamic
8.1.2 Rotary-wing MAVs
8.1.3 Flapping-wing or biomimetic MAVs
8.1.4 Hybrid MAVs
8.2 Computer vision as a biological inspiration
8.3 The role of sensing in MAVs
8.3.1 Pose-estimation sensors
8.3.2 Environmental awareness sensors
8.3.3 Sonar ranging sensor
8.3.4 Infrared-range sensors
8.3.5 Thermal imaging
8.3.6 LIDAR
8.3.7 Cameras
8.4 Illumination
8.5 Navigation, pathfinding, and orientation
8.6 Communication and polarisation-inspired machine vision applications
8.6.1 Robot orientation and navigation
8.6.2 Polarisation-opponent sensors
8.7 CCD cameras and applications in machine vision
8.8 Error modelling of environments with uncertainties
8.9 Further work and future trends
8.9.1 MAV challenges
8.9.2 Proposed solutions for MAV design challenges
8.9.3 New frontiers in sensors
8.10 Conclusion
References
9 Computer vision in UAV using ROS
9.1 Introduction
9.2 Computer vision on ROS
9.3 Applications
9.3.1 OpenCV in ROS
9.3.1.1 Object detection
9.3.2 Visual navigation
9.3.2.1 Parallel tracking and mapping (PTAM)
9.3.2.2 ROS package –autonomous flight
9.3.2.3 tum ardrone GUI
9.3.2.4 PTAM UAV camera feed and navigation
9.3.3 Setting the drone state estimation node
9.3.3.1 Simple navigation
9.4 Future developments and trends in ROS
9.5 Conclusion
References
10 Security aspects of UAV and robot operating system
10.1 Introduction
10.2 Unmanned aerial vehicles
10.3 ROS basic concepts
10.4 Security UAV review
10.5 Security ROS review
10.6 UAV security scenarios
10.7 Security assessment on consumer UAV operation with ROS
10.8 Future trends
10.9 Conclusion
References
11 Vision in indoor and outdoor drones
11.1 Computer vision in unmanned aerial vehicles
11.1.1 Indoor environments
11.1.2 Outdoor environments
11.2 Other approaches handling both indoor and outdoor environments
11.3 Conclusion
References
12 Sensors and computer vision as a means to monitor and maintain a UAV structural health
12.1 Introduction
12.1.1 Case study: aeroelastic instability flutter phenomenon
12.2 Related work
12.2.1 Structural health monitoring
12.2.2 Computer vision for structural health
12.2.3 Flutter certification
12.2.4 Computer vision and in in-flight measurements: future trends
12.3 Signal processing on flutter certification
12.4 Experiments and results
12.4.1 Synthetic data
12.4.1.1 Model of the typical wing section
12.4.1.2 Pre-processing
12.4.1.3 Extraction of dynamic characteristics
12.4.1.4 Results for synthetic data
12.4.2 Wind tunnel experiment
12.4.2.1 Experiment description
12.4.2.2 Results for experimental data
12.5 Discussion
12.5.1 Computer vision
12.6 Final remarks
References
13 Small UAV: persistent surveillance made possible
13.1 Introduction
13.2 System view
13.2.1 System description
13.2.2 Hardware components
13.2.3 Components recommendation
13.3 Software components
13.3.1 Camera calibration
13.3.2 Image stitching
13.3.3 Stabilization
13.3.4 Background subtraction
13.3.5 Object tracking
13.3.6 Geo-location pointing
13.4 Future trends
13.5 Conclusion
References
14 Conclusions
Index
Back Cover