Virtual and Augmented Reality (VR/AR): Foundations and Methods of Extended Realities (XR)

✍ Scribed by Ralf Doerner (editor), Wolfgang Broll (editor), Paul Grimm (editor), Bernhard Jung (editor)

• Publisher: Springer
• Year: 2022
• Tongue: English
• Leaves: 435
• Edition: 1st ed. 2022
• Category: Library

✦ Synopsis

This comprehensive textbook offers a scientifically sound and at the same time practical introduction to Virtual and Augmented Reality (VR/AR). Readers will gain the theoretical foundation needed to design, implement or enhance VR/AR systems, evaluate and improve user interfaces and applications using VR/AR methods, assess and enrich user experiences, and develop a deeper understanding of how to apply VR/AR techniques.

Whether utilizing the book for a principal course of study or reference reading, students of computer science, education, media, natural sciences, engineering and other subject areas can benefit from its in-depth content and vivid explanation. The modular structure allows selective sequencing of topics to the requirements of each teaching unit and provides an easy-to-use format from which to choose specific themes for individual self-study.

Instructors are provided with extensive materials for creating courses as well as a foundational text upon which to build their advanced topics. The book enables users from both research and industry to deal with the subject in detail so they can properly assess the extent and benefits of VR/AR deployment and determine required resources.

Technology enthusiasts and professionals can learn about the current status quo in the field of VR/AR and interested newcomers can gain insight into this fascinating world.  Grounded on a solid scientific foundation, this textbook, addresses topics such as perceptual aspects of VR/AR, input and output devices including tracking, interactions in virtual worlds, real-time aspects of VR/AR systems and the authoring of VR/AR applications in addition to providing a broad collection of case studies.

✦ Table of Contents

Foreword
Preface
Contents
Chapter 1: Introduction to Virtual and Augmented Reality
1.1 What Is VR/AR About?
1.1.1 The Perfect Virtual Reality
1.1.2 The Simulation of the World
1.1.3 Suspension of Disbelief
1.1.4 Motivation
1.2 What Is VR?
1.2.1 Technology-Centered Characterizations of VR
1.2.2 VR as an Innovative Kind of Human–Computer Interaction
1.2.3 Mental Aspects of the VR Experience
1.3 What Is AR?
1.4 Historical Development of VR and AR
1.5 VR Systems
1.6 AR Systems
1.7 Using the Book
1.7.1 Structure of the Book
1.7.2 Usage Instructions
1.7.3 Target Groups
Lecturers in the Field of VR/AR
Students
Users and Those Who Want to Become Users
The Technology-Savvy
1.8 Summary and Questions
Recommended Reading
References
Chapter 2: Perceptual Aspects of VR
2.1 Human Information Processing
2.2 Visual Perception
2.2.1 Stereo Vision
2.2.2 Perception of Space
2.3 Multisensory Perception
2.3.1 Auditory Perception
2.3.2 Haptic Perception
2.3.3 Proprioception and Kinaesthesia
2.3.4 Perception of Movement
2.3.5 Presence and Immersion
2.4 Phenomena, Problems, Solutions
2.4.1 Deviating Observation Parameters
2.4.2 Double Vision
2.4.3 Frame Cancellation
2.4.4 Vergence-Focus Conflict
2.4.5 Discrepancies in the Perception of Space
2.4.6 Discrepancies in the Perception of Movement
2.4.7 Cybersickness
2.4.8 Vertical Parallax Problem
2.5 Use of Perceptual Aspects
2.5.1 Salience
2.5.2 User Guidance
2.6 Summary and Questions
Recommended Reading
References
Chapter 3: Virtual Worlds
3.1 Introduction
3.1.1 Requirements on 3D Object Representations for Virtual Worlds
3.1.2 Creation of 3D Models
3.1.3 Preparation of 3D Models for VR/AR
3.1.4 Integration of 3D Models into VR/AR Runtime Environments
3.2 Scene Graphs
3.3 3D Objects
3.3.1 Surface Models
Polygonal Representations
Polygons
Polygon Meshes
Triangle Strips
3.3.2 Solid Models
Boundary Representations (B-Reps)
Primitive Instancing
3.3.3 Appearance
Materials
Textures
Shader
3.3.4 Optimization Techniques for 3D Objects
Simplification of Polygon Meshes
Level-of-Detail Techniques
Texture Baking
Billboards
3.4 Animation and Object Behavior
3.4.1 Keyframe Animation
3.4.2 Physics-Based Animation of Rigid Bodies
3.4.3 Object Behavior
3.4.4 Behavior and Animation in Scene Graphs
3.5 Light, Sound, Background
3.5.1 Light Sources
3.5.2 Sound
3.5.3 Backgrounds
3.6 Special Purpose Systems
3.6.1 Virtual Humans
3.6.2 Particle Systems
3.6.3 Terrain
3.6.4 Vegetation
3.7 Summary and Questions
Recommended Reading
References
Chapter 4: VR/AR Input Devices and Tracking
4.1 Fundamentals of Input Devices
4.2 Tracking Techniques
4.2.1 Acoustic Tracking
4.2.2 Magnetic Field-Based Tracking
4.2.3 Inertial Tracking
4.2.4 Laser-Based Tracking
4.2.5 Outdoor Position Tracking
4.3 Camera-Based Tracking
4.3.1 Marker-Based Methods
4.3.2 Tracking Using Black and White Markers
Use of Marker Tracking
Basic Operation
Intrinsic and Extrinsic Camera Parameters
4.3.3 Feature-Based Tracking Techniques
Geometry-Based Tracking
Other Feature-Based Tracking Techniques
4.3.4 Visual SLAM
4.3.5 Hybrid Tracking Techniques
Cloud-Based Tracking
Microsoft Hololens Tracking
4.4 Finger Tracking
4.5 Eye Tracking
4.5.1 Eye Movements
4.5.2 Methods
4.5.3 Functionality of an Eye Tracker
4.5.4 Calibration
4.5.5 Eye Tracking in Head-Mounted Displays
4.5.6 Remote Eye Tracker
4.6 Further Input Devices
4.6.1 3D Mouse
4.6.2 Mechanical Input Devices
4.6.3 Treadmills for Virtual Reality
4.7 Summary and Questions
Recommended Readings
References
Chapter 5: VR/AR Output Devices
5.1 Introduction
5.2 Basics of Visual Output
5.3 Head-Mounted Displays (HMDs)
5.3.1 VR Glasses
5.3.2 AR Glasses
5.3.3 General Characteristics and Properties of HMDs
5.3.4 Special HMDs
5.4 Stationary VR Systems
5.4.1 Single-Sided Displays
5.4.2 Multi-Sided Displays
5.4.3 Tiled Displays
5.4.4 Stereo Output Methods
5.5 Audio Output Devices
5.6 Haptic Output Devices
5.7 Summary and Questions
Recommended Reading
References
Chapter 6: Interaction in Virtual Worlds
6.1 Fundamentals of Human–Computer Interaction
6.2 System Control
6.3 Selection
6.3.1 Pointing in Virtual Worlds
6.3.2 Interaction Design
6.3.3 Examples of Selection Techniques
6.4 Manipulation of Objects
6.5 Navigation
6.5.1 Control Techniques for Traveling
6.5.2 Walking Technique for Natural Movement Control
6.5.3 Leaning Interfaces for Movement Control
6.5.4 Teleportation for Movement Control
6.5.5 Route Plan, Goal-Based and Guided Movement Techniques
6.5.6 Criteria for Navigation Techniques
6.6 Processes for the Design and Implementation of Interaction
6.6.1 Characteristics of VR/AR User Interfaces
6.6.2 Human-Centered Design of VR/AR Interactions
Analysis of the Context of Use
Specification of Requirements
Concept, Design and Implementation
Evaluation (Especially User Tests)
6.7 User Tests
6.8 Ethical, Social and Legal Aspects of VR/AR
6.9 Summary and Questions
Recommended Reading
References
Chapter 7: Real-Time Aspects of VR Systems
7.1 Latency in VR Systems
7.1.1 What Are the Requirements on Latency?
7.1.2 Where Do Latencies Actually Arise?
7.1.3 Is Latency in a VR System Constant?
7.1.4 What Are the Approaches to Determining Latency?
Latency Estimation from Datasheets
Measuring the Latency of Tracking Systems
Measuring End-to-End Latency
7.1.5 Summary of Latency
7.2 Efficient Collision Detection in Virtual Worlds
7.2.1 Bounding Volumes
Axis-Aligned Bounding Box (AABB)
Bounding Spheres
Oriented Bounding Boxes (OBBs)
Discrete-Oriented Polytopes (k-DOPs)
7.2.2 Bounding Volume Hierarchies and Space Partitioning Techniques
Bounding Volume Hierarchies (BVHs)
Space Partitioning Techniques
7.2.3 Collision Detection in Large Environments
Broad Phase Collision Detection
Narrow Phase Collision Detection
7.2.4 Summary and Advanced Techniques
7.3 Real-Time Rendering of Virtual Worlds
7.3.1 Algorithmic Strategies
View Volume Culling
Hierarchical View Volume Culling
Occlusion Culling
Backface Culling
Small Feature Culling
Portal Culling
Level of Detail (LOD)
7.3.2 Hardware-Related Strategies
Object Size
Indexing
Caching
Stripping (Triangle and Quadrilateral Strips)
Minimizing State Changes
7.3.3 Software Systems for Virtual Worlds
Scene Graph Systems
Game Engines
7.4 Summary and Questions
Recommended Reading
References
Chapter 8: Augmented Reality
8.1 Introduction
8.1.1 Getting Started
8.1.2 AR – An Overview
8.2 Registration
8.2.1 Geometric Registration
8.2.2 Photometric Registration
8.3 Visual Output
8.3.1 Handheld Devices
8.3.2 Projection-Based Output
8.3.3 Further Types of Spatial AR
8.3.4 AR Mirrors
8.4 Special AR Techniques
8.4.1 Head-Up Content
8.4.2 Occlusions and Phantom Objects
8.4.3 Crossfading Markers
8.4.4 Virtual Holes
8.4.5 X-Ray Vision
8.5 Special AR Interaction Techniques
8.5.1 Interaction by Navigation
8.5.2 Gaze-Based Interaction
8.5.3 Tangible User Interfaces
8.6 AR Applications
8.7 Diminished and Mediated Reality
8.7.1 Diminished Reality
8.7.2 Mediated Reality
8.8 Summary and Questions
Recommended Reading
References
Chapter 9: VR/AR Case Studies
9.1 Introduction and Overview
9.2 Using Virtual Reality for Design Processes in the Automotive Industry
9.3 AR/VR Revolutionizes Your In-Car Experience
9.4 VR-Based Service Training in the Life Sciences and Diagnostics Industry
9.5 Utilizing Augmented Reality for Visualizing Infrastructure
9.6 Enhancing the Spatial Design Process with CADwalk
9.7 The aixCAVE at RWTH Aachen University
9.8 Augmented Reflection Technology: Stroke Rehabilitation with XR
9.9 Collaborative Virtual Trainers in VR Applications
9.10 Virtual Patients: A Case Study from Research to Real-World Impact
9.11 Embodied Social XR for Teaching, Learning and Therapy
9.12 Virtual Reality for Teaching Literacy to Prisoners
References
References for Sect. 9.1
References for Sect. 9.2
References for Sect. 9.3
References for Sect. 9.5
References for Sect. 9.6
References for Sect. 9.7
References for Sect. 9.8
References for Sect. 9.9
References for Sect. 9.10
References for Sect. 9.11
References for Sect. 9.12
Chapter 10: Authoring of VR/AR Applications
10.1 Supporting Authors
10.2 Foundations of Authoring Software
10.2.1 Unity
How It Works
VR/AR Development with Unity
Summary
10.2.2 Unreal Engine
How It Works
Developing VR/AR Applications
Summary
10.2.3 AR Frameworks: ARCore and ARKit
Availability
Tracking and Mapping
Reconstruction
Estimation of Environment Light
Summary
10.3 Examples of the Creation of VR/AR Applications
10.3.1 Making of: Immersive VR Presentation of CAD Files with the Vive Cosmos in Unity
10.3.2 Making of: Interaction in VR Using the Vive Cosmos and Unreal Engine
10.3.3 Making of: An Application for the Microsoft HoloLens 2 with Unity
10.3.4 Making of: Basics for the Development of a Native ARCore Application for Android
10.4 Summary and Questions
Recommended Reading
References
Software, Online Documentation and Tutorials
Chapter 11: Mathematical Foundations of VR/AR
11.1 Vector Spaces
11.2 Geometry and Vector Spaces
11.3 Points and Affine Spaces
11.4 Euclidean Space
11.5 Analytical Geometry in ℝ3
11.6 Matrices
11.7 Affine Transformations
11.8 Determination of Transformation Matrices
About the Authors
Index