Ultrasound Mid-Air Haptics for Touchless Interfaces (Human–Computer Interaction Series)

✍ Scribed by Orestis Georgiou (editor), William Frier (editor), Euan Freeman (editor), Claudio Pacchierotti (editor), Takayuki Hoshi (editor)

• Publisher: Springer
• Year: 2022
• Tongue: English
• Leaves: 411
• Category: Library

✦ Synopsis

Over the last decade, ultrasound mid-air haptic technology has emerged and rapidly advanced to engage multidisciplinary scientific communities within and adjacent to the haptics and HCI fields. Additionally, this haptic technology has been adopted by a number of industry sectors (e.g., automotive, virtual reality, digital signage, neuroscience research) who appear keen to exploit its unique value proposition: the ability to deliver rich haptic sensations from a distance, without the need to touch, wear or hold anything in order to enhance touchless interfaces, novel applications, and experiences.
This book is the first, and currently the only one, that provides a comprehensive description of the technology, encapsulating almost all aspects relating to electronic prototyping, acoustics, haptics, psychology and perception, user experience and end-user HCI applications. Through its 18 chapters written by 30 expert co-authors, this book is therefore an excellent introduction to the technology for anyone coming from any of those fields. Specifically, the reader will benefit by getting a unique and multi-dimensional perspective on the state-of-the-art of this enabling haptic technology while also understanding its history, relevant best research practices, and an overview of the various open challenges and opportunities.

✦ Table of Contents

Foreword
Preface
Contents
Editors and Contributors
Introduction to Ultrasonic Mid-Air Haptic Effects
1 Introduction
2 Acoustic Radiation Pressure
2.1 Mathematical Expression
2.2 Derivation Based on Kinetic Theory of Gases
2.3 Diagonal Incidence
2.4 Acoustic Streaming
3 Phased Array Focusing
3.1 Focal Point
3.2 Grating Lobes
3.3 Multiple Focal Points
4 Vibrotactile Stimulation
5 Audible Sound Radiation
5.1 Self-demodulation
5.2 Movement of Focal Point
6 Conclusion
Appendix
A. Derivation of Spatial Distribution of Ultrasound on Focal Plane
References
User Experience and Mid-Air Haptics: Applications, Methods, and Challenges
1 Introduction
2 Applications
2.1 Automotive
2.2 Touchless Displays in Public Spaces
2.3 Augmented, Virtual, and Mixed Reality
2.4 Touchless Computer Interfaces in Hospitals
2.5 Neuroscience Research Studies
3 Design Guidelines for Effective Mid-Air Haptic Interfaces
3.1 Presence of Controls
3.2 System Status and Changes
3.3 Confirmation
3.4 Latency and Timing
3.5 Interaction Zone and Hand Positioning
3.6 Haptic Congruence
3.7 Improving Perceived Haptic Intensity
3.8 Shape Recognition
3.9 Haptic Switching Duration
3.10 Haptic Sensation Priority
3.11 Static or Moving?
3.12 Multimodal Feedback and Synthesis
3.13 Summary
4 Methods
4.1 Why Care About User Experience Design?
4.2 Establish Requirements
4.3 Design Alternatives
4.4 Prototype
4.5 Evaluate
5 Conclusion
References
Ultrasound Haptic Feedback for Touchless User Interfaces: Design Patterns
1 Introduction
2 Background
3 Ultrasound Haptic Design Patterns
3.1 Tracked Fingertips
3.2 Tracked Palm
3.3 Floating Screen
3.4 Forcefield
3.5 Object Outline
3.6 Motion Patterns
3.7 Special Effects
4 Discussion
4.1 Retrospective Look at Haptic Design
4.2 Selecting Design Patterns
5 Conclusion
References
Opportunities for Multisensory Mid-Air Interactions Featuring Ultrasound Haptic Feedback
1 Introduction
2 Mid-air Touch in an Emerging Contactless World
3 Opportunities for Multisensory Mid-air Haptics
3.1 Challenge 1: Mid-air Tactile Dimensions
3.2 Challenge 2: Multisensory Integration
3.3 Challenge 3: Agency and Responsibility
4 Discussion, Conclusions, and Future Directions
References
Augmenting Automotive Gesture Infotainment Interfaces Through Mid-Air Haptic Icon Design
1 Introduction
2 Formative Studies
3 Stimulus Creation: Ideation and Prototyping
4 Perceptual Optimisation
4.1 Instant Identification
4.2 Recognisability and Distinguishability
4.3 Instant Articulatory Directness
4.4 Deriving the Exemplar Haptic Icons
5 Mid-Air Haptic Icon Designs
5.1 Telephone Calling Features
5.2 Audio Features
5.3 Cabin Temperature Features
5.4 Seat Temperature (Heater and Cooler) Feature
5.5 Fan Speed Feature
5.6 Navigation Features
5.7 Home Screen (Landing Page) Feature
6 Contextual Validation
7 Conclusion
References
Ultrasound Mid-Air Tactile Feedback for Immersive Virtual Reality Interaction
1 Introduction
2 State of the Art and Challenges for Integrating UMH in Immersive VR
3 Integrating UMH Devices into the VR Ecosystem
3.1 Integrating with Input Devices
3.2 Integrating with Feedback Devices
3.3 Workspace Compatibility
4 Perception of UMH Stimuli
4.1 DOLPHIN: A Framework for Designing and Studying Perception of UMH Stimuli
4.2 Perceptual Thresholds
5 UMH Rendering
5.1 Rendering Local Shape
5.2 Rendering Stiffness
6 Conclusion and Perspectives
References
Multimodal Interaction with Mid-Air Haptics
1 Introduction
2 Multimodal Interfaces Using Ultrasound Haptics
2.1 Visual Feedback
2.2 Auditory Feedback
2.3 Touch Feedback
2.4 Summary
3 Use Cases
3.1 AirPiano: Enhancing Music Playing Experience in VR with Mid-Air Haptics
3.2 Refinity: Haptic-Hologram for Novel Shopping Experience
3.3 Mid-Air Thermo-tactile Feedback Using Ultrasound Haptics
3.4 TangibleData: Interactive Data Visualization with Mid-Air Haptics
4 Conclusion
References
Touchless Tactile Interaction with Unconventional Permeable Displays
1 Introduction
2 Unconventional Display Devices
2.1 Permeable Displays
2.2 Head-Mounted Displays
3 Touchless Tactile Interaction
3.1 Ultrasound Haptics
4 Touchless Tactile Permeable Displays
5 Discussion
6 Conclusions
References
Modulation Methods for Ultrasound Midair Haptics
1 Introduction
2 Amplitude Modulation (AM)
3 Lateral Modulation (LM)
4 Spatiotemporal Modulation (STM)
5 Hybrid and Variant Modulation Methods
6 Conclusion
References
Multiunit Phased Array System for Flexible Workspace
1 Large-Scale Phased Array
2 Advantages and Challenges of Large Arrays
2.1 Advantage: Resolution
2.2 Advantage: Lesser Sensitivity to Acoustic Saturation
2.3 Challenge: Directivity
2.4 Challenge: Wiring
2.5 Challenge: Phase Synchronization
3 Implementation of AUTD v3
3.1 Requirement 1: Phase Quantization and Clock Synchronization
3.2 Requirement 2: Refresh Rate and Data Bus Capacity
3.3 Implementation Using EtherCAT
4 AUTD v3 Driver Software
4.1 Driving Signal Generation via FPGA
4.2 Modulation
4.3 Silent Mode
5 Haptic Navigation
5.1 Haptic Pursuit
5.2 Virtual Handrail
5.3 Other Applications
6 Conclusion
References
Sound-Field Creation for Haptic Reproduction
1 Acoustic Field Reconstruction
1.1 Forward Problem of the Acoustic Field
1.2 Inverse Problem
2 Overview of Various Algorithms
2.1 Eigenmethod
2.2 Semidefinite Relaxation
2.3 Gerchberg–Saxton Algorithm
2.4 GS-PAT
2.5 Levenberg–Marquardt Algorithm
2.6 Combinatorial Optimization
3 Summary
References
Superimposing Visual Images on Mid-Air Ultrasonic Haptic Stimulation
1 Introduction
2 Realism of Experience Reproduced by Ultrasonic Haptic Displays
3 Wearable Versus Stationary Visual Display
4 Display of Mid-air Button: Providing a Response to Active Touch
5 Passive Haptic Stimulation with Projected Image
6 3D Stationary Haptic VR Object with Visual Image
7 3D Human–Human Interaction by HaptoClone
8 Haptic Interaction with Full 3D Computer Graphics
9 Integration with HMD
10 Summary and Future Possibilities
References
Ultrasound Mid-Air Haptic Feedback at the Fingertip
1 Introduction
2 Ghostrokes
2.1 Ultrasound Modulation Technique
2.2 Stroking Patterns
3 Experimental Evaluation
3.1 Procedure
3.2 Results
3.3 Discussion
4 Applications
4.1 Active Stroking Feedback with Virtual Tools
4.2 Passive Stroking Feedback from Virtual Objects
5 Conclusion
References
The Physical Principles of Arrays for Mid-Air Haptic Applications
1 Introduction
2 Modelling Ultrasonic Fields
2.1 A Simple Analytical Model of a Circular Focussed Emitter
2.2 Modelling a Phased Array
2.3 Implications of the Array Model
3 Theory of Acoustic Radiation Forces and Streaming
3.1 Acoustic Radiation Force on Large Objects
3.2 Plane Wave Interpretation of the Theory
4 Modelling Acoustic Streaming Flow
5 Comparison to Experiment
6 Effects of Rapid Phase Changes
7 Conclusions
References
Prototyping Airborne Ultrasonic Arrays
1 Introduction
2 Components
2.1 Emitters
2.2 Drivers
2.3 Signal Generation
3 Geometries
4 Simulation of the Emitted Field
5 Focusing Algorithms
5.1 Simple Focus
5.2 Multi-focus
5.3 Amplitude Images
6 Conclusion
References
Safety of High-Intensity Ultrasound
1 International Guidelines for (Ultrasound) Noise Exposure
1.1 Effects of Noise Exposure
1.2 Non-linear Acoustic Effects
1.3 Summary of Guidelines
2 Estimating Typical Exposure from Ultrasound Mid-Air Haptics
3 Recent Safety Research
3.1 Audiology
3.2 Subjective Effects
3.3 Ear Defenders
3.4 Heating and Mechanical Effects
4 Effects on Animals
4.1 Physiological Effects
4.2 Ultrasonic Pest Deterrents
4.3 Additional Considerations
5 Concluding Remarks
References
Ultrasound Exposure in Mid-Air Haptics
1 Introduction
2 Conventional Exposure Tolerance
2.1 Safety Standards
2.2 Frequency of Ultrasound
2.3 Historical Background
3 Perceptual/Harmful Thresholds
3.1 Tactile Sensation (Heat)
3.2 Auditory (Threshold Shift)
3.3 Auditory (Hearing)
3.4 Tactile Sensation (Pressure, Vibration, and Airflow)
4 Harmful Effect Thresholds Lower Boundaries
5 Upper Boundary of Sound Pressure Level of Airborne Ultrasounds at User’s Position
5.1 Nonlinear Absorption
5.2 Spherical Wave
5.3 Experiments and Results
6 Conclusion
References
Mid-Air Haptics: Future Challenges and Opportunities
1 Introduction
2 Hardware
3 Haptic Sensations and Rendering Algorithms
4 User Experience and Applications
5 Conclusions and Perspectives
References