Representations and Techniques for 3D Object Recognition and Scene Interpretation

✍ Scribed by Derek Hoiem, Silvio Savarese

• Publisher: Springer
• Year: 2011
• Tongue: English
• Leaves: 158
• Series: Synthesis Lectures on Artificial Intelligence and Machine Learning
• Edition: 1
• Category: Library

✦ Synopsis

One of the grand challenges of artificial intelligence is to enable computers to interpret 3D scenes and objects from imagery. This book organizes and introduces major concepts in 3D scene and object representation and inference from still images, with a focus on recent efforts to fuse models of geometry and perspective with statistical machine learning. The book is organized into three sections: (1) Interpretation of Physical Space; (2) Recognition of 3D Objects; and (3) Integrated 3D Scene Interpretation. The first discusses representations of spatial layout and techniques to interpret physical scenes from images. The second section introduces representations for 3D object categories that account for the intrinsically 3D nature of objects and provide robustness to change in viewpoints. The third section discusses strategies to unite inference of scene geometry and object pose and identity into a coherent scene interpretation. Each section broadly surveys important ideas from cognitive science and artificial intelligence research, organizes and discusses key concepts and techniques from recent work in computer vision, and describes a few sample approaches in detail. Newcomers to computer vision will benefit from introductions to basic concepts, such as single-view geometry and image classification, while experts and novices alike may find inspiration from the book's organization and discussion of the most recent ideas in 3D scene understanding and 3D object recognition. Specific topics include: mathematics of perspective geometry; visual elements of the physical scene, structural 3D scene representations; techniques and features for image and region categorization; historical perspective, computational models, and datasets and machine learning techniques for 3D object recognition; inferences of geometrical attributes of objects, such as size and pose; and probabilistic and feature-passing approaches for contextual reasoning about 3D objects and scenes. Table of Contents: Background on 3D Scene Models / Single-view Geometry / Modeling the Physical Scene / Categorizing Images and Regions / Examples of 3D Scene Interpretation / Background on 3D Recognition / Modeling 3D Objects / Recognizing and Understanding 3D Objects / Examples of 2D 1/2 Layout Models / Reasoning about Objects and Scenes / Cascades of Classifiers / Conclusion and Future Directions

✦ Table of Contents

Cover
Copyright Page
Title Page
Contents
Preface
Acknowledgments
Figure Credits
Interpretation of Physical Space from an Image
Background on 3D Scene Models
Theories of Vision
Depth and Surface Perception
A Well-Organized Scene
Early Computer Vision and AI
Modern Computer Vision
Single-view Geometry
Consequences of Projection
Perspective Projection with Pinhole Camera: 3D to 2D
3D Measurement from a 2D Image
Automatic Estimation of Vanishing Points
Summary of Key Concepts
Modeling the Physical Scene
Elements of Physical Scene Understanding
Elements
Physical Interactions
Representations of Scene Space
Scene-Level Geometric Description
Retinotopic Maps
Highly Structured 3D Models
Loosely Structured Models: 3D Point Clouds and Meshes
Summary
Categorizing Images and Regions
Overview of Image Labeling
Guiding Principles
Creating Regions
Choosing Features
Classifiers
Datasets
Image Features
Color
Texture
Gradient-based
Interest Points and Bag of Words
Image Position
Region Shape
Perspective
Summary
Examples of 3D Scene Interpretation
Surface Layout and Automatic Photo Pop-up
Intuition
Geometric Classes
Approach to Estimate Surface Layout
Examples of Predicted Surface Layout
3D Reconstruction using the Surface Layout
Make3D: Depth from an Image
Predicting Depth and Orientation
Local Constraints and Priors
Results
The Room as a Box
Algorithm
Results
Summary
Recognition of 3D Objects from an Image
Background on 3D Recognition
Human Vision Theories
The Geon Theory
2D-view specific templates
Aspect graphs
Computational theories by 3D alignment
Conclusions
Early Computational Models
Modeling 3D Objects
Overview
Single instance 3D category models
Single instance 2D view-template models
Single instance 3D models
Mixture of Single-View Models
2-1/2D Layout Models
2-1/2D Layout by ISM models
2-1/2D Layout by view-invariant parts
2-1/2D hierarchical layout models
2-1/2D Layout by discriminative aspects
3D Layout Models
3D Layout Models constructed upon 3D prototypes
3D Layout Models without 3D prototypes
Recognizing and Understanding 3D Objects
Datasets
Supervision and Initialization
Modeling, Learning and Inference Strategies
Examples of 2D 1/2 Layout Models
Linkage Structure of Canonical Parts
The view-morphing formulation
Supervision
View-morphing models
Learning the model
Detection and viewpoint classification
Results
Conclusions
Integrated 3D Scene Interpretation
Reasoning about Objects and Scenes
Objects in Perspective
Object Size
Appearance Features
Interaction Between Objects and Scene via Object Scale and Pose
Scene Layout
Occlusion
Summary
Cascades of Classifiers
Intrinsic Images Revisited
Intrinsic Image Representation
Contextual Interactions
Training and Inference
Experiments
Feedback-Enabled Cascaded Classification Models
Algorithm
Experiments
Summary
Conclusion and Future Directions
Bibliography
Authors' Biographies