计算机视觉：一种现代的方法 英文版

Part I Image Formation and Image Models

1 CAMERAS

1.1 Pinhole Cameras

1.1.1 Perspective Projection

1.1.2 Affine Projection

1.2 Cameras with Lenses

1.2.1 Paraxial Geometric Optics

1.2.2 Thin Lenses

1.2.3 Real Lenses

1.3 The Human Eye

1.4 Sensing

1.4.1 CCD Cameras

1.4.2 Sensor Models

1.5 Notes

Problems

2 GEOMETRIC CAMERA MODELS

2.1 Elements of analytical Euclidean Geometry

2.1.1 Coordinate Systems and Homogeneous Coordinates

2.1.2 Coordinate System Changes and Rigid Transformations

2.2 Camera Parameters and the Perspective Projection

2.2.1 Intrinsic Parameters

2.2.2 Extrinsic Parameters

2.2.3 A Characterization of Perspective Projection Matrices

2.3 Affine Cameras and Affine Projection Equations

2.3.1 Affine Cameras

2.3.2 Affine Projection Equations

2.3.3 A Characterization of Affine Projection Matrices

2.4 Notes

Problems

3 GEOMETRIC CAMERA CALIBRATION

3.1 Least-Squares Parameter Estimation

3.1.1 Linear Least-Squares Methods

3.1.2 Nonlinear Least-Squares Methods

3.2 A Linear Approach to Camera Calibration

3.2.1 Estimation of the Projection Matrix

3.2.2 Estimation of the Intrinsic and Extrinsic Parameters

3.2.3 Degenerate Point Configurations

3.3 Taking Radial Distortion into Account

3.3.1 Estimation of the Projection Matrix

3.3.2 Estimation of the Intrinsic and Extrinsic Parameters

3.3.3 Degenerate Point Configurations

3.4 Analytical Photogrammetry

3.5 An Application:Mobile Robot Localization

3.6 Notes

Problems

4 RADIOMETRY-MEASURING LIGHT

4.1 Light in Space

4.1.1 Foreshortening

4.1.2 Solid Angle

4.1.3 Radiance

4.2 Light at Surfaces

4.2.1 Simplifying Assumptions

4.2.2 The Bidirectional Reflectance Distribution Function

4.2.3 Example:The Radiometry of Thin Lenses

4.3 Important Special Cases

4.3.1 Radiosity

4.3.2 directional Hemispheric Reflectance

4.3.3 Lambertian Surfaces and Albedo

4.3.4 Specular Surfaces

4.3.5 The Lambertian+Specular Model

4.4 Notes

Problems

5 SOURCES,SHADOWS,AND SHADING

5.1 Qualitative Radiometry

5.2 Sources and Their Effects

5.2.1 Radiometric Properties of Light Sources

5.2.2 Point Sources

5.2.3 Line Sources

5.2.4 Area Sources

5.3 Local Shading Models

5.3.1 Local Shading Models for Point Sources

5.3.2 Area Sources and Their Shadows

5.3.3 Ambient Illumination

5.4 Application:Photometric Stereo

5.4.1 Normal and Albedo from Many Views

5.4.2 Shape from Normals

5.5 Interreflections:Global Shading Models

5.5.1 An Interreflection Models

5.5.2 Solving for Radiosity

5.5.3 The Qualitative Effects of Interreflections

5.6 Notes

Problems

6 COLOR

6.1 The Physics of Color

6.1.1 Radiometry for Colored Lights:Spectral Quantities

6.1.2 The Color of Sources

6.1.3 The Color of Surfaces

6.2 Human Color Perception

6.2.1 Color Matching

6.2.2 Color Receptors

6.3 Representing Color

6.3.1 Linear Color Spaces

6.3.2 Non-linear Color Spaces

6.3.3 Spatial and Temporal Effects

6.4 A Model for Image Color

6.4.1 Cameras

6.4.2 A Model for Image Color

6.4.3 Application:Finding Specularities

6.5 Surface Color from Image Color

6.5.1 Surface Color Perception in People

6.5.2 Inferring Lightness

6.5.3 Surface Color from Finite-Dimensional Linear Models

6.6 Notes

Problems

Part II Early Vision:Just One Image

7 LINEAR FILTERS

7.1 Linear Filters and Convolution

7.1.1 Convolution

7.2 Shift Invariant Linear Systems

7.2.1 Discrete Convolution

7.2.2 Continuous Convolution

7.2.3 Edge Effects in Discrete Convolutions

7.3 Spatial Frequecny and Fourier Transforms

7.3.1 Fourier Transforms

7.4 Sampling and Aliasing

7.4.1 Sampling

7.4.2 Aliasing

7.4.3 Smoothing and Resampling

7.5 Filters as Templates

7.5.1 Convolution as a dot Product

7.5.2 Changing Basis

7.6 Technique:Normalized Correlation and Finding Patterns

7.6.1 Controlling the Television by Finding Hands by Normalized Correlation

7.7 Technique:Scale and Image Pyramids

7.7.1 The Gaussian Pyramid

7.7.2 Applications of Scaled Representations

7.8 Notes

Problems

8 EDGE DETECTION

8.1 Noise

8.1.1 Additive Stationary Gaussian Noise

8.1.2 Why Finite Differences Respond to Noise

8.2 Estimating Derivatives

8.2.1 Derivative of Gaussian Filters

8.2.2 Why Smoothing Helps

8.2.3 Choosing a Smoothing Filter

8.2.4 Why Smooth with a Gaussian?

8.3 Detecting Edges

8.3.1 Using the Laplacian to Detect Edges

8.3.2 Gradient-Based Edge Detectors

8.3.3 Technique:Orientation Representations and Corners

8.4 Notes

Problems

9 TEXTURE

9.1 Representing Texture

9.1.1 Extracting Image Structure with Filter Banks

9.1.2 Representing Texture Using the Statistics of Filter Outputs

9.2 Analysis(and Synthesis)Using Oriented Pyramids

9.2.1 The Laplacian Pyramid

9.2.2 Filters in the Spatial Frequency Domain

9.2.3 Oriented Pyramids

9.3 Application:Synthesizing Textures for Rendering

9.3.1 Homogeneity

9.3.2 Synthesis by Sampling Local Models

9.4 Shape from Texture

9.4.1 Shape from Texture for Planes

9.5 Notes

Problems

Part III Early Vision:Multiple Images

10 THE GEOMETRY OF MULTIPLE VIEWS

10.1 Two Views

10.1.1 Epipolar Geometry

10.1.2 The Calibrated Case

10.1.3 Small Motions

10.1.4 The Uncalibrated Case

10.1.5 Weak Calibration

10.2 Three Views

10.2.1 Trifocal Geometry

10.2.2 The Calibrated Case

10.2.3 The Uncalibrated Case

10.2.4 Estimation of the Trifocal Tensor

10.3 More Views

10.4 Notes

Problems

11 STEREOPSIS

11.1 Reconstruction

11.1.1 Image Rectification

11.2 Human Stereopsis

11.3 Binocular Fusion

11.3.1 Correlation

11.3.2 Multi-Scale Edge Matching

11.3.3 Dynamic Programming

11.4 Using More Cameras

11.4.1 Three Cameras

11.4.2 Multiple Cameras

11.5 Notes

Problems

12 AFFINE STRUCTURE FROM MOTION

12.1 Elements of Affine Geometry

12.1.1 Affine Spaces and Barycentric Combinations

12.1.2 Affine Subspaces and Affine Coordinates

12.1.3 Affine Transformations and Affine Projection Models

12.1.4 Affine Shape

12.2 Affine Structure and Motion from Two Images

12.2.1 Geometric Scene Reconstruction

12.2.2 Algebraic Motion Estimation

12.3 Affine Structure and Motion from Multiple Images

12.3.1 The Affine Structure of Affine Image Sequences

12.3.2 A Factorization Approach to Affine Structure from Motion

12.4 From Affine to Euclidean Images

12.4.1 Euclidean Constraints and Calibrated Affine Cameras

12.4.2 Computing Euclidean Upgrades from Multiple Views

12.5 Affine Motion Segmentation

12.5.1 The Reduced Row-Echelon Form of the Data Matrix

12.5.2 The Shape Interaction Matrix

12.6 Notes

Problems

13 PROJECTIVE STRUCTURE FROM MOTION

13.1 Elements of Projective Geometry

13.1.1 Projective Spaces

13.1.2 Projective Subspaces and Projective Coordinates

13.1.3 Affine and Projective Spaces

13.1.4 Hyperplanes and Duality

13.1.5 Cross-Ratios and Projective Coordinates

13.1.6 Projective Transformations

13.1.7 Projective Shape

13.2 Projective Structure and Motion from Binocular Correspondences

13.2.1 Geometric Scene Reconstruction

13.2.2 algebraic Motion Estimation

13.3 Projective Motion Estimation from Multilinear Constraints

13.3.1 Motion Estimation from Fundamental Matrices

13.3.2 Motion Estimation from Trifocal Tensors

13.4 Projective Structure and Motion from Multiple Images

13.4.1 A Factorization Approach to Projective Structure from Motion

13.4.2 Bundle Adjustment

13.5 From Projective to Euclidean Images

13.6 Notes

Problems

Part IV Mid-Level Vision

14 SEGMENTATION BY CLUSTERING

14.1 What Is Segmentation?

14.1.1 Model Problems

14.1.2 Segmentation as Clustering

14.2 Human Vision:Grouping and Gestalt

14.3 Applications:Shot Boundary Detection and Background Subtraction

14.3.1 Background Subtraction

14.3.2 Shot Boundary Detection

14.4 Image Segmentation by Clustering Pixels

14.4.1 Segmentation Using Simple Clustering Methods

14.4.2 Clustering and Segmentation by K-means

14.5 Segmentation by Graph-Theoretic Clustering

14.5.1 Terminology for Graphs

14.5.2 The Overall Approach

14.5.3 Affinity Measures

14.5.4 Eigenvectors and Segmentation

14.5.5 Normalized Cuts

14.6 Notes

Problems

15 SEGMENTATION BY FITTING A MODEL

15.1 The Hough Transform

15.1.1 Fitting Lines with the Hough Transform

15.1.2 Practical Problems with the Hough Transform

15.2 Fitting Lines

15.2.1 Line Fitting with Least Squares

15.2.2 Which Point Is on Which Line?

15.3 Fitting Curves

15.3.1 Implicit Curves

15.3.2 Parametric Curves

15.4 Fitting as a Probabilistic Inference Problem

15.5 Robustness

15.5.1 M-estimators

15.5.2 RANSAC

15.6 Example:Using RANSAC to Fit Fundamental Matrices

15.6.1 An Expression for Fitting Error

15.6.2 Correspondence as Noise

15.6.3 Applying RANSAC

15.6.4 Finding the distance

15.6.5 Fitting a Fundamental Matrix to Known Correspondences

15.7 Notes

Problems

16 SEGMENTATION AN FITTING USING PROBABILISTIC METHODS

16.1 Missing Data Problems,Fitting,and Segmentation

16.1.1 Missing Data Problems

16.1.2 The EM Algorithm

16.1.3 The EM Algorithm in the General Case

16.2 The EM Algorithm in Practice

16.2.1 Example:Image Segmentation,Revisited

16.2.2 Example:Line Fitting with EM

16.2.3 Example:Motion Segmentation and EM

16.2.4 Example:Using EM to Identify Outliers

16.2.5 Example:Background Subtraction Using EM

16.2.6 Example:EM and the Fundamental Matrix

16.2.7 Difficulties with the EM Algorithm

16.3 Model Selection:Which Model Is the Best Fit?

16.3.1 Basic Ideas

16.3.2 AIC-An Information Criterion

16.3.3 Bayesian Methods and Schwartz'BIC

16.3.4 Description Length

16.3.5 Other Methods for Estimating Deviance

16.4 Notes

Problems

17 TRACKING WITH LINEAR DYNAMIC MODELS

17.1 Tracking as an Abstract Inference Problem

17.1.1 Independence Assumptions

17.1.2 Tracking as Inference

17.1.3 Overview

17.2 Linear Dynamic Models

17.2.1 Drifting Points

17.2.2 Constant Velocity

17.2.3 Constant Acceleration

17.2.4 Periodic Motion

17.2.5 Higher Order Models

17.3 Kalman Filtering

17.3.1 The Kalman Filter for a 1D State Vector

17.3.2 The Kalman Update Equations for a General State Vector

17.3.3 Forward-Backward Smoothings

17.4 Data Association

17.4.1 Choosing the Nearest-Global Nearest Neighbours

17.4.2 Gating and Probabiistic Data Association

17.5 Applications and Examples

17.5.1 Vehicle Tracking

17.6 Notes

Problems

Part V High-Level Vision:Geometric Methods

18 MODEL-BASED VISION

18.1 Initial Assumptions

18.1.1 Obtaining Hypotheses

18.2 Obtaining Hypotheses by Pose Consistency

18.2.1 Pose Consistency for Perspective Cameras

18.2.2 Affine and Projective Camera Models

18.2.3 Linear Combinations of Models

18.3 Obtaining Hypotheses by Pose Clustering

18.4 Obtaining Hypotheses Using Invariants

18.4.1 Invariants for Plane Figures

18.4.2 Geometric Hashing

18.4.3 Invariants and Indexing

18.5 Verification

18.5.1 Edge Proximity

18.5.2 Similarity in Texture,Pattern,and Intensity

18.6 Application:Registration in Medical Imaging Systems

18.6.1 Imaging Modes

18.6.2 Applications of Registration

18.6.3 Geometric Hashing Techniques in Medical Imaging

18.7 Curved Surfaces and Alignment

18.8 Notes

Problems

19 SMOOTH SURFACES AND THEIR OUTLINES

19.1 Elements of Differential Geometry

19.1.1 Curves

19.1.2 Surfaces

19.2 Contour Geometry

19.2.1 The Occluding Contour and the Image Contour

19.2.2 The Cusps and Inflections of the Image Contour

19.2.3 Koenderink's Theorem

19.3 Notes

Problems

20 ASPECT GRAPHS

20.1 Visual Events:More Differential Geometry

20.1.1 The Geometry of the Gauss Map

20.1.2 Asymptotic Curves

20.1.3 The Asymptotic Spherical Map

20.1.4 Local Visual Events

20.1.5 The Bitangent Ray Manifold

20.1.6 Multilocal Visual Events

20.2 Computing the Aspect Graph

20.2.1 Step 1:Tracing Visual Events

20.2.2 Step 2:constructing the Regions

20.2.3 Remaining Steps of the Algorithm

20.2.4 An Example

20.3 Aspect Graphs and Object Localization

20.4 Notes

Problems

21 RANGE DATA

21.1 Active Range Sensors

21.2 Range Data Segmentation

21.2.1 Elements of Analytical Differential Geometry

21.2.2 Finding Step and Roof Edges in Range Images

21.2.3 Segmenting Range Images into Planar Regions

21.3 Range Image Registration and Model Acquisition

21.3.1 Quaternions

21.3.2 Registering Range Images Using the Iterative Closest-Point Method

21.3.3 Fusing Multiple Range Images

21.4 Object Recognition

21.4.1 Matching Piecewise-Planar Surfaces Using Interpretation Trees

21.4.2 Matching Free-Form Surfaces Using Spin Images

21.5 Notes

Problems

Part VI High-Level Vision:Probabilistic and Inferential Methods

22 FINDING TEMPLATES USING CLASSIFIERS

22.1 Classifiers

22.1.1 Using Loss to Determine Decisions

22.1.2 Overview:Methods for Building Classifiers

22.1.3 Example:A Plug-in Classifier for Normal Class-conditional Densities

22.1.4 Example:A Nonparametric Classifier Using Nearest Neighbors

22.1.5 Estimating and Improving Performance

22.2 Building classifiers from Class Histograms

22.2.1 Finding Skin Pixels Using a Classifier

22.2.2 Face Finding Assuming Independent Template Responses

22.3 Feature Selection

22.3.1 Principal Component Analysis

22.3.2 Identifying Individuals with Principal Components Analysis

22.3.3 Canonical Variates

22.4 Neural Networks

22.4.1 Key Ideas

22.4.2 Minimizing the Error

22.4.3 When to Stop Training

22.4.4 Finding Faces Using Neural Networks

22.4.5 Convolutional Neural Nets

22.5 The Support Vector Machine

22.5.1 Support Vector Machines for Linearly Separable Datasets

22.5.2 Finding Pedestrians Using Support Vector Machines

22.6 Notes

Problems

22.7 Appendix I:Backpropagation

22.8 Appendix II:Support Vector Machines for Datasets That Are Not Linearly Separable

22.9 Appendix III:Using Support Vector Machines with Non-Linear Kernels

23 RECOGNITION BY RELATIONS BETWEEN TEMPLATES

23.1 Finding Objects by Voting on Relations between Templates

23.1.1 Describing Image Patches

23.1.2 Voting and a Simple Generative Model

23.1.3 Probabilistic Models for Voting

23.1.4 Voting on Relations

23.1.5 Voting and 3D Objects

23.2 Relational Reasoning Using Probabilistic Models and Search

23.2.1 Correspondence and Search

23.2.2 Example:Finding Faces

23.3 Using Classifiers to Prune Search

23.3.1 Identifying Acceptable Assemblies Using Projected Classifiers

23.3.2 Example:Finding People and Horses Using Spatial Relations

23.4 Technique:Hidden Markov Models

23.4.1 Formal Matters

23.4.2 Computing with Hidden Markov Models

23.4.3 Varieties of HMMs

23.5 Application:Hidden Markov Models and Sign Language Understanding

23.5.1 Language Models:Sentences from Words

23.6 Application:Finding People with Hidden Markov Models

23.7 Notes

24 GEOMETRIC TEMPLATES FROM SPATIAL RELATIONS

24.1 Simple Relations between Object and Image

24.1.1 Relations for Curved Surfaces

24.1.2 Class-Based Grouping

24.2 Primitives,Templates,and Geometric Inference

24.2.1 Generalized Cylinders as Volumetric Primitives

24.2.2 Ribbons

24.2.3 What Can One Represent with Ribbons?

24.2.4 Linking 3D and 2D for Cylinders of Known Length

24.2.5 Linking 3D and Image Data Using Explicit Geometric Reasoning

24.3 Afterword:Object Recognition

24.3.1 The Facts on the Ground

24.3.2 Current Approaches to Object Recognition

24.3.3 Limitations

24.4 Notes

Problems

Part VII Applications

25 APPLICATION:FINDING IN DIGITAL LIBRARIES

25.1 Background:Organizing Collections of Information

25.1.1 How Well Does the System Work?

25.1.2 What Do Users Want?

25.1.3 Searching for Pictures

25.1.4 Structuring and Browsing

25.2 Summary Representations of the Whole Picture

25.2.1 Histograms and Correlograms

25.2.2 Textures and Textures of Textures

25.3 Representations of Parts of the Picture

25.3.1 Segmentation

25.3.2 Template Matching

25.3.3 Shape and Correspondence

25.3.4 Clustering and Organizing Collections

25.4 Video

25.5 Notes

26 APPLICATION:IMAGE-BASED RENDERING

26.1 Constructing 3D Models from Image Sequences

26.1.1 Scene Modeling from Registered Images

26.1.2 Scene Modeling from Unregistered Images

26.2 Transfer-Based Approaches to Image-Based Rendering

26.2.1 Affine View Synthesis

26.2.2 Euclidean View Synthesis

26.3 The Light Field

26.4 Notes

Problems

BIBLIOGRAPHY

INDEX