methods [1]. This paper focuses on a generalization of the method proposed by Horn and Schunck for the estimation This paper presents a new approach based on the differential framework proposed by Horn and Schunck, to the problem of of optical flow, which is a differential-based method [2].
recursive optical flow estimation from image sequences. The
The differential framework methods start with a brightoriginal method of Horn and Schunck is applicable only to the ness constraint equation which forms a single linear equaproblem of estimating the optical flow between a pair of images tion per each pixel, constraining its motion vector [1,2]. from an image sequence. When we aim at estimating the optical Such linear constraints are posed over all the pixels in the flow for long image sequences recursively, the question is whether current image, forming an ill-posed estimation problem.
and how can we gain from previous estimates. In this paper we
The various differential-based methods thus vary in the show that gain is achieved from both computational and accuway they add constraints in order to ensure a single and racy points of view. Incorporation of the time axis into the estimaaccurate solution to the estimation problem. For example, tion process is done by assuming temporal smoothness of the optical flow, resulting in simplified spatial-temporal models. The Lucas and Kanade [3] assume that the optical flow is locally obtained models permit incorporation of the constrained constant, thus making possible the construction of a weighted least squares (CWLS) estimator. This estimator is weighted combination of several constraints, assuming shown to yield RLS and LMS adaptive filter versions for recursive they have the same solution. The Horn and Schunck apoptical flow estimation in time. An interesting and desirable proach [2] was a regularization based on the assumption property of the proposed estimation algorithms is their flexibility of spatial smoothness over the optical flow field.
with respect to performance versus computational requirements.
In a comparison between different optical flow estima-By a simple choice of a parameter these algorithms can be modition algorithms made by Barron [4], it was found that the fied to exploit the available time to improve their performance with respect to estimation error. The convergence properties of performance of the Horn and Schunck algorithm is inferior these estimation algorithms are analyzed. Simulations for varito other differential methods. However, for image seous image sequences support the analysis and demonstrate the quences showing distant static objects filmed with camera performance of the estimation algorithms. © 1998 Academic Press motion, the spatial smoothness assumption is highly valid and the algorithm of Horn and Schunck performs quite well. Moreover, adopting the modifications in the way the
1. Introduction
local gradients are estimated proposed by Barron [4] can also improve the performance of the algorithm, resulting in an attractive method of optical flow estimation. Thus, Optical flow is the displacement field related to each of for many applications the above assumptions about the the pixels in an image sequence. Such a displacement field motion field are quite reasonable [1,2,[4][5][6][7][8]. results from the apparent motion of the image brightness Most algorithms for the estimation of optical flow concenin time. Estimating the optical flow is a fundamental probtrate on estimating the motion field between succeeding imlem in low-level vision and can undoubtedly serve many ages in a sequence, disregarding the estimates obtained for applications in image sequence processing. There are many the previous image pair. Among such procedures we can different methods of estimating the optical flow [1-9, 14, count both the Horn and Schunck [2] and the Lucas and 18-27]. Roughly speaking, these methods can be divided into correlation, energy, phase, and differential-based Kanade [3] methods. However, several attempts have al-119