The use of optical flow for the analysis of non-rigid motions

[Journal Name], [Volumn Number], 1?? ([Volumn Year])c [Volumn Year] Kluwer Academic Publishers, Boston. Manufactured in The Netherlands.The use of optical ow for the analysis of non-rigid motionsANDREA [email protected] [email protected] di Fisica, Universit�a di Genova, Via Dodecaneso 33, 16146 Genova, ItalyAbstract.This paper analyses the 2D motion �eld on the image plane produced by the 3D motion of a planeundergoing simple deformations. When the deformation can be represented by a planar linear vector�eld, the projected vector �eld, i.e. the 2D motion �eld of the deformation, is at most quadratic. This2D motion �eld has one singular point, with eigenvalues identical to those of the singular point describingthe deformation. As a consequence, the nature of the singular point of the deformation is a projectiveinvariant. When the plane moves and experiences a linear deformation at the same time, the associated 2Dmotion �eld is at most quadratic with at most 3 singular points. In the case of a normal rototranslation,i.e. when the angular velocity is normal to the plane, and of a linear deformation, the 2D motion �eldhas one singular point and substantial information on the rigid motion and on the deformation can berecovered from it. Experiments with image sequences of planes movingand undergoing linear deformationsshow that the proposed analysis can provide accurate results. In addition, experiments with deformableobjects, such as water, oil, textiles and rubber show that the proposed approach can provide informationon more general 3D deformations.IntroductionThe majority of available algorithms for theanalysis and recovery of 3D motion of moving ob-jects from image sequences, makes the assump-tion of opacity and rigidity (Fennema & Thomp-son, 1979; Hildreth, 1984; Longuet-Higgins, 1984;Nagel, 1983; Francois & Bouthemy, 1990). Asthese objects are seen by an imaging device, the3D motion �eld of moving objects in the scene istransformed into a 2D motion �eld in the imageplane (Gibson, 1950; Horn & Schunck, 1981). Theassumption of opacity implies that at any locationin the image plane, the 2D motion �eld is singlevalued, that is the 2D motion �eld is uniquely de-termined. In the case of transparent objects, the2D motion �eld is not single valued and two di�er-ent velocities can be assigned to the same locationin the image plane. In the case of opaque objects,the 2D motion �eld is a planar vector �eld, whichcan be analysed with the tools of dynamical sys-tems theory (Hirsch & Smale, 1974).The assumption of rigidity is usually made inorder to simplify the problem and, by using prop-erties of singular points, useful information onthe 3D motion can be recovered (Verri, Girosi &Torre, 1989). The problem of non-rigid motionhas already been considered: Ullman (1984) in-troduced an incremental approach to recover thestructure from motion, even in the case of non-rigid bodies, and Jasinschi & Yuille (1989) have re-cently used the same approach with more sophisti-cated mathematical tools. Bergholm and Carlsson(1991) used the ow of "visual directions" to re-cover information about complex motions. Penna(1992) considered the problem of shape from mo-tion for objects undergoing non-rigid isometricmotions. Yakamoto, Boulanger, Beraldin & Ri-oux (1993) introduced a deformable net model to�nd the constraints for a direct estimation of 3Dmotion. Other approaches to the non-rigid motionhave been proposed by Koenderink & Van Doorn

2 THE AUTHORS???(1986), Shulman and Aloimonos (1988), Subbarao(1989).In this paper we relax the hypothesis of rigid-ity and we analyse a special case of moving anddeformable objects, that is of deformations of aplane undergoing a normal rototranslation. De-formations on the plane are modelled by a linearvector �eld, allowing to introduce the three ele-mentary deformations: expansions, rotations andshears (Helmholtz, 1858). The deformable plane isalso supposed to move in the 3D space. The justi-�cation for studying this speci�c case is threefold.Firstly, given the 2D motion �eld, it is possible toprovide almost a complete solution to the problemof the recovery of deformations and rigid motion. Secondly, the proposed approach is a good lo-cal approximation of a generic deformation occur-ring on the surface of an opaque object. Thirdly,as shown in the experimental section, several de-formations of real objects can be treated in �rstapproximation as linear planar deformations.Planar deformations have already been studiedby several authors. Wohn and Waxman (1990)analysed deformation components (up to the sec-ond order) of the 2D motion �eld on the imageplane to study the 3D motion of rigid surfaces;Rao and Jain (1992) proposed the representationof ow patterns with linear di�erential equations;Shu and Jain (1993) and Ford, Strickland andThomas (1994) proposed algorithms for the recov-ery of the linear deformation components of ow�elds. This paper analyses the recovery of 3D lin-ear planar deformations, given their projectionson the image plane, presents some new analyticalresults and shows how to recover the 3D motionand linear deformations in a simple case. In ad-dition, an extensive experimentation on syntheticand real images is presented.The paper is organized as follows: Section 1introduces linear planar deformations and provesthat for this class of deformations there are use-ful perspective invariants. By using this propertythe combined case of deformable and moving planecan be treated almost completely. Section 2 showsthat in the case of a normal rototranslation (i.e.when the angular velocity is perpendicular to theplane) a simple recovery of the 3D motion andof the linear deformation can be obtained. Ad-ditional properties of the singular points are pre-

sented in Section 3. Section 4 presents severalexperiments with synthetic and real images. Ex-periments with image sequences of real deformableobjects ( liquid, textiles, rubber...) show that theproposed procedure for recovering motion and de-formation can also be used in the case of moregeneral deformations and more complex objects.1. Properties of linear deformations overa planeIn this section we introduce the problem underinvestigation and we prove the existence of someperspective invariants for planar linear deforma-tions (see eqn. 17). A well known theorem ofHelmoltz (1858) (see also Sommerfeld 1950) hasshown that the most general motion of a su�-ciently small element of a deformable (i.e. notrigid) body can be represented as the sum of atranslation, a rotation and an extension (or con-traction) in three mutually orthogonal directions.In the presence of opaque objects, the visible de-formations are those occurring at the object sur-face. These deformations can be assumed to oc-cur locally on the plane tangent to the surface.Following the approach of Sommerfeld (1950), let~X� be a point of an element of a plane � andX�� , X�� its coordinates in the reference system(O� ; �; �) where O� is the center of reference and�; � two orthogonal unit vectors. We want tostudy the motion �eld in the plane around thepoint ~P � = (P �� ; P �� ). In the case of a gen-eral non-rigid motion both points ~X� and ~P �will experience changes in position, which we de-note ~V � = (V �� ; V �� ) and ~V �0 = (V �0�; V �0�) for ~X�and ~P � respectively. If the element is su�cientlysmall, it is possible to use a Taylor expansion upto the �rst order, so that:~V �( ~X�) = ~V �0 + L( ~X� � ~P �) (1)the term ~V �0 is the rigid translation, while the sec-ond term, represented by the linear operator L iscaused by the rotation and the non rigid compo-nent of the motion. In the system of referencepreviously introduced, L can be expressed as:� V ��V �� � = � V �0�V �0� �+ � L11 L12L21 L22 �� X�� � P ��X�� � P �� �(2)

THE TITLE??? 3It is evident therefore that the matrix L canbe used to characterize the rotation and non rigidpart of the motion of a su�ciently small elementof a plane or locally of a surface. Now we dropthe translational term and we consider the plane� with a stationary point ~P � and a 2D motion�eld determined by the linear deformation ~V �D onits surface, given by:~V �D = L( ~X� � ~P �) (3)where L is the matrix previously de�ned. It iswell known that the matrix L describing the de-formation can be decomposed as:L = � L11 L12L21 L22 � = E� 1 00 1 �++!� 0 1�1 0 �+ S1� 1 00 �1 �+ S2� 0 11 0 �(4)whereE = L11 + L222 ! = L12 � L212S1 = L11 � L222 S2 = L12 + L212 (5)are the elementary deformation components:expansion, rotation and shears respectively, repro-duced in Fig. 2 A, C and E. From eqn. (5) it isevident that the component of expansion E is sim-ply equal to Trace L=2.Now we suppose that the deforming plane is ly-ing in the 3D space and is observed by an imagingdevice with the optical center centered in the ori-gin O of a reference system (O, e1; e2; e3) as shownin Fig. 1. [Fig. 1 near here]~X = (X1; X2; X3) indicates a point in the 3Dspace and ~V its velocity. The optical axis of theimaging device is assumed to coincide with the e3axis and the image plane to have equation X3 = fwhere f is the focal length of the imaging de-vice. We want to analyse the relation betweenthe 2D vector �eld ~VD describing the deformationon the plane � and its perspective projection ~vDon the image plane of the imaging device and we

will show the existence of some useful perspec-tive invariants (eqn.17). If is the unit vectorperpendicular to the deforming plane and d is thedistance between the plane and the optical center,the equation of the plane is: � ~X = d (6)Let us now write the motion �eld (3) of the defor-mation on � as a 3D vector �eld in the referencesystem (O; e1; e2; e3). If we choose the origin ofthe reference system on � so that the position ofO� in the reference system (O; e1; e2; e3) is ~O suchthat ~O is perpendicular to the plane �, we have~O � � = 0, ~O � � = 0, ~O � = d. If ~X is on theplane �, and ~X� is the position of the same pointin the reference system (O� ; �; �), we have thatX�� = ~X � � and X�� = ~X � �. The motion �eld inthe reference system (O; e1; e2; e3) can be writtenas:~VD = (V1; V2; V3) == 8>><>>: L11[� � ( ~X � ~P )]�+ L22[� � ( ~X � ~P )]�++L12[� � ( ~X � ~P )]�+ L21[� � ( ~X � ~P )]�if ~X 2 �0 otherwise (7)where ~P is the 3D position of the stationary pointof the deformation in the reference of the imagingdevice. If we rewrite the 3D motion �eld (7) interms of its components we have:Vi = L11[� � ( ~X � ~P )]�i + L22[� � ( ~X � ~P )]�i ++L12[� � ( ~X � ~P )]�i + L21[� � ( ~X � ~P )]�i( ~X 2 �) (8)where �i = � � ei, �i = � � ei (i = 1; 2; 3).Now we want to �nd the perspective projectionof ~VD over the image plane (see Fig. 1). The wellknown perspective projection formulas are:~x = fX3 ~X (9)

4 THE AUTHORS???~v = fX23 [e3 � (~V � ~X)] == fX23 [(e3 � ~X)~V � (~V � e3) ~X] (10)[Fig. 2 near here]Because of eqn. (10) the vector �eld ~VD( ~X 2 �) istransformed into the 2D motion �eld on the imageplane ~vD as:~vD( ~X) = fX23X3fL11[� � ( ~X � ~P )]�++L22[� � ( ~X � ~P )]� + L12[� � ( ~X � ~P )]�++L21[� � ( ~X � ~P )]�g � fX23 fL11[� � ( ~X � ~P )]�3++L22[� � ( ~X � ~P )]�3 + L12[� � ( ~X � ~P )]�3++L21[� � ( ~X � ~P )]�3g ~X ( ~X 2 �) (11)Eq. (11) can be rewritten, using (9) as:~vD(~x) = (v1; v2) = L11[� � (~x� �~x �~P ~P )]�++L22[� � (~x� �~x �~P ~P )]� + L12[� � (~x� �~x �~P ~P )]�++L21[� � (~x� �~x �~P ~P )]� � fL11[� � (~x� �~x �~P ~P )]�3++L22[� � (~x� �~x �~P ~P )]�3 + L12[� � (~x� �~x �~P ~P )]�3++L21[� � (~x� �~x �~P ~P )]�3g~xf (12)and by introducing the elementary deformationcomponents (5) and rearranging the di�erentterms as shown in Appendix A, we obtain the sim-ple expression:v1 = a13x21 + a23x1x2 + (a33 � a11)fx1+�a21fx2 � a31f2v2 = a13x1x2 + a23x22 + (a33 � a22)fx2+�a12fx1 � a32f2 (13)

withaij � E iPjfd + !( � ej � eif � i(~P � )jfd )++S1�j(~P � �)i + �j(~P � �)ifd ++S2 � �j(~P � �)i + �j(~P � �)ifd (14)The motion �eld (??) has the same mathemat-ical structure of the one caused by a rigid mo-tion of a plane (Verri, Girosi & Torre, 1989), andonly the parameters aij are di�erent. As a con-sequence, the motion �eld (13) and the motion�eld of an arbitrary rigid motion of a plane havesome common properties. For instance both mo-tion �elds have at most three singular points andcannot have limit cycles (Aicardi & Verri, 1990).The 2D motion �eld (??) has a singular point ~p(i.e. a point such that ~vD(~p) = 0) that is theperspective projection of the point ~P over theimage plane: ~p = f ~PP3 . In the case of a lin-ear expansion, characterized by the coe�cient E(L11 = L22 = E;L12 = L21 = 0) eqn. (??) be-comes:~vE = E[(� � ~x)�+ (� � ~x)�]� E( � ~x)d [(� � ~P )�++(� � ~P )�]� Ef [(� � ~x)�3 + (� � ~x)�3]~x++E( � ~x)fd [(� � ~P )�3 + (� � ~P )�3]~x (15)Remembering that (��~x)�+(� �~x)�+( �~x) = ~xand (� � ~x)�3 + (� � ~x)�3 + ( � ~x) 3 = f we have:~vE = E � ~xfd (~xP3 � f ~P ) (16)If we analyse, in the general case, the Jacobianmatrix M of the 2D motion �eld on the imageplane in its singular point ~p, by simple, but longcalculations (see Appendix A), we obtain the im-portant result:TraceL = TraceMDetL = DetM (17)

THE TITLE??? 5where L = 0BB@ @V ��@X�� ���� ~X�=~P� @V ��@X�� ���� ~X�=~P�@V ��@X�� ���� ~X�=~P� @V ��@X�� ���� ~X�=~P� 1CCAM = 0BBB@ @v1@x1 ����~x=~p @v2@x1 ����~x=~p@v1@x2 ����~x=~p @v2@x2 ����~x=~p 1CCCA (18)Therefore, given a linear deformation, the cen-ter of deformation is projected into a singularpoint having eigenvalues equal to those of the cen-ter of deformation on the plane �, that is theeigenvalues of a linear deformation are perspectiveinvariants. Also the elementary components E, !and the sum of the squared components of shear(the components of shear depend on the choice ofthe unit vectors) S21 + S22 are perspective invari-ants. These invariant properties can also be seenin Fig. 2, which illustrates the perspective pro-jection of an expansion (B), a rotation (D) and ashear (F). It is evident that the projected vector�eld is no more a linear vector �eld, but has thesame kind of singular point.2. The motion �eld of a moving plane withlinear deformationsIn this section we will consider properties of the2D motion �eld produced by a translating planeundergoing a linear deformation of the kind dis-cussed in the previous section.In this section the plane � is assumed to move,so that its distance d(t) from the optical centervaries with time and its equation is: � ~X = d(t) (19)If � is rigid and moves by a pure translationT = (T1; T2; T3), the associated 2D motion �eldon the image plane is:~vT = � ~xfd(t) (~Tf � ~xT3) (20)

which has the same structure of the 2D motion�eld projected on the image plane by the linearexpansion (16). The two 2D motion �elds (16)and (20) become identical when:E(t)~P (t) = �~T (21)As a consequence the 2D motion �eld of theplane � ~X = d(t) translating with speed ~T is in-stantaneously identical to the 2D motion �eld ofa linear expansion on a �xed plane lying in thesame position and centered in the point:~Q = d(t)~T � ~T (22)which can be written in the reference system(O� ; �; �) as:~Q� = (Q��; Q��) = d(t) � ~T (~T � �; ~T � �) (23)The eigenvalue of the expansion is equal to:ET (t) = � � ~Td(t) (24)As a consequence, the 2D motion �eld of aplane undergoing the linear deformation ~VD (sumof expansion, rotation and shear) with center ~Pand the translation ~T is instantaneously equal tothe perspective projection of the linear deforma-tion ~V �D + ~V �T occurring on the same plane withthe center of deformation in an appropriate point~P � = (P ��; P �� )~V �D + ~V �T = � V ��V �� � == � L11 L12L21 L22 �� X�� � P ��X�� � P �� �++ET � X�� � Q��X�� � Q�� � = L0� X�� � P ��X�� � P �� � (25)The Jacobian of this linear deformation is :L0 = L+ET I = � L11 + ET L12L21 L22 + ET � (26)The 2D motion �eld on the image plane is thendescribed by eqs. (13) and (14) replacing ~P with

6 THE AUTHORS???~P � and E with E + ET . The trace and determi-nant of the Jacobian M of the singular point ofthe 2D motion �eld are:DetM = DetL0 =L11L22 � L12L21 + ET (L11 + L22) +E2T (27)TrM = TrL0 = L11 + L22 + 2ET (28)Let us see how these results can be used to re-cover some useful motion parameters. If we writethe reciprocal of ET :1ET = �d(0) + ( � ~T )t � ~T (29)and we set, for simplicity, the time origin so as tohave d(0) = 0, we obtain ET = 1t : the time vari-able t can now be interpreted as the time whichhas to elapse before collision between the movingplane and the optical center of the imaging device,quantity which we will call time to collision. It isevident that the value of t can be obtained by asimple derivation of TrM=2 (i.e. the componentof expansion):@(TrM)@t = @@t "� � ~T( � ~T )t# == ( � ~T )2[( � ~T )t]2 = 1t2 = E2T (30)The sign of t can be recovered by a furtherderivation of TrM=2 and the eigenvalues of thereal deformation can be obtained as :TrL = L11 + L22 = TrM � 2ET (31)DetL = DetM �ETTrM+ 2E2T (32)In the same way we can recover the componentsof the real deformation:E = TrL=2 = TrM � 2ET (33)! = L12 � L212 = M12 �M212 (34)

S2tot = S21 + S22 == (L12 + L21)2 + (L11 � L22)22 == (M12 +M21)2 + (M11 �M22)22 (35)This result can be extended to the case of thenormal rototranslation, i.e. when a rigid rotation!0 with axis perpendicular to the moving plane (sothat ~! k ) is added. In this case the location ofthe singular point on the image plane is di�erentbut the motion �eld is still equivalent to a linearone, and the only change in the matrixM is that! has to be replaced by ! + !0.3. Evolution of singular points of the 2Dmotion �eld of linear planar deforma-tionsSingular points of the 2D motion �eld on theimage plane, i.e. those points ~p such that ~v(~p) = 0,have been shown to capture many features of themotion of rigid bodies (Verri et al., 1989). Forinstance, from the time evolution of ~p on the im-age plane and the analysis of its eigenvalues, it ispossible to recover substantial information on the3D rigid motion. Therefore it is interesting to seewhether the analysis of the evolution of singularpoints of the 2Dmotion �eld of deformable objectsmay provide similar information. In this sectionwe will analyse the evolution of the singular pointof the 2D motion �eld originating from a planeundergoing a linear deformation and a translationFirst of all it is useful to notice that the sin-gular point of the 2D motion �eld projected bya linear planar deformation on the image planeevolves in such a way that the components of ro-tation and shear remain unchanged over time (seeeqs. 34{35). This property allows to distinguishbetween the 2D motion �eld produced by a rigidplane undergoing an arbitrary rototranslation andthe motion �eld of a deforming plane undergoinga normal rototranslation. In the former case thecomponents of shear are expected to change withtime, while in the latter case they are expected toremain unchanged.

THE TITLE??? 7The singular point ~p will change its location onthe image plane with time and, as shown in Ap-pendix B, its trajectory will be a conic in the gen-eral case (see eqn. (B22)). When the time tocollision is very large (see eqn. (B15)) we have:limt!1(p1; p2) = (T1T3 ; T2T3 ) (36)as a consequence the location of the singular pointfor large values of the time to collision is related tothe direction of the 3D translation and this prop-erty can be used to recover the direction of trans-lation. [Fig. 3 near here]When the plane is perpendicular to the opticalaxis and the translation is parallel to the opticalaxis, the trajectory of the singular point ~p has avery simple shape, as shown in Fig. 3A. As shownin Appendix B, the following cases can be found:� in the case of a pure expansion the singularpoint moves on a segment with equation (B26)(solid line).� in the case of a pure rotation the singular pointmoves on a circle with equation (B30) (brokenline).� in the case of a shear the singular point moveson a hyperbola with equation (B32) (dottedline).Fig. 4 illustrates 2D motion �elds obtainedfrom eqs. (13) and (14) with = (0; 0; 1) and~T = (0; 0;�T ) in the case of a pure expansion (A,B), a pure rotation (D,E) and a pure shear (G,H).[Fig. 4 near here]In the �rst column the 2D motion �elds are rep-resented when the time to collision is very largeand in the second column the 2D motion �eld arerepresented on the image plane just before the col-lision. The last column reproduces the trajecto-ries of the singular point on the image plane forthe three cases. The arrows point toward the col-lisions.Because of the rigid motion the singular point ~pmay change its qualitative nature during its evo-lution on the image plane, that is it may vary the

value of the trace and of the determinant. Theproperties of the singular point can be describedby analysing the trajectory of the point in the(TrM,DetM) plane (Verri, Girosi & Torre 1989)as shown in Fig. 3B. This plane is divided intoseveral regions, according to the sign of TrM andof DetM. When a point is above the parabolaDetM = (TrM=2)2 (the dotted line in Fig. 3B),the singular point is a spiral; below this parabolathe singular point is a node or a saddle point. Be-low the DetM axis, the singular point is a saddlepoint. For t!1, from eqn. (??) we obtain thatET ! 0 and the singular point on the image planehas the same eigenvalues as those of the pure de-formation. As a consequence when t is very large,the direction of translation can be recovered fromthe location of the singular point on the image andthe deformation can be characterized by the eigen-values of the singular point. For t! 0 (collision)we haveDetM ! 1TrM ! 1 (37)and the eigenvalues of M become very close toET giving immediate information about the timeto collision.It is easy to show that the singular point in the(TrM; DetM) plane can never cross the aboveparabola and the only possible trajectories areshown as solid lines in Fig. 3B.4. Experimental resultsIn order to verify the theoretical results pre-sented in previous sections we have designed threesets of experiments. The �rst set was intended totest whether the invariant properties implied by(??) could be veri�ed with image sequences of de-forming planes. The second set aimed at verifyingto which extent the recovery of the true 3D mo-tion and of the deformations, outlined in Section2, could be obtained in practice. In the third set ofexperiments we analysed image sequences of realdeformable objects, such as liquids, textiles andrubber.Deforming planes[Fig. 5 near here]

8 THE AUTHORS???Fig. 5 A and B illustrate synthetic images of atiger undergoing a pure expansion, while Fig. 5 Cand D show images of the same tiger undergoinga shear deformation. Under these conditions theproperties of the original deformation were exactlyknown. A sequence of images of successive defor-mations was used and the optical ow was com-puted using the technique described in De Micheliet al. (1993). The optical ow was obtained by�rst convolving images with a symmetric Gaus-sian �lter 1p2� exp� x2+y22�2 with � = 1:5 pixels andby using the values of 0:4 and 0:3 for the con-trol parameters dH and cH respectively, which al-low the selection of reliable displacements. In thiscase a sparse optical ow is obtained. Fig. 6A andFig. 6B reproduce the optical ows obtained fromthe sequence of the expanding and of the deform-ing tiger respectively. From these sparse optical ows, the best linear vector �eld through each op-tical ow can be estimated (see Fig. 6 C and Drespectively). As a consequence, it is possible toestimate the eigenvalues of the matrix M and torecover the deformations of the tiger by using theresults presented in Section 2. Table 1 illustratesa comparison between the recovered and the orig-inal deformations of Fig. 5.[Fig. 6 near here][Table 1 near here]The true expansion was 1.98�10�2 frame�1 andthe experimentally obtained value was correctwith an error not greater than 3%. A similar pre-cision was also observed for the image sequencesreproducing the shear. Similar results were ob-tained in other image sequences in which a patternwas deformed under controlled conditions and theangle � between the image plane (i.e. the viewingcamera) and the deforming plane was varied from0 to 60 degrees. A comparison between recoveredand original deformations, shown in Table 2, indi-cates a reasonable agreement between the recov-ered and original deformations when � is below 45degrees. It is evident that the accuracy of the esti-mation deteriorates when � becomes larger than45 degrees. In this case the second order term ofthe 2D motion �eld is predominant and the esti-

mation of the linear term of the 2D motion �eldbecomes rather sensitive to noise. These resultssuggest that the eigenvalues of linear planar de-formations can be useful perspective invariants ina variety of experimental conditions.[Table 2 near here]Moving and deforming planesThe same deformations were used in another setof experiments, but in the presence of a known rel-ative motion between the plane and the viewingcamera. We have also used synthetic images ofclouds deforming with a shear, a rotation and anexpansion while the viewing camera was movingtowards the deforming plane.[Fig. 7 near here]Fig. 7A illustrates a frame of an image sequenceof simulated deforming clouds translating towardthe camera. Fig. 7B and C illustrate a sparseoptical ow obtained from the sequence and itsbest linear approximation . From this linear ap-proximation it is possible to estimate the di�erentparameters describing the motion and the defor-mation. The time to collision was estimated by us-ing equation (??). The deformation componentswere computed from eqns. (33{35). A compar-ison between the estimated values and the realtime to collision and deformation parameters is il-lustrated in Table 3. The three deformation com-ponents were recovered with a precision of about90%. The estimation of the time to collision ob-tained with a di�erentiation (see eqn. (??)) canbe rather noisy. In this case, the error can be e�-ciently reduced with an appropriate �ltering (see�gure caption). Fig. 7D shows a comparison be-tween the �ltered values (diamonds) and the truevalues (solid line).[Table 3 near here]Deforming objectsIn the third series of experiments image se-quences of liquids and deforming objects wereanalysed. In these experiments the real deforma-tion could be measured in an indenpedent way anda qualitative and quantitative test of the proposedapproach could be obtained.

THE TITLE??? 9[Fig. 8 near here]Fig. 8A and B illustrate two frames of an im-age sequence taken while a drop of a black liquidwas deforming in a jar �lled with water. Theseimages were acquired at video rate and the twoimages in A and B were the �rst and the 12thin the sequence. In this case the deformation iscaused by convection and di�usion. Fig. 8 C andD illustrate optical ows computed with the tech-niques of De Micheli et al., 1993 and of Campani& Verri, 1992 respectively. The optical ow in C isnot dense and is signi�cant only at the edge of theblack spot, while the optical ow in D is dense, butblurred. The average expansion around the edgesof the black spot computed with the two proce-dures was 0.0142 and 0.0139 frame�1 respectivelyand corresponds fairly well to the real displace-ment of the edge of the black liquid, as illustratedin Fig 8E. The edge contours of the deformingdrop in the two images in A and B are repro-duced with the superimposed optical ow. This ow was magni�ed by 12 times corresponding tothe number of frames separating image A and B.Fig 8 F illustrates the expansion coe�cient com-puted from �ve consecutive optical ows obtainedwith the technique of De Micheli et al. (1993).[Fig. 9 near here]Fig 9 illustrates the case of a rotating magnet de-forming a mixture of water and small dark parti-cles. The angular velocity of the rotating magnetwas 0:125 rad/frame. Two images of this deform-ing liquid are shown in Fig 9 A and B. The op-tical ow obtained with a correlation technique(described in the �gure legend) is shown in Fig9 C. Contrary to the case of a rigid object, theinstantaneous velocity does not increase linearlywith the distance from the singular point. How-ever, the angular velocity computed very near thesingular point was close to the true angular ve-locity of the stirring magnet. The decrease of theangular velocity (see Fig. 9 D) at more distantpoints is caused by the viscosity of the liquid.Fig 10 illustrates the deformation of an eraserrubber deformed by the pressure exerted by twoclamps. One clamp was �xed, while the other was

moved by 1 mm. between each frame. The widthof the eraser was 25 mm.[Fig. 10 near here]Fig 10 A and B illustrate the �rst and �fthframe of the sequence and the deformation of thesqueezed eraser is clearly evident. C and D re-produce a sparse optical ow obtained from theentire image and a dense optical ow around thesingular point respectively. The obtained optical ow around the singular point was decomposedin the four elementary components, providing theestimates: �0:014 frame�1, �0:001 rad. frame�1,0:019 frame�1 and �0:001 frame�1 for E;!; S1and S2 respectively. Evidently, the major defor-mations were E and S1. If we approximated thereal deformation with a linear one, shrinking therubber in the y direction by a factor 1:04 everyframe (i.e. 25/24), the deformation had the twocomponents: E = �0:018 frame�1 and S1 = 0:018frame�1. The values of E and S1 obtained by theproposed approach and those calculated assum-ing a linear deformation were in reasonable agree-ment.The deformation of an elastic textile is illus-trated in Fig 11. In this case two parallel sidesof the textile were kept �xed, while the other twoorthogonal sides were moved apart so as to havean expansion in one direction by a factor 1.02 atevery frame, corresponding to an ideal linear de-formation with E = 0:01 frame�1 and S1 = �0:01frame�1. [Fig. 11 near here]Fig 11 A and B illustrate the �rst and the �fthframe of the sequence in which the textile is de-formed. C and D reproduce a subsampled op-tical ow obtained from the entire image and adense optical ow around the singular point re-spectively. The obtained optical ow around thesingular point was decomposed in the four elemen-tary components and had the values of E = 0:010frame�1, ! = 0:002 rad. frame�1, S1 = �0:014frame�1, S2 = 0:002 frame�1.These values agree with those corresponding tothe real linear deformation.

10 THE AUTHORS???5. DiscussionThe aim of this paper is to begin an analysis ofimage sequences of deforming objects. The majorcontribution of the paper consists in the identi�-cation of some perspective invariants of planar lin-ear deformations (see eqn. (17)). The use of theseinvariants allows the recovery of the 3D motionand of the parameters describing the deformationsin the simple case of normal rototranslations (seeSection 2). In addition, the paper shows that it ispossible to obtain a meaningful optical ow alsoin the case of deformable objects, such as liquidsand gases.Optical ow and deformable objectsIt is now well established that an almost cor-rect optical ow can be computed from image se-quences of moving rigid objects (Barron et al.,1994, Otte and Nagel, 1994) and it is thereforeinteresting to see whether a reasonable optical ow can be obtained also in more general cases.Our experimentation indicates that the optical ow of uids and deforming objects can be es-timated whenever the underlying dynamics is nottoo chaotic (Lichtenberg & Lieberman, 1992).The analysis of deformationsThis paper studies the simple case of a planemoving by a normal rototranslation (a motionduring which the axis of rotation is perpendicu-lar to the plane itself) and undergoing a lineardeformation. The results obtained in sections 1{3show that the 2Dmotion �eld of a linearly deform-ing and moving plane is at most quadratic. It isalso shown that the translation and the deforma-tions occurring on the plane can be recovered bylooking at the properties of singular points, as pro-posed in Verri et al., 1989 for rigid moving objects.The proposed approach can be used to analyse ageneral deformation occurring at the surface of anopaque object provided that the Taylor expansionof eqn. (1) can be truncated up to the �rst order,as in the case of a su�ciently small surface ele-ment. As a consequence, it is adequate to analysedeformations of opaque objects locally.Our approach makes use of the computation ofoptical ow and it is useful also to observe that itsestimation and the recovery of its �rst order prop-erties are usually reliable whenever the 2D motion�eld is essentially a linear or at most quadratic

vector �eld. Therefore the proposed approach willfail when the approximationof eqn.(1) is valid overa too small surface element and when the normalvector to the surface becomes parallel to the im-age plane (see Table 2). In this case second or-der terms of the 2D optical ow will have to beconsidered, with a considerably heavier computa-tion. The deformations reproduced in Fig. (8{11)could be satisfactorily analysed with the proposedtechnique, thus indicating that the underlying as-sumptions are veri�ed in a variety of real cases.Future workThe analysis of image sequences of deformableobjects is an important �eld with a wide range ofrelevant applications. The analysis of the blood ow and cardiac motion (Amartur & Vesselle,1993), and the study of calcium waves in livingcells (Gallione et al., 1993) are two major appli-cations to biology. The study of the convectionof clouds and of other meteorological phenomenais another important �eld which may bene�t fromtechniques developed in computer vision. How-ever it is important to observe that these problemshave speci�c constraints and peculiarities and thetheoretical results obtained in this paper can cer-tainly be extended to more realistic cases.References1. Amartur, S.C. and Vesselle, H.J. 1993. A new ap-proach to study cardiac motion: the optical ow ofcine MR images. Nuclear Magnetic Resonance inMedicine. 12, 59{67.2. Aicardi, F. and Verri, A. 1990. Limit Cycles of theTwo-Dimensional Motion Field. Biol. Cybern. 64,141{144.3. Barron, J.L., Fleet, D.J., and Beauchemin,S.S. 1994Performance of optical ow techniques. Int. Journalof Computer Vision 12, 1:43{77.4. Bergholm, F. and Carlsson, S. 1991 A "Theory" ofOptical Flow. Comput. Vis. Graph. Image Proc.: Im-age Understanding 53,2: 171{1885. Campani,M. and Verri, A. 1992.Motion analysis from�rst order properties of optical ow. Comput. Vis.Graph. Image Proc.: Image Understanding 56, 1:90{107.6. De Micheli, E., Uras, S., and Torre, V. 1993. The accu-racy of the computation of optical ow and of the re-covery of motion parameters. IEEE Trans. Patt. Anal.Machine Intell. 15, 5:434{447.7. Fennema, C.L. and Thompson, W.B. 1979. Velocitydetermination in scenes containing several moving ob-jects. Comput. Vis. Graph. Image Proc. 9, 301{315.8. Ford, R. M., Strickland,R.N and Thomas, B. A. 1994Image Models for 2-D Flow Visualization and Com-pression CVGIP: GMIP 56,1:75{93

THE TITLE??? 119. Francois, E. and Bouthemy, P. 1990. The derivationof qualitative information in motion analysis. ImageVis. Comput. J. 8, 279{287.10. Gallione, A., Mc Dougull, A., Busa, W.B., Will-mott, N., Cillot, I. and Whitaker, M. 1993. Redun-dant Mechanisms of Calcium{Induced Calcium Re-lease Underlying Calcium Waves During Fertilizationof Sea Urchin Eggs. Science 261, 348{250.11. Gibson, J.J. 1950. The perception of the visual world.Boston: Houghton Mi�in.12. Helmholtz, H. 1858. Uber Integrale der hydrody-namischen Gleichungen welche den Wirbelwegungenentsprechen. Crelles J. 55, 25.13. Hildreth, E.C. 1984. The Computation of the velocity�eld. Proc. R. Soc. London B 221, 189{220.14. Hirsch, M.W. and Smale, S. 1974. Di�erential equa-tions, dynamical systems, and linear algebra. NewYork: Academic Press.15. Horn, B.K.P. and Schunck, B.G. 1981. Determiningoptical ow. Artif. Intell. 17, 185{203.16. Jasinschi, R. and Yuille, A. 1989. Nonrigidmotion andRegge calculus.J. Opt. Soc. Amer. A, 6, 7: 1085{1095.17. Koenderink, J.J. and van Doorn, A.J. 1986. Depthand shape from di�erential perspective in the pres-ence of bending deformations. J. Opt. Soc. Amer. A,3, 2: 242{249.18. Lichtenberg,A.J. and Lieberman,M.A. 1992. Regularand Chaotic Dynamics. Springer Verlag.19. Longuet-Higgins, H.C. 1984. The visual ambiguity ofa moving plane. Proc. R. Soc. London B 223, 165{175.20. Otte, M. and Nagel, H.H. 1994 Optical ow estima-tion: advances and comparisons. Proc. Third Europ.Conf. on Comp. Vision Stockholm, Sweden, vol. 1,51{60.21. Nagel, H.H. 1983. Displacement vectors derived from2nd order intensity variations in image sequences.Comput. Vision Graph. Image Process. 21, 85{117.22. Penna,M. A. 1992. Non-rigidmotion analysis: isomet-ric motion. Comput. Vis. Graph. Image Proc.: ImageUnderstanding 56 3,366{381.23. Rao, A. R. and Jain R.C. 1992 Computer Flow FieldAnalysis: Oriented texture Fields IEEE Trans. Pat-tern Anal. Machine Intell. 14, 7: 693{70624. Shu, C. and Jain R.C. 1993 Direct Estimation andError Analysis for Oriented Patterns. CVGIP: ImageUnderstanding 58, 383{39825. Shulman, D and Aloimonos, J,Y. (1988) (Non-) rigidmotion interpretation : a regularized approach. Proc.Royal Soc. of London B 233, 217{23426. Sommerfeld,A. (1974)Mechanics of Deformable Bod-ies, Academic Press, New York, USA27. Subbarao, M. 1989. Interpretation of image ow: Aspatio-temporal approach. IEEE Trans. Patt. Anal.Machine Intell. 3, 266{278.28. Ullman, S. 1984. Maximizing rigidity: The incremen-tal recovery of 3D structure from rigid and nonrigidmotion. Perception, 13 255{274.29. Verri, A., Girosi, F., and Torre, V. 1989. Mathemat-ical properties of the two-dimensional motion �eld:from singular points to motion parameters. J. Opt.Soc. Am. A 6, 698{712.

30. Verri, A. and Poggio, T. 1989. Motion �eld and opti-cal ow: qualitative properties. IEEE Trans. PatternAnal. Machine Intell. 11, 490{498.31. Wohn, K and Waxman, A.M. 1990 The AnalyticStructure of Image Flows: Deformation and Segmen-tation. CVGIP 49, 127{15132. Yakamoto, M., Boulanger, P., Beraldin, J.A. and Ri-oux, M. 1993. Direct estimation of range ow on de-formable shape from a video rate range camera. IEEETrans. on PAMI 15, 1: 82{89.Appendix AFirst order properties of the singular pointEqn. (12) can be rewritten as a function of theelementary deformation components. As L11 =E+S1, L22 = E�S1, L12 = !+S2, L21 = S2�!,we have for v1:v1 = Efdf[(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3) + [(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)g++S1fdf[(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)� [(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)g++ !fdf[(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)� [(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)g++S2fdf[(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)� [(� � ~x)( � ~P )+�( � ~x)(� � ~P )](�1f � x1�3)g (A1)As � ~P = d is the distance between the op-tical center and the deforming plane, using some

12 THE AUTHORS???some well known properties of vector calculus (i.e.(~a�~b)�~c = (~b�~c)�~a , (��~x)�i+(� �~x)�i+( �~x) i =xi , (aibj � biaj) = (~a �~b) � ~ek ) we obtain:v1 = Efd ( � ~x)(P3x1 � P1f)++S1fd [(~P � �) � ~x(�3x1 � �1f)++(~P � �) � ~x(�3x1 � �1f)]++ !fd [(e1 � ~x) � f � (e3 � ~x) � x1+�( � ~x)[(e1 � ~P ) � f + (e3 � ~P ) � x1]++S2fd (~P � �) � ~x)(�1f � �3x1)+�(~P � �) � ~x)(�1f � �3x1) (A2)By rearranging eqn.(??) we obtain:v1 = Efd ( 1P3x21 + 2P3x1x2 + ( 3P3+� 1P1)fx1 � 2P1fx2 � 3P1f2)++S1fdf[(~P � �)1�3 + (~P � �)1�3]x21++[(~P � �)2�3 + (~P � �)2�3]x1x2++[(~P � �)3�3 + (~P � �)3�3+�(~P � �)1�1 + (~P � �)1�1]fx1+�[(~P � �)2�1 + (~P � �)2�1]fx2+�[(~P � �)3�1 + (~P � �)3�1]f2g++ !fdf[ 2fd + (~P � ~ )3 1)]x21++[� 1fd+ (~P � ~ )3 2]x1x2++[(~P � ~ )3 3 � (~P � ~ )1 1]fx1+�[ 3fd + (~P � ~ )1 2]fx2+

�[ 2f2d+ (~P � ~ )1 3]f2g++S2fdf[�(~P � �)1�3 + (~P � �)1�3]x21++[�(~P � �)2�3 + (~P � �)2�3]x1x2++[�(~P � �)3�3 + (~P � �)3�3++(~P � �)1�1 � (~P � �)1�1]fx1+�[(~P � �)2�1(~P � �)2�1]fx2+�[�(~P � �)3�1 + (~P � �)3�1]f2g (A3)In the same way a similar expression for v2 canbe obtained and the two components can be writ-ten as:v1 = a13x21 + a23x1x2 + (a33 � a11)fx1+�a21fx2 � a31f2v2 = a13x1x2 + a23x22 + (a33 � a22)fx2+�a12fx1 � a32f2 (A4)where:aij � E iPjfd + !( � ej � eif � i(~P � )jfd )+S1�j(~P � �)i + �j(~P � �)ifd +S2 � �j(~P � �)i + �j(~P � �)ifd (A5)Let us now compute the eigenvalues of the Jaco-bian matrix of the motion �eld M in the singularpoint ~p = f ~P =P3. First we rewrite eqn.(12) as afunction of ~p:~vD = L11[� � (~x� �~x �~p ~p)]�+ L22[� � (~x� �~x �~p~p)]�++L12[� � (~x� �~x �~p~p)]�+ L21[� � (~x� �~x �~p~p)]�+� 1f fL11[� � (~x� �~x �~p ~p)]�3 + L22[� � (~x� �~x �~p~p)]�3+L12[� � (~x � �~x �~p~p)]�3 + L21[� � (~x� �~x �~p~p)]�3g~x(A6)We de�ne, for simplicity:

THE TITLE??? 13� = � � (~x� � ~x � ~p~p) (A7)� = � � (~x � � ~x � ~p ~p) (A8)The motion �eld ~vD can now be written as:~vD = L11��+ L22�� + L12��+ L21��+�~xf (L11��3 + L22��3 + L12��3 + L21��3) (A9)and the components are:v1 = L11��1 + L22��1 + L12��1 + L21��1+�x1f (L11��3 + L22��3 + L12��3 + L21��3)v2 = L11��2 + L22��2 + L12��2 + L21��2+�x2f (L11��3 + L22��3 + L12��3 + L21��3)(A10)Let us now compute the partial derivative:@v1@x1 = L11�1 @�@x1 + L22�1 @�@x1 + L12�1 @�@x1+L21�1 @�@x1 � 1f fL11��3 + L22��3 + L12��3++L21��3g � x1f fL11�3 @�@x1 + L22�3 @�@x1++L12�3 @�@x1 + L21�3 @�@x1g (A11)As �(~p) = �(~p) = 0, the above partial deriva-tive, computed in ~p becomes:@v1@x1 ����~x=~p = L11(�1 � �3p1f ) @�@x1++L22(�1 � �3p1f ) @�@x1 + L12(�1 � �3p1f ) @�@x1++L21(�1 � �3p1f ) @�@x1 (A12)

Similarly we have:@v2@x2 ����~x=~p = L11(�2 � �3p2f ) @�@x2++L22(�2 � �3p2f ) @�@x2 + L12(�2 � �3p2f ) @�@x2++L21(�2 � �3p2f ) @�@x2 (A13)@v1@x2 ����~x=~p = L11(�1 � �3p1f ) @�@x2++L22(�1 � �3p1f ) @�@x2 + L12(�1 � �3p1f ) @�@x2++L21(�1 � �3p1f ) @�@x2 (A14)@v2@x1 ����~x=~p = L11(�2 � �3p2f ) @�@x1++L22(�2 � �3p2f ) @�@x1 + L12(�2 � �3p2f ) @�@x1++L21(�2 � �3p2f ) @�@x1 (A15)@�@x1 , @�@x2 , @�@x1 , @�@x2 do not depend on ~x and areequal to:@�@x1 = �1 � � � ~p � ~p 1 (A16)@�@x2 = �2 � � � ~p � ~p 2 (A17)@�@x1 = �1 � � � ~p � ~p 1 (A18)@�@x2 = �2 � � � ~p � ~p 2 (A19)Substituting eqns. (A16{A19) into (A12{A15) we�nally obtain:

14 THE AUTHORS???@v1@x1 ����~x=~p = L11(�1 � �3p1f )(�1 � ��~p �~p 1) + L22(�1+��3p1f )(�1 � ��~p �~p 1) + L12(�1 � �3p1f )(�1 � ��~p �~p 1)++L21(�1 � �3p1f )(�1 � ��~p �~p 1) (A20)@v2@x2 ����~x=~p = L11(�2 � �3p2f )(�2 � ��~p �~p 2) + L22(�2+��3p2f )(�2 � ��~p �~p 2) + L12(�2 � �3p2f )(�2 � ��~p �~p 2)++L21(�2 � �3p2f )(�2 � ��~p �~p 2) (A21)@v1@x2 ����~x=~p = L11(�1 � �3p1f )(�2 � ��~p �~p 2) + L22(�1+��3p1f )(�2 � ��~p �~p 2) + L12(�1 � �3p1f )(�2 � ��~p �~p 2)++L21(�1 � �3p1f )(�2 � ��~p �~p 2) (A22)@v2@x1 ����~x=~p = L11(�2 � �3p2f )(�1 � ��~p �~p 1) + L22(�2+��3p2f )(�1 � ��~p �~p 1) + L12(�2 � �3p2f )(�1 � ��~p �~p 1)++L21(�2 � �3p2f )(�1 � ��~p �~p 1) (A23)Now we can compute the trace and the deter-minant of the Jacobian matrixM:M = 0BBBBB@ @v1@x1 ����~x=~p @v1@x2 ����~x=~p@v2@x1 ����~x=~p @v2@x2 ����~x=~p 1CCCCCA (A24)We can choose the reference system (O�; �; �)such that e2 � �, and therefore: �1 = �3 = 0 ,

�2 = 1, �2 = 2 = 0, �1 = 3, �3 = � 1. As aconsequence, we have:@v1@x1 ����~x=~p = L11(�1 � �3p1f )(�1 � � � ~p � ~p 1)++L12(�1 � �3p1f )(� � � ~p � ~p 1) (A25)@v2@x2 ����~x=~p = L22 + L12(��3p2f ) (A26)@v1@x2 ����~x=~p = L12(�1 � �3p1f ) (A27)@v2@x1 ����~x=~p = L11(��3p2f )(�1 � � � ~p � ~p 1)++L22(� � � ~p � ~p 1) + L12(��3p2f )(� � � ~p � ~p 1)++L21(�1 � � � ~p � ~p 1) (A28)and the trace of M becomes:TrM = @v1@x1 ����~x=~p + @v2@x2 ����~x=~p == L11(�1 � �3p1f )(�1 � � � ~p � ~p 1)++L12(�1 � �3p1f )(� � � ~p � ~p 1) + L22 + L12(��3p2f )(A29)Remembering that �1 = 3 and 1 = ��3, wehave:TrM = L11f [�1( 3f + 1p1) + �3 � � ~p � ~p ( 1p1++ 3f)] + L22 + L12f [( 3f + 1p1)(� p2 � ~p 1) + �3p2](A30)

THE TITLE??? 15TrM = L11f [ 3( � ~p) + �3(� � ~p)]++L22 + L12f [( � ~p) 1p2 � ~p � 1p2](A31)TrM = L11ff + L22 + L12 � 0 = L11 + L22 (A32)In a similar way we compute the determinant ofM: DetM = @v1@x1 @v2@x2 � @v1@x2 @v2@x1 == [L11(�1 � �3p1f )(�1 � ��~p �~p 1)++L12(�1 � �3p1f )(� ��~p �~p 1)][L22 + L12(��3p2f )]+�[L12(�1 � �3p1f )][L11(��3p2f )(�1 � ��~p �~p 1)++L22(� ��~p �~p 1) + L12(��3p2f )(� ��~p �~p 1)++L21(�1 � ��~p �~p 1)] (A33)DetM = L11L22(�1 � �3p1f )(�1 � ��~p �~p 1)++L22L12(�1 � �3p1f )(� ��~p �~p 1)++L212(�1 � �3p1f )(� ��~p �~p 1)(��3p2f )++L11L12(��3p2f )+�L11L12(�1 � �3p1f )(�1 � ��~p �~p 1)(��3p2f )+�L212(�1 � �3p1f )(� ��~p �~p 1)(��3p2f )+�L22L12(�1 � �3p1f )(� ��~p �~p 1)+�L12L21(�1 � �3p1f )(�1 � ��~p �~p 1) (A34)

DetM = L11L22(�1 � �3p1f )(�1 � ��~p �~p 1)++L11L12(��3p2f )� L11L12(�1 � �3p1f )(�1+� ��~p �~p 1)(��3p2f ) � L12L21(�1 � �3p1f )(�1 � ��~p �~p 1)(A35)As we have already found that:(�1 � �3p1f )(�1 � � � ~p � ~p 1) = 1 (A36)the previous equation becomes:DetM = L11L22 � L12L21 + L11L12(��3p2f )+�L11L12(��3p2f ) = L11L22 � L12L21 (A37)Therefore the two matrices:L = 0BBBB@ @V ��@X�� ���� ~X�=~P� @V ��@X�� ���� ~X�=~P�@V ��@X�� ���� ~X�=~P� @V ��@V �� ���� ~X�=~P� 1CCCCAand M = 0BBBBB@ @v1@x1 ����~x=~p @v1@x2 ����~x=~p@v2@x1 ����~x=~p @v2@x2 ����~x=~p 1CCCCCAhave the same trace and the same determinant,and, of course, the same eigenvalues.Appendix BSingular point locationWe want to study the evolution of the posi-tion of the singular point of the 2-D motion �eldproduced by the 3-D motion of a plane with nor-mal vector translating with uniform speed ~T andundergoing a linear deformation with center ~P (t)and matrix L. The solution can be simpli�ed re-membering that at each time t, we can replacethe 3-D motion with the equivalent 2-D linear de-formation on the plane � ~X = d(t). This �eld is

16 THE AUTHORS???easily written in the reference system of the plane,(O�; �; �) and the equations for the singular pointof this equivalent �eld on the plane � are:V �� = L11(X�� � P �� ) + L12(X�� � P �� )++Etr(X�� � Q��) = 0V �� = L21(X�� � P �� ) + L22(X�� � P �� )++Etr(X�� � Q��) = 0(B1)where Etr = � �~Td and ~Q� = � 1Etr (~T � �; ~T � �).The singular point of the 2-D motion �eld on theimage plane is the projection of the singular pointof the equivalent 2-D �eld (B1). The singularpoint of the equivalent 2-D motion (B1) on theplane � is:X�� =(DetL)P��+E2trQ��+L11EtrP��+L22EtrQ��+L12Etr(P�� �Q�� )(DetL)+Etr((TrL)+Etr )X�� =(DetL)P�� +E2trQ��+L11EtrQ�+L22EtrP�� +L21Etr(P���Q��)(DetL)+Etr((TrL)+Etr ) (B2)Now it is useful to write the coordinates ofthe singular point (B2) in the reference system(O; e1; e2; e3). As we have taken the 3D positionof O� as ~O = d( � e1; � e2; � e3), the 3D coordi-nates of the point (X�� ; X�� ) of � are:Xi = X���i +X�� �i + d i i = 1; 2; 3(B3)and the position of the singular point is givenby: X�i = X���i +X���i + d i i = 1; 2; 3(B4)Replacing X�� and X�� with the expressions of(B2), we have:X�i = Ni(DetL) +Etr((TrL) +Etr) (B5)

where Ni is de�ned as:Ni = (DetL)Pi +E2trQi + EtrfL11[(~P � �)�i++(~Q � �)�i + (~P � ) i] + L22[(~Q � �)�i++(~P � �)�i + (~P � ) i] + L12(~P � � � ~Q � �)�i++L21(~P � �� ~Q � �)�ig (B6)Remembering the perspective projection for-mula: x�i = f X�iX�3 (B7)we have:x�1 = f N1N3x�2 = f N2N3 (B8)Now we introduce the temporal dependences for~P , Etr and ~Q as:Pi(t) = Pi(0) + Tit (B9)Etr(t) = � ( � ~T )d(0) + ( � ~T )t (B10)~Q(t) = d(0) + ( � ~T )t( � ~T ) ~T (B11)In order to simplify the calculations it is con-venient to set, as in the text, the time origin sothat d(0) = 0. In this case for t = 0 the planepasses through the optical center. The equationsbecome:Etr(t) = �1t (B12)~Q(t) = ~T t (B13)introducing (B9),(B12) and (B13) into eqn.(B6) we have:

THE TITLE??? 17Ni = 1tfTi[(DetL)t2 � (TrL) + 1] + (DetL)Pi(0)t++[L12(~T � �)�i + L21(~T � �)�i]t� L11(~P (0) � �)�i+�L22(~P (0) � �)�i + L12(~P (0) � �)�i++L21(~P � �)�i (B14)and the position of the singular point becomes:x�1 = f nT1T3 + �1t+�1T23 [(DetL)t2�(TrL)t+1]+T3(DetL)P3(0)tox�2 = f nT2T3 + �2t+�2T23 [(DetL)t2�(TrL)t+1]+T3(DetL)P3(0)to(B15)where�1 = T3[(DetL)P1(0) + L12(~T � �)�1++L21(~T � �)�1]� T1P3(0)(B16)�1 = �T3[L11(~P (0) � �)�1 + L22(~P (0) � �)�1++L12(~P (0) � �)�1 + L21(~P (0) � �)�1](B17)�2 = T3[(DetL)P2(0) + L12(~T � �)�2++L21(~T � �)�2]� T2P3(0)(B18)�2 = �T3[L11(~P (0) � �)�2 + L22(~P (0) � �)�2++L12(~P (0) � �)�2 + L21(~P (0) � �)�2](B19)From (B15) it is evident that, fot t ! 1, i.e.when the plane is far from the collision, the posi-tion of the singular point is ~x� = (T1=T3; T2=T3).The curve described by the evolution of thepoint is found eliminating t from the system. Witha few steps we have (we drop the stars in the fol-lowing):t = �1(x2T3 � fT2)� �2(x1T3 � fT1)�2(x1T3 � fT1)� �1(x2T3 � fT2) (B20)

From the �rst of eqs. (B15) and (B20), dividingby (x1T3�fT1) we �nd a second-order polynomialequation describing the trajectory of the singularpoint:T3(DetL)[�1(x2T3 � fT2)� �2(x1T3 � fT1)]2++[(DetL)� T3(TrL)][�1(x2T3 � fT2)+��2(x1T3 � fT1)][�2(x1T3 � fT1)� �1(x2T3+�fT2)] + [�2(x1T3 � fT1) ��1(x2T3 � fT2)]2+�(�22�1 ��1�2�2)(x1T3 � fT1)f � (�21�2+��1�2�1)(x2T3 � fT2) = 0 (B21)This equation can be simpli�ed with a changeof variables: y1 = (x1 � f T1T3 ); y2 = (x2 � f T2T3 ):(DetL)[(�1y2) � (�2y1)]2 + [(DetL)+�T3(TrL)][(�1y2)� (�2y1)][(�2y1) � (�1y2)]++[(�2y1) � (�1y2)]2 � (�22�1 � �1�2�2)y1+�(�21�2 � �1�2�1)y2 = 0 (B22)In this way we have proved that the curve de-scribed by the singular point is a conic.Now we can see in more detail the equationsof the conics in the three cases of Fig. 3, withthe translation perpendicular to the image plane(T1 = T2 = 0, T3 = T ) with the deforming planeparallel to the image plane and a deformation de-scribed just by one elementary component. Inthis case 1 = 2 = 0, P (0) � = 0 and wecan choose the reference systems so as to have� � e1, � � e2. In the case of a pure expansion(i.e. L11 = L22 = E, L12 = L21 = 0) eqn. (B15)becomes:x1 = f EP1(0)T (Et � 1) (B23)x2 = f EP2(0)T (Et � 1)

18 THE AUTHORS???If Et 6= 1 we can write:T (Et � 1)x1 �EP1(0)f = 0T (Et � 1)x2 �EP2(0)f = 0 (B24)and we obtain:t = EP1(0)f + Tx1TEx1 = EP2(0)f + Tx2TEx2 (B25)The equation of the curve is the straight line:P1(0)x2 � P2(0)x1 = 0 (B26)In the case of a pure rotation (i.e. L11 = L22 =0 and L12 = �L21 = !) the system becomes:x1 = f !2(P1(0)!t � P2(0)T (!2t2 + 1))x2 = f !2(P2(0)!t + P1(0)T (!2t2 + 1)) (B27)multiplying the �rst equation by x2, the second byx1 and subtracting the two equations we obtain:t = 1! P1(0)x1 + P2(0)x2P1(0)x2 � P2(0)x1 (B28)from eqs. (??) and (??) we have:x1T 2(P1(0)x1 + P2(0)x2)2 + x1T 2(P1(0)x2+�P2(0)x1)2 � TP1(0)f(P1(0)x1++P2(0)x2)(P1(0)x2 � P2(0)x1)++!P2(0)Tf(P1(0)x2 � P2(0)x1)2 = 0 (B29)and after a few steps we obtain the equation of acircle:Tx21 + Tx22 + !fP2(0)x1 � !fP1(0)x2 = 0 (B30)When L11 = �L22 = S1 and L12 = L21 = 0,the singular point is located at:x1 = f �S1TP1(0)(S1t+ 1)T 2(1� S21t2) (B31)x2 = f �S1TP2(0)(S1t� 1)T 2(1� S21t2)

and, after a few steps, the curve is found to be thehyperbola:2Tx1x2 � S1P2(0)fx1 + S1P1(0)fx2 = 0 (B32)

THE TITLE??? 19FIGURE LEGENDSFig. 1: The reference system (O� ; �; �) solid tothe plane � with normal unit vector and thereference system (O; e1; e2; e3) solid to the imag-ing device, with O coinciding with the optical cen-ter and the image plane coinciding with the planeX3 = f .Fig. 2: Linear deformations (A: expansion; C: ro-tation; E: shear) and their projections (B, D andF respectively) on the image plane. The normalvector to the plane is (0.92,0,0.40).Fig. 3: A: trajectories of the singular point onthe image plane in the case of a translation andan expansion (solid line, see eqn. B26), a rotation(broken line, see eqn. (B30)) and a shear (dot-ted line, see eqn. (B32)). The translating planeis always parallel to the image plane. B: allowedtrajectories of the singular point in the TrM andDetM plane.Fig. 4: Simulation of the 2D motion �eld onthe image plane of moving and deforming planes.A: 2D motion �eld on the image of an expand-ing and translating plane with center of deforma-tion ~P � = (148; 148) pixel and eigenvalue E =1 frame�1. The moving plane is perpendicularto the optical axis with distance from the opti-cal center d = 3000 pixel, and translates with~T = (0; 0;�500) pixel frame�1. B: Same as A,but with d = 500 pixel. C: evolution of the po-sition of the singular point on the image planefor the expanding and translating plane in A andB. D: 2D motion �eld on the image of a rotatingand translating plane with center of deformation~P � = (148; 148) pixel and angular speed ! = 1rad frame�1. The moving plane is perpendicu-lar to the optical axis with distance from the op-tical center d = 3000 pixel, and translates with~T = (0; 0;�500) pixel frame�1. E: same as in Dbut with d = 500 pixel. F: evolution of the posi-tion of the singular point on the image plane forthe rotating and translating plane in D and E.G: 2D motion �eld on the image of a deformingand translating plane with center of deformation~P � = (148; 148) pixel and component of shearS1 = 1 frame �1. The moving plane is perpen-dicular to the optical axis with distance from the

optical center d = 3000 pixel, and translates with~T = (0; 0;�500) pixel frame�1. H: same as in Gbut with d = 500 pixel. I: evolution of the positionof the singular point on the image plane for the de-forming and translating plane in G and H. In C,F and I the arrows point toward the collisions.Fig. 5: A and B: two images (256 � 256) of anexpanding tiger. C and D: two images of the sametiger undergoing a shear deformation.Fig. 6: Examples of the sparse optical ows ob-tained with the algorithm of De Micheli et al.,1993 for the expanding tiger (Fig. 5 A and B)and for the deforming tiger (Fig. 5 C and D) re-spectively. Images were convolved with a Gaus-sian �lter 1p2� exp� x2+y22�2 with � = 1:5 pixels andthe control parameters dH and cH were set equalto 0:4 and 0:3 respectively, so as to obtain a sparsebut reliable optical ow (see De Micheli et al, 1993for further details). C and D are the best linearvector �eld through the sparse optical ows shownin A and B respectively, obtained with a leastsquares �t. From these estimation of the optical ow, the deformation components were obtainedfrom eqs. (33{35)Fig. 7: A reproduces an image of synthetic de-forming clouds. B and C are the sparse optical ow (computed with the technique of De Micheliet al., 1993 with the values of 1:5; 0:4 and 0:3for �; dH and cH respectively) and its best lin-ear approximation respectively. D reproduces aplot of the time-to-collision against number offrame. This values are obtained by �ltering thevalues computed with eqn. (30) with the �lter:ttc(k) = aa+1 [ttc(k�1)�1]+ 1a+1 ttc(k) with a de-pending on the measurement error (see De Micheliet al., 1993 for further details).Fig. 8: A and B reproduce two images of a blackdrop of liquid deforming in water. C, D are theoptical ows computed with the technique of DeMicheli et al., 1993 (� = 1:5; dH = 0:4; cH = 0:3)and Campani and Verri, 1992 (after a gausssian�ltering with � = 1:5 pixels and with patches of41� 41 pixels) respectively. E represents the con-tour of the drop at the two frames of A and Bwith the superimposed optical ow computed at

20 THE AUTHORS???the edges of the �rst frame with the technique ofCampani and Verrin and magni�cated by a fac-tor equal to the number of frames (12) separatingthe two images. F reproduces the coe�cient ofexpansion obtained from the best linear approxi-mation of the sparse optical ow computed withthe technique of De Micheli et al., 1993.Fig. 9: A and B reproduce two images of a stir-ring magnet deforming a mixture of water anddark particles. C is an example of an optical ow computed with a correlation-based technique.This technique consists of the comparison with asuitable distance function, of the grey level pat-tern in a patch of 41�41 pixels around each pixelof the frame with other patterns of the same size inthe successive frame centered in a set of points ap-propriately shifted from the original position. Theshift minimizing the distance function is taken asthe true motion. D is the angular velocity againstthe distance from the center of rotation. The an-gular velocity at the distance � was computed asthe average value of the rotational component ofthe optical ow in the annulus inside the two cir-cles with radius � � 5 pixels.Fig. 10: A and B reproduce two frames of aneraser rubber squeezed by two clamps. C and Dreproduce the subsampled optical ow obtainedfrom the entire image (computed with the tech-nique of Campani and Verri (1990) over patches of41� 41 pixels) and a dense ow relative to the re-

gion near the singular point (the black dot). Theanalysis of the optical ow around the singularpoint provides an estimate of the deformation pa-rameters.Fig. 11: A and B reproduce two frames of anelastic textile expanded along y direction. C andD reproduce the optical ow obtained from the en-tire image and a dense ow relative to the regionnear the singular point (black dot). The optical ow was computed as in the case of Fig. 10.Table 1: Estimations of the components of de-formation obtained from the sparse optical owsrelative to the sequences of Fig. 5. Eqs. (30) and(33{35) were used. � is the standard deviation.Table 2: The recovery of the components of de-formation for di�erent values of the angle � be-tween the deforming plane and the image plane.Eqs. (30) and (33{35) were used. � is the stan-dard deviation.Table 3: Estimation of the true time to colli-sion and components of deformation for a translat-ing and deforming plane. The average estimatedexpansion, rotation and shear were 2:08 � 10�2frame�1 , 1:74 � 10�2 frame�1 and 1:98 � 10�2frame�1 respectively, while the true values were1:98 � 10�2 frame�1 , 1:74 � 10�2 frame�1 and1:98 � 10�2 frame�1.