Lifeng Wang, Stephen Lin, Member, IEEE, Seungyong Lee, Member, IEEE,
Baining Guo, Member, IEEE, and Heung-Yeung Shum, Senior Member, IEEE
Abstract :
We present a 2D feature-based technique formorphing 3D objects represented by light fields. Existing light field morphing methods require the user to specify corresponding 3D feature elements to guide morph computation. Since slight errors in 3D specification can lead to significant morphing artifacts, we propose a scheme based on 2D feature elements that is less sensitive to imprecise marking of features. First, 2D features are specified by the user in a number of key views in the source and target light fields. Then the two light fields are warped view by view as guided by the corresponding 2D features. Finally, the two warped light fields are blended together to yield the desired light field morph. Two key issues in light field morphing are feature specification and warping of light field rays. For feature specification, we introduce a user interface for delineating 2D features in key views of a light field, which are automatically interpolated to other views. For ray warping, we describe a 2D technique that accounts for visibility changes, and present a comparison to the ideal morphing of light fields. Light field morphing based on 2D features makes it simple to incorporate previous image morphing techniques such as non-uniform blending, as well as to morph between an image and a light field.
Index Terms
3D morphing, Light field, Ray correspondence, Ray space warping, Ray resampling.
I. INTRODUCTION
METAMORPHOSIS, or morphing, is a technique for generating a visually smooth transformation from one object to another. Often morphing is performed between objects each represented as an image. This approach of image morphing which has been popular in the entertainment industry, computes a transformation based on corresponding 2D features between the images. This 2D feature-based interpolation of images can be extended to 3D geometry morphing that deals with objects whose geometry and surface properties are known. 3D geometry morphing techniques have been developed for polygonal meshes and volumetric presentations [9], [10], [11], and they allow for additional effects such as changes in viewpoint any real objects, however, have geometry and surface properties that are too difficult to model or recover by traditional graphics and computer vision techniques. Light fields are often used to model the appearance of such objects from multiple viewpoints , and a method for morphing based on the light field epresentation has recently been proposed [12]. With user specification of 3D polygonal features in the light fields, this approach effectively combines the advanced morphing effects of 3D geometry morphing with the fine appearance details of images.
The principal drawback in the light field morphing technique of [12] is its reliance on 3D feature polygons. Computing an accurate 3D position of a feature polygon requires very precise user specification of the polygon in two views, presenting a major challenge to the user. Slight misalignment of the feature polygons even by only a couple pixels in these two views can be amplified into increasingly large errors in the polygon’s image position at more divergent viewing angles, leading to prominent morphing inaccuracies. Careful avoidance of this misalignment is a cumbersome task, which involves much zooming in and out and substantial trial-and-error. In cases such as untextured regions, the exact pixel positions of corresponding polygon vertices can be far from obvious to the user, even with the computer vision aids that are incorporated into the user interface.
In this paper, we present a new approach to light field morphing based entirely on 2D features. Unlike the 3D method in [12], our 2D technique does not compute or rely on the 3D positions of feature points. Rather than warping geometrically in 3D, our technique records correspondences among multiple views and warps between these views by feature point interpolation. In this way, specification errors have only a local effect, and are not propagated and magnified globally as with 3D features. This approach significantly improves the usability of light field morphing by reducing the sensitivity of morphing quality to user specification error. Additional benefits of 2D features in light fields are that they allow for easy incorporation of prior image morphing techniques such as non-uniform blending, and for morphing between an image and a light field.
The remainder of the paper is organized as follows. Section II reviews previous morphing methods related to our technique. In Section III, we present an overview of light field morphing and outline the previous technique
based on 3D features. Section IV describes our method based on 2D features, including the user interface and the concept of ray space warping. In Section V, we
analyze our 2D technique with respect to ideal light field morphing when geometry is known, and then provide a comparison with other object morphing approaches.
Experimental results are exhibited in Section VI, and conclusions are presented in Section VII.
II. RELATED WORK
Progress in 2D morphing has led to simplification and greater generality in feature specification. In early work, meshes have been used to define position correspondences in an image pair, where mesh coordinates are interpolated by bicubic splines to form the warp sequence [13]. To facilitate specification, field morphing utilizes only line correspondences, with which a warp is determined according to the distance of points from the lines [1]. Even greater generality is afforded by using point-based correspondence primitives that include lines and curves. This general form of feature specification is used by most subsequent image morphing works.
Based on correspondences of point features, several methods have been proposed for generating morphs that are smoother and require less user interaction. Warping functions that are C1-continuous and one-to-one, which precludes folding of warped images, have been presented in an energy minimization framework [14]. An approach called multilevel free-form deformation computes a warp that is both one-to-one and C2-continuous. Additionally, snakes are employed in this method to aid the user in feature specification. Further reducing the load on users, image analysis has also been used to automatically determine correspondences between similar objects [15]. Since these image morphing methods operate on only a pair of images, changes in visibility, which is a significant issue for light fields, cannot be accurately handled.
Visibility information is available in 3D morphing where geometric objects are represented and processed in 3D space. The appearance details of 3D geometric models generally lack the realism of actual images though, particularly for objects with complex surface properties. The approach we propose in this paper, however, can handle arbitrary surface properties, such as hair and fur, because it is image-based. The plenoptic editing method proposed by Seitz and Kutulakos [16] can be used for 3D morphing of objects represented by images, and a technique by Jeong et al. [17] has been presented for morphing between surface light fields. These approaches, though, face the fundamental difficulties of recovering surface geometry from images [18]. Our proposed method does not require any 3D reconstruction from images, nor does it need to employ a complex 3D morphing process.
III. OVERVIEW
The light field morphing problem can be described as follows. Given the source and target light fields L0 and L1 representing objects O0 and O1, construct an intermediate light field L®; 0 · ® · 1, that represents plausible objects O® with the essential features of O0 and O1. The blending coefficient ® denotes the relative
similarity to the source and target light fields, and as ® changes from 0 to 1, L® should smoothly transform from its appearance in L0 to that in L1.
In practice, a sequence of intermediate light fields is never fully computed because of the prohibitive amount of storage this would require [12]. We instead compute only an animation sequence along a user-specified camera path, and then compute only the morph images along this path, beginning in L0 with ® = 0 and ending in L1 with ® = 1. Camera positions are expressed in the twoplane parameterization of light fields [19] as coordinates in the (s; t) plane, while (u; v) coordinates index the image plane. A given light field L can be considered either as a collection of 2D images nL(s;t)o, where each image is called a view of light field L, or as a set of 4D rays fL(u; v; s; t)g. In L(s;t), the pixel at position (u; v) is denoted as L(s;t)(u; v), which is quivalent to ray L(u; v; s; t).
Fig. 1 provides an overview of our light field morphing pipeline. Given two input light fields L0 and L1, the user specifies the camera path for the orphing sequence and the corresponding object features in L0 and L1 at several key views. To generate a morph image within an intermediate light field L®, we first determine the positions of the features in the image of L® by interpolating them from L0 and L1. Then, images of two warped light fields ^L 0 and ^L1 are computed from L0 and L1 such that the corresponding features of L0 and L1 are aligned at their interpolated positions in ^L0 and ^L1. Finally, the warped image of ^L0 and of ^L1 are blended together to generate the morph image of L®.
The two most important operations in light field morphing are feature specification and warping of light field rays. In feature specification, the user interface (UI) should be intuitive, and an accurate morphing result should be easily achievable. Our proposed method based on 2D features presents a UI that is very similar to image morphing, and morph quality is not significantly reduced by misalignment of features by the user. The warping of 4D light field rays resembles the 2D and 3D warping used in image and volume morphing, respectively. For light fields, however, visibility changes among object patches need to be addressed. For this, we propose a method for resampling occluded rays from other light field views in a manner similar to ray-space warping in [12]. For the 3D features specified in [12], feature visibility can easily be determined by depth sorting, but in our work the depth ordering cannot be accurately inferred for roughly-specified 2D features. Our method deals with this problem by automatically identifying views where feature visibility is ambiguous and having the user order the 2D features by depth in these views.
IV. ALGORITHM
In feature-based morphing, corresponding features on the source and target objects are identified as pairs of feature elements [5]. In this section, we propose a
2D feature-based approach to light field morphing that circumvents the need for 3D geometry processing.
A. Feature Specification
In the first step of the morphing process, the camera path and the corresponding features in the two light field objects are specified with the user interface exhibited in Fig. 2. For the given source and target light fields L0 and L1, windows (1) and (2) in Fig. 2 display the light field views L0;(s;t) and L1;(s;t) for a user-selected viewing parameter (s; t). In the (s; t) plane shown in window (3), the user draws a camera path that is copied to window (4), and selects some key views for feature specification. The morphing process is controlled by two kinds of feature elements: feature line A feature line is a 2D polyline in a light field view. It is used to denote corresponding prominent features on the source and target light field objects, similar to image morphing. feature polygon A feature polygon is a 2D polygon formed by a closed polyline. Unlike feature lines which simply give positional constraints on a warping function, feature polygons are used to partition the object into patches to be handled independently from each other in the morphing process. Parts of a light field view not belonging to any feature polygon form the background
region. The features in the source and target light fields have a one-to-one correspondence, and although these features are specified in 2D space, they are stored as 4D elements that include the positions in the image and camera planes. In the snapshot shown in Fig. 2, windows (1) and (2) show feature elements being specified in a key view of the light fields. In this example, a pair of feature polygons (marked by yellow polylines) outline the faces. Within these two feature polygons, feature lines (indicated in light blue) are specified to mark prominent facial elements such as the eyes, nose, and mouth. Two more pairs of feature polygons (marked in pink and blue) are used to indicate the necks and right ears. With the specified feature elements in several key views, our system automatically interpolates these fea-ture elements to the views on the camera path using the view interpolation algorithm of Seitz and Dyer [20]. Depending on the light field parameterization, we choose different forms of view interpolation. For the classical
two-plane parameterization , the camera orientation is the same for all light field views, so a simple linear interpolation provides physically correct view interpolation results [20]. If camera orientations vary among the light field views, the general form of view interpolation is employed to obtain correct results. Our system supports both the two-plane parameterization [21], [19] and the concentric mosaic [22], for which pre-warping and post-warping must be applied in conjunction with linear interpolation to produce physically correct results [20].
At views along the camera path, the interpolated feature polygons can possibly overlap, leading to ambiguities in feature visibility in the morph sequence. For example, Fig. 3 shows a view where the right ear polygon (red) is occluded by the face polygon (yellow). Our method addresses this issue by determining the views on the camera path in which the visibility conditions could change, as indicated by feature polygon intersections. These views are added to the key views, and our system prompts the user for a depth ordering of the overlapping feature polygons. At an arbitrary viewpoint on the camera path, feature visibility can then be determined to be the same as that of the preceding key view in the morph sequence. Since objects are generally composed of only a small number of feature polygons and there are not many instances of depth ordering uncertainty in a morph sequence, little effort is required from the user to provide depth ordering information.
The user can browse through the morph sequence in the user interface to examine the interpolation results. If the interpolated feature positions do not align acceptably with the actual feature positions in a certain view, this view can be added to the set of key views, and features can then be specified to improve morphing accuracy. This additional key view is used for re-interpolation of features, as well as adjusting the positions of key views that signify visibility changes.
The user specification process is greatly facilitated by the use of 2D features. In [12] where 3D polygons are used, errors in specified feature positions propagate globally to all views and can lead to significant feature misplacement in the warping process. By instead specifying 2D features in a number of views, error propagation is only local, and can easily be limited by adding key views where needed.
B. Ray space deformation
From the corresponding feature polygons and lines specified by the user in L0 and L1, warping functions are determined for resampling the light field rays. To
obtain a warped light field ^L0 from a given light field L0, we first determine the positions of feature polygons and feature lines in ^L0 by linearly interpolating those in L0 and L1. Then, the correspondence of the polygons and lines between L0 and ^L0 defines a warping function w for backward resampling [13] of ^L0 from L0, where w gives the corresponding ray (u0; v0; s0; t0) in L0 for each ray (u; v; s; t) in ^L0:
In our proposed approach, we approximate the 4D warping function w as a set of 2D warping functions nw(s;t)o defined at the views of ^L0. That is, for each view ^L0;(s;t) of ^L0, we compute a 2D warping function w(s;t) between L0;(s;t) and ^L0;(s;t) from the feature correspondence. This reduces Eq. (1) to
^L 0(u; v; s; t) = L0(u0; v0; s; t);
where (u0; v0) = w(s;t)(u; v): (2)
In Section V-A, we show the validity of this approximation when depth values and surface normals of corresponding feature points do not differ substantially.
To compute the warping function w(s;t), we first organize the rays in ^L0;(s;t) according to the feature and background regions. Since feature regions in ^L0;(s;t) can possibly overlap, there may exist rays ^L0;(s;t)(u; v) in the warped light field that project to more than one feature region. In such cases, the ray should belong to the foremost region which is visible, as determined from the depth ordering described in the previous subsection. Fig. 4 shows the ray classification results for a few views of ^L0, where the colors indicate which region each ray is associated with. In the center image, the feature region of the right ear is visible, while in the left image, its image position overlaps with the face region. Since the face region is closer to the viewer than the ear region according to the depth order in the center image, the rays that would correspond to the ear region belong to the face region instead
Since each feature region, defined by the polygon boundary and the feature lines within, can have a warping distortion different from the others, the feature regions in ^L0 are then warped individually. Let R and ^R be a pair of corresponding feature regions in L0(s; t) and ^L0(s; t), respectively. For the rays in ^L0 associated with ^R, w(s;t) is determined from the corresponding boundary and feature lines of R and ^R. That is, the warping function w(s;t) in Eq. (2) is defined with respect to feature region ^R:
The background region, defined by the boundaries of all the feature regions, is handled in the same manner. The 2D warping function w(s;t)(u; v; ^R) can be chosen from any image morphing technique. In this paper, we employ the method of Beier and Neely [1], which is based on coordinate system transformations defined by
line segment movements. With this algorithm, the colors of the rays ^L0(u; v; s; t) in ^L0;(s;t) associated with ^R can be determined by sampling the rays L0(u0; v0; s; t) in the corresponding region R of L0;(s;t).
C. Occluded ray resampling :
As illustrated in Fig. 5, an object point visible in ^L 0;(s;t) may be occluded in L0;(s;t), which poses a problem in the ray resampling process. These occlusions are detected when a ray ^L0(u; v; s; t) in ^L0;(s;t) and its corresponding ray L0(u0; v0; s; t) in L0;(s;t) are associated 
with different feature regions. In such cases, a different corresponding ray must be resampled from other views in the light field L0, such that the resampled ray belongs to the correct feature region. Since the depth of the object point is not known and cannot be utilized, we take advantage of the correspondence of feature primitives among the views in L0 to determine this orresponding ray. For the feature region R in L0;(s;t) that contains an occluded point that is visible in ^L0;(s;t), let R0 be the corresponding region in another view L0;(s0;t0). For the 2D warping function w0 between R and R0, let L0(u00; v00; s0; t0) be the ray in R0 that maps to the ray L0(u0; v0; s; t) in R. To obtain the color value of L0(u0; v0; s; t) from another view in L0, we determinethe nearest view L0;(s0;t0) along the morph trajectory such that the ray L0(u00; v00; s0; t0) belongs to region R0 in L0;(s0;t0). Such a view in L0 must exist, since the complete object area of feature region R must be visible in some view of L0 for it to be specified by the user. To avoid self-occlusion within a feature region, the user should ideally specify regions that are approximately planar, as done in [12]. The effects of selfocclusion when feature regions are non-planar, however, are not obvious when the key views are not sampled too sparsely. When key views are not too far apart, the difference in self-occlusion area tends not to be large. Furthermore, although the resampling of the selfoccluded rays is imprecise, the rays are nevertheless resampled from the correct feature region, which is generally composed of similar appearance characteristics. In the experimental results of Section VI, the self-occlusion problem does not present noticeable morphing artifacts. Unlike [12], our 2D method does not employ a visibility map for recording feature visibility at all light field viewpoints. While visibility can easily be computed from the positions of 3D features in [12], the 2D approach requires the user to provide a depth ordering of features, which can change view-to-view. Hence, the user would need to partition the (s; t) camera plane into areas of like visibility, such that numerous key views are defined along the area boundaries. Clearly, this places a heavy burden on the user, so our implementation instead records visibility only along the camera path of the morph sequence.
V. ANALYSIS
In this section, we analyze the 2D feature-based light field morphing approach with respect to the ideal correspondence of light field rays, and then compare our technique with other methods for object morphing.
A. Comparison with ideal ray correspondence
We analyze the efficacy of our 2D feature-based approach for light field morphing by a comparison to the ideal ray correspondence given when the exact geometries of object O0 in L0 and O1 in L1 are known. Two components of morph generation, feature position
interpolation and ray coloring, are examined separately. We first consider the differences that arise from linear interpolations of feature positions. In ideal ray correspondence, feature point interpolation prescribes a linear trajectory in 3D between corresponding points (X0; Y0;Z0) on O0 and (X1; Y1;Z1) on O1. A perspective projection of this warp path onto the image plane (x; y) yields a linear trajectory in 2D between the corresponding object points, according to
The 2D linear interpolation of image warping employed in our method also produces a linear trajectory between the object points in image space:
This trajectory maintains a constant velocity in image warping; however, the velocity in the ideal ray correspondence varies according to changes in depth of the object point. In general, the depth variations between corresponding object points is small in comparison to
the viewing distance, so the 2D interpolation of feature positions closely approximates that of ideal ray correspondence.
The coloring of rays in the ideal ray correspondence depends on the surface normals and reflectance properties of the corresponding object points. For the case of Lambertian reflectance and a linear interpolation of surface normals, the ray color I® in ideal ray correspondence is
where ½ is albedo, n is surface normal, I is the ray color, and l is the light direction. In contrast, our 2D image morphing gives a linear interpolation of the observed colors:
While both of these methods produce a straight-line path of the morphed ray color from I0 to I1, the color of our 2D technique changes at a constant rate, and that of ideal ray correspondence depends on the surface normals n0 and n1. In general, the difference between the two surface normals is not large enough to produce a visible discrepancy between our 2D method and ideal ray correspondence.
The main deviation from ideal ray correspondence occurs when the reflectance is non-Lambertian, because surface normals must then be known to accurately determine the ray colors, especially in the presence of specular reflections. Although the visual hull of light
field objects can be computed from silhouette information [23], recovery of the surface normals cannot be performed reliably. This lack of precise geometric information is a tradeoff for the appearance detail provided by an image-based representation. Even with this drawback, the experiments on highly specular objects presented in Section VI nevertheless yield convincing results.
B. Comparisons with other approaches
An image is the simplest form of a light field. Hence, image morphing can be considered as a special case of light field morphing. User interfaces for image morphing
are quite similar to that of our light field morphing with 2D features in that only scattered features are specified. In image morphing systems, however, only feature correspondences are used to determine pixel correspondences between a pair of images, and there exists
no consideration of region association and occlusions. In contrast, our interface for light field morphing allows specification of regions which are used for handling visibility changes.
In image morphing by Beier and Neely [1], holes and overlaps are simply avoided by backward resampling in the image warping process. In [2], [14], one-to-one warp functions prevent holes and overlaps from disrupting the morph. In image-based rendering techniques, holes and overlaps due to visibility changes are usually handled by copying pixel values from neighbor pixels. These approaches, however, do not actually resolve the visibility problem and simply try to disguise its effects.
In comparison with 3D feature-based light field morphing [12], the major difference is the user interface. The 3D approach requires the user to specify 3D feature polygons, which is generally difficult to do accurately enough to avoid morphing artifacts. In contrast, the 2D approach is less sensitive to specification error because correspondences are given among multiple views. This less tedious method of user specification allows for simple handling of light fields containing complicated objects.
VI. RESULTS
We implemented the proposed 2D feature-based approach for light field morphing on a Pentium III Windows PC with 1.3 GHz CPU and 256M RAM. In this section, we show some experimental results using both synthetic and real data. “Queen” and “Deer & horse” are morphing examples with real data captured using a consumer-grade digital video camera, while the other examples were generated from 3D StudioMaxTM renderings. Our system produces animation sequences that allow the user to observe a morphing process from a camera moving along any chosen path in the (s; t)-plane of light fields or the viewing path of concentric mosaics. Morph trajectories outside of the captured views could
be generated using the animation technique described .
Experiments were performed to compare light field morphing using 2D features and 3D features. Quantitative data for these morphing examples is listed in Table VI. We note in particular the difference in user interaction times between the two methods. To obtain an accurate morph using 3D features, a considerable amount of care is required, which leads to long interaction times.
In Fig. 6 and 7, we exhibit results when a user’s interaction time for marking 3D features is limited to the time needed for 2D feature specification. Columns 1 and 4 in Fig. 6 highlight in yellow some of the specification errors, where the top row shows the result with the time constraint and the bottom row shows the result without the time constraint. Columns 2 and 3 compare the resulting morphs for ® = 0:5 between the two images. In the time it takes to accurately specify 2D features, the specification of 3D features cannot be performed accurately, as evidenced by the double imaging apparent in the nose. Only with a substantially longer interaction time period is the user able to produce satisfactory results.
Fig. 7 displays a comparison of the 3D feature approach (top row) with the 2D feature approach (bottom row) where the specification time is limited to that needed for 2D specification. It can be seen that the 2D method produces clearer morphing results, while the 3D method contains double images due to inaccuracies in user specification.
Morphing between two light fields:
We present three morphing results between light fields, which can be interactively displayed by light field rendering [21]. Fig. 8 exhibits a morph between faces of a Caucasian
male and an Asian male. Both light fields are 33 £ 33 in the (s; t)-plane and 256 £ 256 in the (u; v)-plane, rendered from two Cyberscan mesh models which were not used in the morphing process.
Figs. 9 and 10 provide examples of morphing between 360 £ 360 concentric mosaics. The bronze deer and bronze horse of Fig. 9 exhibit specular reflections and obvious occlusions in the legs. Fig. 10 demonstrates a 360± change of viewpoint for the morphing of two ceramic heads. The lack of texture in the ceramic material makes accurate user specification of feature points challenging. The surface reflectance properties in both
examples are difficult to model realistically with traditional graphics techniques, so they are best represented by image sets.
Non-uniform blending of two light fields:
In image morphing, non-uniform blending was introduced to generate more interesting intermediate images between the two original images [2], [24]. In non-uniform blending, different blending rates are applied to different parts of the objects. Light field morphing with 2D features can similarly be extended to allow non-uniform blending of two light fields.
To control the blending rates for an intermediate light field, we use the blending function, B(u; v; s; t; ¿ ) = ¯, 0 · ¯ · 1, which gives a blending rate ¯ for each ray (u; v; s; t) at a given animation time ¿ . Note that this definition of a blending function is a direct extension of those used in image morphing [2], [24]. We define a blending function for the rays in the source light field, though it could be defined for the target light field instead. To non-uniformly blend two input light fields, two modifications are made to our original method.
First, in determining the intermediate positions of feature polygons and feature lines, the blending coefficient is obtained by evaluating the blending function. Second, in ray color interpolation between the warped light fields, the blending rates assigned to the warped source rays are used.
Figure 11 displays a non-uniform blending between two light fields. One light field was captured from a real bronze statue, and the other was rendered from a model of Egyptian queen Nefertiti. Both light fields are represented by 3D concentric mosaics of length 200 and size 300£600. The surface of the antique bronze statue exhibits complicated non-uniformities that are difficult to model with a textured geometric model. The different
transition effects are evident in the head, which does not transform as rapidly as the neck and base of the morphed statue.
Morphing between an image and a light field :Since an image can be considered a simple light field, we can extend our method to morph between an image and a light field. Although an image and a light field have a different number of views, we can generate additional views from the image using the perspective distortion of view morphing [20].
To compensate for the reduced accuracy of the reconstructed views, we employ non-uniform blending. Let I0 and L1 be the given image and light field, respectively, where the view of I0 is matched to light field view L1;(s0;t0). Let L® be an intermediate light field nerated
from I0 and L1. We define a blending function such that the effect of I0 on L® is strong near the viewpoint (s0; t0) and diminishes when the viewpoint moves away from (s0; t0).
Fig. 12 exhibits an example of morphing between an image and a light field. The images of size 256 £ 256 in the upper row are obtained by view morphing [20], and the bottom row shows the morphing transformation to the light field. The light field data is identical to the target light field of Fig. 8 with a 33£33 (s; t)-plane and a 256 £ 256 (u; v)-plane.
VII. CONCLUSION
In this paper, we presented a novel approach for light field morphing which is based on 2D features. We also analyzed the ideal morphing of light field objects, and demonstrated that the 2D approach provides a reasonable approximation. Our method was extended to handle morphing between an image and a light field and to incorporate nonuniform blending. Experimental results demonstrate that the proposed approach generates good results. With the light field morphing technique proposed in this paper, 3D object morphing can be performed without specifying 3D primitives. The user interacts with the system in a manner similar to image morphing, but 3D morphing results can be obtained. With 2D features, high-quality light field morphs can be obtained with significantly simplified user interaction.
ACKNOWLEDGMENTS :
First of all, we want to thank Hua Zhong and Rong Xian, who developed early versions of the light field morphing system at Microsoft Research Asia. Since then, they have moved on to other exciting things and did not have time to work on the current system. Many thanks also to Sing Bing Kang for useful discussions and for supplying the Mike-Wang face models, Xin Tong for providing many helpful comments, and Zhunping Zhang for discussions on the initial idea and the UI. Finally, we thank the anonymous reviewers whose comments have tremendously helped to improve the final manuscript.
