Free-Form Object Reconstruction from Silhouettes, Occluding Edges and Texture Edges: A Unified and Robust Operator Based on Duality

• Published in: IEEE transactions on pattern analysis and machine intelligence 2008-01, Vol.30 (1), p.131-146
• Main Authors: Shubao Liu, Kongbin Kang, Tarel, J.-P., Cooper, D.B.
• Format: Article
• Language: English
• Subjects: 3D reconstruction robust to degeneracies and noise; Accuracy; Algorithms; Applied sciences; Artificial Intelligence; Computer Science; Computer science; control theory; systems; dual manifold; duality in differential form; dynamic programming; Exact sciences and technology; Image edge detection; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image Processing; Image reconstruction; Imaging, Three-Dimensional - methods; Information Storage and Retrieval - methods; Interpolation; Manifolds; Multi-stage noise shaping; multi-view reconstruction; Noise robustness; Noise shaping; Operators; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Planes; Reproducibility of Results; Sensitivity and Specificity; Shape; shape from occlusions and textures; shape from silhouettes; Studies; Surface layer; Surface reconstruction; Surface texture; Surface treatment; Texture; Three dimensional
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2007.1143

In this paper, the duality in differential form is developed between a 3D primal surface and its dual manifold formed by the surface's tangent planes, that is, each tangent plane of the primal surface is represented as a four-dimensional vector that constitutes a point on the dual manifold. The iterated dual theorem shows that each tangent plane of the dual manifold corresponds to a point on the original 3D surface, that is, the "dual" of the "dual" goes back to the "primal." This theorem can be directly used to reconstruct 3D surface from image edges by estimating the dual manifold from these edges. In this paper, we further develop the work in our original conference papers resulting in the robust differential dual operator. We argue that the operator makes good use of the information available in the image data by using both points of intensity discontinuity and their edge directions; we provide a simple physical interpretation of what the abstract algorithm is actually estimating and why it makes sense in terms of estimation accuracy; our algorithm operates on all edges in the images, including silhouette edges, self occlusion edges, and texture edges, without distinguishing their types (thus, resulting in improved accuracy and handling locally concave surface estimation if texture edges are present); the algorithm automatically handles various degeneracies; and the algorithm incorporates new methodologies for implementing the required operations such as appropriately relating edges in pairs of images, evaluating and using the algorithm's sensitivity to noise to determine the accuracy of an estimated 3D point. Experiments with both synthetic and real images demonstrate that the operator is accurate, robust to degeneracies and noise, and general for reconstructing free-form objects from occluding edges and texture edges detected in calibrated images or video sequences.