Practical generation of video textures using the auto-regressive process

• Published in: Image and vision computing 2004-09, Vol.22 (10), p.819-827
• Main Authors: Campbell, Neill, Dalton, Colin, Gibson, David, Oziem, David, Thomas, Barry
• Format: Article
• Language: English
• Subjects: Auto-regressive process; Image-based rendering; Motion signatures; Principal component analysis; Video textures
• ISSN: 0262-8856; 1872-8138
• DOI: 10.1016/j.imavis.2004.02.008

Recently, there have been several attempts at creating ‘video textures’, that is, synthesising new (potentially infinitely long) video clips based on existing ones. One method for achieving this is to transform each frame of the video into an eigenspace using Principal Components Analysis so that the original sequence can be viewed as a signature through a low-dimensional space. A new sequence can be generated by moving through this space and creating ‘similar’ signatures. These signatures may be derived using an auto-regressive process (ARP). Such an ARP assumes that the signature has Gaussian statistics. For many sequences this assumption is valid, however, some sequences are strongly non-linearly correlated, in which case their statistical properties are non-Gaussian. We examine two methods by which such non-linearities may be overcome. The first is by modelling the non-linearity automatically using a spline, and the second using a combined appearance model. New video sequences created using these approaches contain images never present in the original sequence and appear very convincing.