Temporal and spatial scaleable coding for video object planes

United States Patent [w]

Chen et al.

US006057884A [ii] Patent Number: [45] Date of Patent:

6,057,884 May 2, 2000

[54] TEMPORAL AND SPATIAL SCALEABLE CODING FOR VIDEO OBJECT PLANES

[75] Inventors: Xuemin Chen; Ajay Luthra; Ganesh Raj an; Mandayam Narasimhan, all ol

San Diego, Calil.

[73] Assignee: General Instrument Corporation,

Horsham, Pa.

[21] Appl. No.: 08/869,493 [22] Filed: Jun. 5, 1997

[51] Int. C I. II04N 7/32

[52] U.S. CI 348/416; 348/397; 348/398;

348/415; 348/438; 348/699; 382/243

[58] Field of Search 348/397-400,

348/402, 407-408, 412-413, 415-416, 438, 699-700; 382/242, 243, 236, 238, 244; H04N 7/32, 7/34, 7/36, 7/46

[56] References Cited

U.S. PATENT DOCUMENTS

5,686,956 11/1997 Oh et al 348/416

5,692,063 11/1997 Lee et al 382/107

5,757,971 5/1998 Kim 271/241

5,778,100 7/1998 Chen et al 348/700

5,790,196 8/1998 Sun et al 348/419

5,802,220 9/1998 Black et al 382/276

5,805,736 9/1998 Kim 382/242

FOREIGN PATENT DOCUMENTS

0 639 031 2/1995 European Pat. Off. .

OTHER PUBLICATIONS

Tseng B. L. et al., "Compatible Video Coding ol Stereoscopic Sequences Using MPEG-2's Scalability and Interlaced Structure," Proceedings of the International Workshop on HDTV, Oct. 26, 1994, pp. 369-378. ISO-IEC JTC1/SC29/WG11 Video Subgroup: "ISO/IEC 13818-2 Amendment 3 (Multi-View Profile)," Sep. 1996, ISO/IEC, Chicago.

Sikora, "The MPEG^l Video Standard Verification Model", IEEE Trans, on Circuits and Systems for Video Technology, vol. 7, No. 1, pp. 19-31, Feb. 1997.

Ferman et al, "Motion and Shape Signatures for ObjectBased Indexing ol MPEG-4 Compressed Video", IEEE, pp. 2601-2604, 1997.

Packwood et al, "Variable Size Block Matching Motion Compensation lor Object-based Video Coupling", IEEE, pp. 56-60.

ISO/IEC JTC1/SC29/WG11 N2196, Coding ol Moving Pictures and Audio, "Overview ol the MPEG-4 Version 1 Standard", Mar. 1998.

Liu et al, "A JAVA-Based MPEG-4 Like Video Codec", IEEE Trans, on Consumer Electronics, vol. 44, No. 1, pp. 200-205, Feb. 1998.

Mulroy, "VRML Gets Real the MPEG-4 Way", Applied Research and Technology, BT Laboratories.

Lee et al., "A Layered Video Object Coding System Using Sprite and Affine Model", IEEE Trans, on Circuits and Systems for Video Technology, vol. 7, No. 1, pp. 130-145, Feb. 1997.

Temporal and spatial scaling ol video images including video object planes (VOPs) in an input digital video sequence is provided. Coding efficiency is improved by adaptively compressing scaled field mode video. Upsampled VOPs in the enhancement layer are reordered to provide a greater correlation with the input video sequence based on a linear criteria. The resulting residue is coded using a spatial transformation such as the DCT A motion compensation scheme is used for coding enhancement layer VOPs by scaling motion vectors which have already been determined for the base layer VOPs. A reduced search area whose center is defined by the scaled motion vectors is provided. The motion compensation scheme is suitable for use with scaled frame mode or field mode video. Various processor configurations achieve particular scaleable coding results. Applications of scaleable coding include stereoscopic video, picture-in-picture, preview access channels, and ATM communications.

36 Claims, 8 Drawing Sheets