Compression,rendering and transmission for 3D and scalable video.
详细信息
文摘
Recent advances in network infrastructure and hardware technologies enable two trends in the development of multimedia applications: increasing spatial, temporal, and viewpoint resolution to enhance viewing experience； and focusing more on user variety to provide more adaptive services to heterogeneous end users. These introduce three new challenges to multimedia technologies. First, a more efficient compression scheme is necessary for the increased data volume； second, new techniques should be developed for the new applications enabled by those new types of video data； third, the heterogeneity of links and devices demand that multimedia content delivery should be adaptive to different device capabilities, variable network conditions and diverse quality of service requirements. In this proposal, we address these challenges with designing new compression methods and virtual view rendering for 3D video, as well as creating new necessary tools for video transmission, respectively. In the first study, we present our efforts in 3D video compression. As a new type of video in multimedia, 3D video is attracting much interest from academia and industry. Since depth is a new component in 3D video compared to conventional 2D video, we focus on depth compression in our research. We observed structural similarity between depth and corresponding video, and proposed algorithms to efficiently compress depth with utilization of such similarity. In the second study, we address a unique requirement of 3D video - rendering virtual views based on captured video and depth sequences. We propose the use of non-rigid object tracking or a human motion model to match previously captured frames to the current frame, in order to recover dis-occluded areas using information in previous frames. In the third and fourth studies, we address the problem of video transmission over various networks with heterogeneous terminal devices. We first propose a 3D video transcoder design to adaptively extract and transmit a required video sequence to the end user. We also proposed a channel distortion modeling scheme for scalable video transmission, where we systematically track the error propagation due to packet loss. The proposed algorithm is able to estimate packet loss induced frame distortion of the received video with high accuracy.