Preview
Properties
24 bit display
8 bit display
Size: 320x224
Frame Rate: 12 fps
Length: 300 frames (25 s)
Max Bit Rate: 1 Mbps
Play...
Play...
Wai-tian Tan Avideh Zakhor
This work is supported in part by NSF under grant ANI-9905799.
Our main goal is to investigate video compression techniques and transport protocols needed for real-time delivery of unicast and multicast video over best effort networks such as the Internet. Our approach for the unicast case is to combine a novel compression method that is error resilient and bandwidth scalable with a low-delay TCP-friendly transport protocol. Specifically, compressed video is packetized into individually decodable packets of equal expected visual importance. Consequently, relatively constant video quality can be achieved at the receiver under lossy conditions. Furthermore, the packets can be truncated to instantaneously meet the time varying bandwidth imposed by a TCP-friendly transport protocol. As a result, adaptive flows that are friendly to other Internet traffic are produced. Actual Internet experiments together with simulations are used to evaluate the performance of the compression, transport, and the combined schemes.
In the context of multicast, the use of scalable video with layered multicast has long been shown to be an effective method to achieve rate control in heterogeneous networks. In terms of error control, we propose a layered FEC framework for scalable video in which different receivers can individually, and efficiently trade-off latency for reception quality based on their respective channel conditions and latency tolerances. The scheme is efficient because (1) the use of scalable video facilitates unequal error protection to better utilize the bits for FEC, and (2) the layered nature of the FEC guarantees minimal delivery cost.
Our current on-going research looks beyond best-effort networks and evaluates several methods of exploiting a DiffServ network for transport of MPEG compressed video.
Error-resilient, Scalable Compression and Unicast Transport
There are two main requirements for Internet video communications: (1) bandwidth adaptability; (2) error-resilience. Lack of bandwidth adaptability results in non-adaptive streams with two sets of disadvantages. First, it leads to congestion collapse when the aggregate bandwidth of the video traffic exceeds network capacity. Second, it competes unfairly with other adaptive traffic, such as TCP, which reduces transmission rate in face of network congestion. Existing approaches to bandwidth adaptability for video traffic over the Internet often require online encoding, which does not extend to pre-encoded material, or transcoding which introduces additional complexity and latency.
Lack of error resilience results in error propagation, and hence widely varying video quality as a function of time. Current approaches to mitigate the effects of error propagation include error control mechanisms at the transport level. This typically takes the form of retransmissions or forward error correction (FEC). Retransmission based error control methods often fail to be real-time, particularly when round-trip propagation delay is large. FEC schemes on the other hand, are often ineffective when losses are bursty.
An attractive alternative is to use bandwidth scalable video with feedback rate control whereby transmission sources adjust their rates in response to changing network conditions. This would solve the bandwidth adaptability problem while the error control problem can be solved by further requiring the compression method to be resilient to packet losses. We propose one such compression algorithm [1,3,4] based on 3-D subband coding, data partitioning in the frequency domain, layered quantization followed by hierarchical block coding [5]. The video bit-stream is scalable and can be truncated to meet the time-varying transmission budget of any flow-controller. Furthermore, the produced packets can be decoded independently and are of roughly equal importance.
We find that the compression scheme admits real-time encoding and decoding with little loss in compression efficiency compared to MPEG-1 at no loss[1,4]. We perform simulated packet loss studies and find that the proposed compression scheme produces relatively constant video quality in face of packet losses as compared to MPEG and several subband-based packetization methods[1,3] . We also perform actual Internet experiments comparing another bandwidth-scalable compression scheme which is similar to the proposed compression scheme except with dependencies among packets. We find that the ability to independently decode packets reduces the resulting distortion significantly even after an adaptive FEC scheme is applied to protect the non-resilient bit-stream [1,3]. The unicast experiments are performed using a TCP-friendly transport protocol that dynamically estimates the throughput of a TCP connection as a basis for rate-control.
The Related Publications section contains links to some relevant papers that are published.
Unicast Streaming Demo and Software Download
Please see the download page for instruction on how to download and install the decoder. Currently, the decoder is only available on the Solaris and Windows NT platforms. After you install the players, you should be able to click at the links to start streaming videos.
However, please note the the following limitations. (1) Each server allows for only 1 connection at one time. (2) We assume that the decoder machine has enough CPU power to decode all the received material fast enough. We did not program the decoder to scale down the bit-rate according to the running machine's CPU power to avoid masking out the effect of the bandwidth adaptation algorithm.
When you click the "Play..." button, a small parameter file will be
downloaded that contains information about the video, like picture sizes, frame rate etc,
as well as the IP address and UDP port number to contact the video server. The
decoder will then contact the video server to initiate the flow of video.
Preview |
Properties |
24 bit display |
8 bit display |
|
Origin: Raiders of the Lost Ark Size: 320x224 Frame Rate: 12 fps Length: 300 frames (25 s) Max Bit Rate: 1 Mbps |
Play... |
Play... |
Disk Demo
Please see the download page for instruction on how to download the decoder.
To decode from a file without using frame-rate scalability, an example bitstream and parameter file are provided. Please rename the bit-stream file to bits.disk if it is saved as another name. The original source is of 24 fps and is coded at 1Mbps. The default decoding rate is 200 kbps. You can change the decoding rate without re-encoding by changing the parameter file. Refer to this document for descriptions of the parameter file.
To decode from a file using frame-rate scalability, an example bitstream and parameter file are provided. Please rename the bit-stream file to bits.fs if it is saved as another name. The original source is of 24 fps and is coded at 1Mbps. The default decoding rate is 200 kbps and at 12 fps.
Frame-rate Scalability
One well-known problem of SNR-scalable video compression in low-bit rates is temporal aliasing. If the picture sizes and frame rates are fixed, then as the bit rate decreases, the temporally low-pass or averaged frame becomes the best representation for the entire set of frames that are compressed together. Due to motion in the input frames, the frame will appear smeared in areas of high motion. To yield better image quality at lower bit rates, one possible approach is to use a lower frame rate at the lower bit-rates. To produce such a bitstream, we first split the input frames into an odd and an even thread, with the odd thread containing the odd frames and even thread containing the even frames. We then code the odd threads in a scalable manner up to a rate R. Thus, decoding at any rate below rate R will yield information about only half of the original frames. While still lower frame rates are available by using the temporally low-pass frames of the odd threads, some temporal aliasing will result. At rates above R, bits can be allocated freely to either the odd frame or the even frame to yield full frame-rate video.
Multicast Transport - Hierarchical FEC
In a multicast environment, besides bandwidth heterogeneity, different viewers may experience different packet loss characteristics and may have different degrees of tolerance to latency. For example, in live multicast of a lecture, participants who want to ask questions and interact with the lecturer desire stringent real-time constraints on the video while passive viewers may be willing to sacrifice latency for higher video quality. Because of the high bandwidth required for video traffic, generating a separate stream for every possible set of user requirements and channel loss rate is infeasible. Instead, we wish to cater to different users with varying latency requirements by providing a framework in which each user can individually trade-off latency for quality.
In this work, we investigate the application of layered FEC as an error control method for scalable video multicasting. By layered FEC, we mean the production of FEC or redundancy streams, each of which are carried using a different multicast group. In such a way, subscribing to more groups corresponds to higher level of protection. There are two advantages to using layered FEC. First, each receiver can independently adjust the desired level of protection based on past reception statistics and the application's delay tolerance. This receiver-driven approach to FEC is more flexible than most existing FEC schemes for multicast where the source determines a set of redundancy packets which are then multicast to every recipient. Second, each receiver will subscribe to only as many redundancy layers as necessary, reducing overall bandwidth utilization. Furthermore, the multicast nature of the FEC layers ensures minimum network utilization through sharing of common streams. An effective means to combat burstiness in packet losses is suggested in [8] where redundancy packets are transmitted at a later time than the actual data packets. Following [8], we delay the transmission of FEC layers to relieve the effects of bursty losses. The additional latency associated with the FEC layers creates a trade-off between higher levels of protection and increased latency, and also as a disincentive for receivers to unnecessarily use FEC.
Preliminary results using FEC for only the first video layer are presented in Packet Video Workshop 99 [2] and the HTML version of the paper is available. A journal version is currently under review.
The low error-resilience of MPEG compressed video have limited its use for Internet streaming. Yet most existing contents in the forms of VCD and DVD are compressed using MPEG. The recent proposal in IETF to augment the best-effort service model of the Internet with Differentiated Services offers an enhanced framework under which MPEG video can be transported. In this work, we explore different methods of transporting MPEG compressed video stream over a DiffServ network. The work is still on-going, and a paper is in preparation.
[1] W.Tan and A.Zakhor. Real-time Internet Video Using Error Resilient Scalable Compression and TCP-friendly Transport Protocol. IEEE Trans. Multimedia, Vol. 1, No. 2, pp 172-186, June 1999. [pdf]
[2] W.Tan and A.Zakhor. Multicast Transmission of Scalable Video using Receiver-driven Hierarchical FEC. Packet Video Workshop 99, April 1999. [html]
[3] W.Tan and A.Zakhor. Resilient Compression of Video for Transmission over the Internet. Proc. 32nd Asilomar Conf. Signal, Sys. and Computers, November 1998. [pdf]
[4] W.Tan and A.Zakhor. Internet Video using Error Resilient Scalable Compression and Cooperative Transport Protocol. Proc. ICIP, Vol. 3, pp 458-462, October 1998. [pdf]
[5] W. Tan, E. Chang, and A. Zakhor. Real Time Software Implementation of Scalable Video Codec. Proc. ICIP, Vol. 1, pp 17-20, September 1996. [pdf]
[6] D.Taubman and A.Zakhor. Multirate 3-D Subband Coding of Video. IEEE Trans. Image Proc., Vol. 3, No. 5, pp 572-88, September 1994.
[7] D. Taubman and A. Zakhor. A Common Framework for Rate and Distortion Based Scaling of Highly Scalable Compressed Video. IEEE Trans. CSVT, Vol. 6, No. 4, pp 329-354, August, 1996.
[8] J.Bolot and A.Vega-Garcia. The case for FEC based Error Control for Packet Audio in the Internet., ACM Multimedia Sys.1997.
[9] W.Tan. Slides for Icip99 [pdf]
Last modified 6/1/00