Searching for an Ideal High-definition Video Streaming Technology

Edgar Huang, Ph.D. School of Informatics
Clifford C. Marsiglio, Testing Center
Indiana University–Purdue University Indianapolis

Methodology

Seven factors were measured in this study. They were:

  1. The cost to set up HD video streaming in a technology
  2. HD video encoding time
  3. Encoded HD video file size
  4. Perceived HD video image quality
  5. Buffering time
  6. Rebuffering frequency, and
  7. Ease of watching a video (Whether a video can be played without downloading anything)

The testing was done both in-house by the researchers and externally by testers through a survey.

 

In-house Testing

In May 2008, a 60-second-long HD video with a 1280x720-pixel resolution and 30fps frame rate containing both extremely fast motion and regular-speed motion was encoded in the following five HD video encoders:

  1. Adobe Flash CS3 Video Encoder
  2. Apple QuickTime Pro 7
  3. DivX Converter Pro 6.8
  4. Microsoft Windows Media Encoder 9 Series
  5. Vividas Vivcast On-demand Encoder

Two computers of similar builds were used for encoding. One was Dual Core AMD Opteron with a Processor 275, 2.21 Ghz, 2GB Ram installed with the Linux Ubuntu 6.06 operating system as required for encoding the Vividas streaming video, and the other, a Dell Precision WorkStation 470 with Intel(R)Xeon(TM) CPU 2.8GHz 2.79GHz and 1534MB memory installed with Windows Vista 32 bit for encoding the rest of the streaming videos. The same video was fed into the different encoders except for Windows Media Encoder. In that case, since the original MOV file was not recognizable to the Windows Media Encoder, it had to be converted to an AVI file, and then the AVI file was converted to the WMV file by the Windows Media Encoder.

Since the ITIF April 2007 report stated that the U.S. average Internet connection speed was 4800Kbps, the encoding of the video in all encoders was based on 4000Kbps with an extra 128Kbps for audio. All encoding was done on a two-pass basis when such an option is available. The encoding time was from the time the media encoding starts and the encoding ends. Preparation time for encoding was not counted. While encoding, no other programs were running except for the online stopwatch. A screen from exactly the same fast-motion moment in the video was captured from all five encoded videos for image quality comparison. Another screen of the regular-speed motion moment was also captured for the same purpose.

The Web authoring for these encoded videos was done in a way that full-screen viewing was possible for each of the five videos. Since QuickTime video and Windows Media video were impossible to be viewed full-screen if they were embedded, they were shown by being linked to an external player.

Data regarding encoding time and encoded file size for all five technologies were collected during the in-house testing. The information regarding the cost of setting up an HD video streaming in each technology was either found on the vendors' Web sites or by consulting with a vendor directly.

 

External Testing

Historically, video image quality and streaming quality have been measured either with an objective approach or a subjective approach.17 The objective approach uses a device and/or software while the subjective approach uses human testers. This study has used a subjective approach based on the philosophy that, after all, the videos are produced for human beings to consume and they have the final say about such qualities.

All the encoded and Web-authored videos were put on the servers for external testers to test. Here is the information for the different servers involved:

A survey Web site was developed to collect the test data regarding user accessibility, streaming quality (buffering and re-buffering) and perceived image quality. The Web site was designed in such a way that users’ operating system (PC or Mac), bandwidth, Web browser and IP address can be automatically detected. An invitation was issued through multiple channels such as discussion board, listserv, mailing list and so on related to streaming media, video, new technology and visual communication for professionals from different geographic areas to participate in the test. Since this study was not based on a systematic random sampling, the findings and conclusions from this study can only be applied to the participants.

The survey site told the participants that a minimum 5000Kbps broadband connection was needed for testing. The survey asked the participants whether, in each technology, they had to download anything to watch the movie, how much time they had to wait to watch the movie, and how much rebuffering they experienced. They were also asked to name the technology that produces the best video image quality based on the screen captures.

Each technology will be ranked on each of the seven factors. Since people may disagree on which factor is more important than which, we decided not to weigh any factors. As a result, all ranking scores each technology earned were averaged. Each technology had an overall score that represents its place in the overall ranking. The lower the score, the closer to 1, the better. Since some streaming media producers may put emphasis on users' streaming media usage experience no matter how much effort the producer has to put into the production and how much the production costs, a user-experience-only version will also be presented.

Notes:

17. See “Video Streaming Quality Measurement with VSQI—Technical Paper” and Shai Berger’s “Measuring and Monitoring Streaming Media Quality.”

NEXT