Edgar Huang, Ph.D. School of Informatics
Clifford C. Marsiglio, Testing Center
Indiana University–Purdue University Indianapolis
Findings
The testers were required to have a minimum 5000Kbps Internet connection. Viewers who do not have a 5000Kbps connection can still view videos encoded at the above bit rate, but it is very likely they will encounter longer buffering and frequent rebuffering. To make the test results reasonable, the authors of this study decided to make 1000Kbps as a cut-off line and to separately report the results from the testers with a 1000Kbps or above connection and from the ones with a 5000Kbps or above connection to see if any obvious differences exist.
In total, 140 testers tested the five HD videos within seven days from late May to early June of 2008. A great majority of the testers (94.6%) came from the United States, and the rest from Canada and the U.K. Testers from the United States represented 31 states. Among these testers, four testers (3%) had a connection below 1000Kbps. As a result, these four testers were dropped out of the rest of the test report. Among the remaining 129 testers, 76% had a 5000Kbps and above connection.
Here is the breakdown of the Web browsers and operating systems all the testers used: Firefox: 42.6%, Safari: 30.2%, and Internet Explorer: 27.1%; Mac OSX: 61.2% and Microsoft Windows: 38.8%. Since most testers were engaged in new media or network development, this picture obviously does not represent the real-world situation in either Web browsers or OS usage, as can be found out at thecounter.com.
Following are the findings from this study.
In the following chart, the lower the ranking number, the closer to 1, the better. Click on the title of each technology on top to view the sample videos. Click on the titles of the dependent variables on the left side for visual presentations of the data.
Adobe Flash CS3 Video Encoder |
Apple QuickTime Pro 7 |
DivX Converter Pro 6.8 |
Microsoft Windows Media Encoder 9 Series |
Vividas Vivcast On-demand Encoder |
|
Video bitrate/encoder |
ON2VP6 |
H.264 |
MPEG4 |
WMV9 |
VP6.2 |
Audio bitrate/encoder (128Kbps for all) |
MP3 |
Hinted AAC |
MP3 |
WMA 9.2 |
Ogg Vorbis III |
$0 for Adobe Flash CS3 Video Encoder $995 for Adobe Flash Media Streaming Server 3 Total: $995 |
$29.99 for Apple QuickTime 7 Pro $0 for QuickTime Streaming Server (Requires Mac OSX Server 10.4+) $0 for Darwin Streaming Server (Requires Mac OSX Workstation or Server, Windows 2003 or 2008 Server, or Red Hat Linux) Total: $29.99 |
$19.99 for Dix 6.8 Pro No dedicated server is required because all DivX media are served on a Web server.
Total: $19.99 |
$0 for Windows Media Encoder $3999 for Windows 2008 Server: 25 user license Total: $3999
|
$900/month (12 month commitment)
On Demand HD
Basic Vividas encoder leased to client for high definition on-demand content
Total: $900x12x3years = $32,400 (Based on the fact that many companies/ schools upgrade technology every three years) |
|
Cost ranking |
3 |
2 |
1 |
4 |
5 |
42.12 |
15.14 |
12 |
19.22 |
62 |
|
Encoding time ranking |
4 |
2 |
1 |
3 |
5 |
32 |
55.1 |
30 |
32 |
30 |
|
Encoding media file size ranking |
3 |
5 |
1 |
3 |
1 |
| The following survey findings are based on the 5000Kbps and above Internet connections | |||||
21.7 |
65.2 |
2.2 |
5.4 |
5.4 |
|
Image quality ranking |
2 |
1 |
5 |
3 |
3 |
97.8 |
56.5 |
85.9 |
24.7 |
11.1 |
|
Initial buffering duration ranking |
1 |
3 |
2 |
4 |
5 |
50.5 |
39.6 |
63.6 |
51.7 |
55.8 |
|
Frequency of rebuffering ranking |
4 |
5 |
1 |
3 |
2 |
96.7 |
84.1 |
31.9 |
60 |
64.1 |
|
Can play ranking |
1
|
2 |
5 |
4 |
3 |
Average Ranking Score
with all factors included
|
2.57 |
2.86 |
2.29 |
3.43 |
3.57 |
Ranking |
2 |
3 |
1 |
4 |
5 |
Average Ranking Score
with only the audience factors (image quality, latency, rebuffering, and can play w/o downloading anything) included |
2 |
2.75 |
3.25 |
3.5 |
3.5 |
Ranking |
1 |
2 |
3 |
4 |
4 |
| The following survey findings are based on the 1000Kbps to 4999Kbps Internet connections | |||||
25.9 |
66.7 |
0 |
3.7 |
3.7 |
|
Image quality ranking |
2 |
1 |
5 |
3 |
3 |
86.7 |
48.3 |
54.2 |
16.7 |
7.4 |
|
Initial buffering duration ranking |
1 |
3 |
2 |
4 |
5 |
23.3 |
32.1 |
50 |
18.2 |
37.5 |
|
Frequency of rebuffering ranking |
4 |
3 |
1 |
5 |
2 |
83.3 |
64.3 |
22.6 |
60 |
68 |
|
Can play ranking |
1
|
3 |
5 |
4 |
2 |
Average Ranking Score
with all factors included
|
2.57 |
2.71 |
2.29 |
4 |
3.29 |
Ranking |
2 |
3 |
1 |
5 |
4 |
Average Ranking Score
with only the audience factors (image quality, latency, rebuffering, and can play w/o downloading anything) included |
2 |
2.5 |
3.25 |
3.5 |
3 |
Ranking |
1 |
2 |
4 |
5 |
3 |
Interactive Chart Weightings
(Click the following chart to manipulate the weights based on your need.)
|