Just for fun, I tested Big Buck Bunny being served through IIS Media Services from the local disk, running on an Amazon EC2 Micro instance (the smallest available). This is a fairly lightweight server, i.e. something you would never want to run in a production environment. Consider this the streaming equivalent of “Will it Blend?”…
The results are in – when it’s going well, it’s surprisingly great, and when it’s not going well, it’s terrible. 😉
Total Chunks: 2691
Chunks Tested: 2691
Manifest Download Time: 310ms
Avg Video Chunk Response Time: 357ms
Avg Audio Chunk Response Time: 61ms
Excellent Chunks: 590 (21.92%)
Good Chunks: 1821 (67.67%)
Warning Chunks: 68 (2.53%)
Degraded Chunks: 6 (0.22%)
Poor Chunks: 206 (7.66%)
Stream 0: video, 2962kbps, 2056kbps, 1427kbps, 991kbps, 688kbps, 477kbps, 331kbps, 230kbps
Stream 1: audio, 128kbps
Things go OK around 89% of the time. But when they don’t, it crashes and burns…
Yikes. You can almost feel the Silverlight players freaking out. 😉
If you are deploying a Smooth Streaming infrastructure, you already know it’s all HTTP-based. Requests for little video chunks hit your web server, and your web server looks up the correct video “chunk” within the audio/video file and serves it up.
However, it’s can be difficult to get a good benchmark on how your infrastructure is doing at serving up chunks, especially when your Silverlight clients are seeing random buffering errors or you run into scaling problems.
First off, there is a lot of information available from the Smooth Streaming Health Monitor app – In a couple seconds you can have a trace of what decisions the Silverlight Adaptive Streaming Media Element is making and export that out to Excel.
But when you just need comprehensive chunk data on all bitrates to diagnose how your origin/CDN is doing, I made this app (almost called it “Chunker”):
Enter the manifest URL of the on-demand smooth stream you want to test (note that this does not currently support live or composite manifests), for example: http://server.com/streams/BigBuckBunny.ism/Manifest
Once you click Begin Test, a new test tab will open and start requesting chunks based on the manifest information. The results will tell you if you may have a problem with your disk IO on your origin or some other problem preventing chunks being delivered in a timely manner.
Note: Only the first 1000 bytes (almost 1K) of each chunk is downloaded. The point here is not to test bandwidth, but rather test your infrastructure’s performance as it relates to reading/seeking fragments and assembling chunks.
Hopefully based on the assessment of each chunk, you can get an idea of how your CDN / origin / standalone box is doing at delivering chunks.
Run Smooth Stream Performance Testing Tool (ClickOnce)