Silverlight Smooth Streaming: WAN optimization or Content Distribution Network?

I received an interesting question from someone setting up their own Smooth Streaming infrastructure recently, and I couldn’t help but blog it.

When setting up an on-demand Smooth Streaming infrastructure for a corporate environment, is it better to invest in WAN optimization, or Content Distribution?

It’s a great question, but let’s take a quick time out to make sure we’re on the same page with these terms.

WAN Optimization, in this case, is a device that functions as a web proxy at the front end of a network. It downloads data and caches it for the next person to conserve internet bandwidth. Simply put, if Bob and John are on a network using WAN Optimization, and Bob visited Google.com, the WAN optimizer would cache the Google logo image. When John visits Google.com a few minutes later, the WAN optimizer serves up the Google logo image from its cache, instead of requesting it again from Google’s servers. There are a few products that do this from a variety of manufacturers (for example, ISA Server from Microsoft).

On the other hand, a Content Distribution Network is a network of servers throughout the world on a really fast backbone with synchronized caching. End-users are routed to the closest/fastest content distribution server near them, and content is delivered at really zippy speeds.

So now that we have that down, the question becomes: Is speed (CDN) or caching (WAN Optimization) the better strategy for On-Demand Smooth Streaming?

The answer, of course, is: It depends.

Exploring Caching (WAN Optimization)

The reason Smooth Streaming works so well is illustrated in this graphic:

While traditional streaming is easy to work with (and easy to cache), there’s a quality tradeoff. Smooth Streaming works differently, in that the client player can dynamically adjust to network conditions, and constantly switch up the bitrate all willy-nilly. Typically, Smooth Streaming “chunks” are requested in 2 second intervals. This generates 8 possible URLs for every 2 seconds of video, plus another 1 URL for every 2 seconds of audio. Compared to a traditional progressive-download stream of a single-bitrate file, this can get interesting quickly:

Number of URLs possibly used to play a video

Video Length	Traditional 1-bitrate Streaming URL Count	8-bitrate Smooth Streaming URL Count
2 seconds	1	9
60 seconds	1	270
120 seconds	1	540
300 seconds (5 min)	1	1,350

 

 

If this doesn’t make sense, take a moment to read Demystifying IIS Smooth Streaming / Live Smooth Streaming.

You might be thinking this is game over for WAN Optimization already, because there’s no way that caching could be effective when a client will only use 11% of the possible URLs available. That’s true, but it doesn’t take into account what actually, or I should say typically, happens.

This is a curve of typical usage we see. Silverlight clients for users on decent home or corporate connections will usually hit their stable bandwidth within 10 seconds, and won’t deviate a whole lot. This brings a lot more predictability into Smooth Streaming – the “in between” bitrates aren’t used much, and the more stable your network, the less jumping around you’ll see. In other words, WAN Optimizer is back in the game. In fact, we tested a small number of videos with just a few users watching over a couple days (using the IIS7 Application Request Routing module with caching), and achieved an over 70% cache rate.

Exploring Content Distribution Networks

The most well-known CDN in the world is currently Akamai. They operate a CDN made up of over 61,000 servers. There are also other popular ones (for example, we use both Akamai and EdgeCast for my company’s Smooth Streaming platform). They are typically deployed in one of two ways; either you provide the origin (content servers) and the CDN retrieves content from you dynamically as users request it, or you place the content directly on the CDN network.

The idea is that you can deliver high-speed content anywhere in the world over a CDN, by basically placing your content geographically “next door” to your viewers. A CDN could very well mitigate the need for WAN optimization, especially if your viewers are scattered in multiple locations.

Make a Decision Already

So, what’s most effective? It depends on the number of videos vs. the number of users on the same network. I summarize my cumulative experience and testing in the following decision grid:

If you have a low number of videos and a high number of users, WAN Optimization will provide Smooth Streaming infrastructure savings.

On the other hand, if you have a high number of videos and a lower number of users, your WAN caching device is a waste of money, and you may as well go with a CDN.

If you have lots of videos and lots of users, then a hybrid strategy is most likely to be effective.

Shameless Plug

This is one of the many problems I’ve had to address with Smooth Streaming technology. Like most technologies, it has an occasional tradeoff in exchange for some great benefits, and this is one of them.

One of my primary efforts over the last few months has been heavy design and development on a cloud-based Smooth Streaming platform that solves problems like this one. You may notice that some of the graphics in this post came from Nimbus (www.nimbushd.com). I figure it may be helpful to shed some light on how the design for Nimbus solves this issue, even if it’s considered a shameless plug.

Nimbus is designed to fit in to all 4 spots on the decision grid above, with multiple CDN provider integration. Media assets in Nimbus are divided up into groups (called Media Bins), and by default, the bins are configured for distribution through Akamai. The idea is to provide the highest quality distribution to a virtually unlimited number of users. However, CDN distribution costs money, and for larger corporations, costs add up. For a corporation providing training or news videos to a large number of internal users, it doesn’t make sense to use a CDN to stream the same content over and over again to the same corporate network, so Nimbus supports the hybrid CDN+WAN model, allowing Media Bins to be changed on the fly between CDNs and on-site WAN caching configurations for the best possible cost savings and caching efficiency.

Hopefully between the theoretical discussion points and the actual Nimbus design, this sheds some light on how to make the WAN Optimization vs. CDN decision. Happy Streaming!

Another great quote

Blogging these here just so I don’t forget them… via @indefensible -

How sad it must be to define yourself solely by what you’re against.

On Change…

Saw a great quote today.

"Change is hard because people overestimate the value of what they have—and underestimate the value of what they may gain by giving that up."

MOSS 2007 – How Web.Config Modifications Work

Background

When adding custom development functionality to MOSS/SharePoint 2007, making changes to Web.Configs is not as straightforward as a regular ASP.NET application. The reason for this is that MOSS uses a multiple-server model, and even if you have only one server, it’s still considered a Web Farm.

So, in the same way that MOSS automatically configures IIS sites, host names, authentication, and more, MOSS also has a centralized concept for modifying Web.Config settings.

Why Web.Configs?

A SharePoint site, as glamorous as it may make itself out to be, is nothing more than an ASP.NET web site running on top of IIS. Sure, there’s tons of functionality in it, but at the end of the day, it’s just an ASP.NET web site. With that in mind, it too will have a web.config file, where certain settings can be adjusted, custom application settings can be added, and so on. To give one example, assembly bindings can be specified (and even re-mapped to alternate versions of assemblies) through web.config file to make your application work.

The Problem

Let’s say you have 5 front-end web servers that serve up your SharePoint site. That means you have 5 copies of IIS, and 5 copies of the SharePoint web application. In turn, you have 5 web.config files to keep up with. If you were to manage this manually, you would have to make sure that whenever you made a change, you manually copy web.config files to each server.

Now, let’s say you have 10 SharePoint web applications running on each of those servers. You then have 10 web.configs per server, so a total of 50 web.configs to manage across your farm.

And finally, let’s say you add a 6th server to your farm. You would have to make sure your 10 web.configs were added to the 6th server, on top of now having 60 web.configs to manage and keep in sync.

Microsoft has solved this problem with the centralized concept of web.config changes.

General  Concept

The general idea of this concept revolves around the fact that a Web.Config is nothing more than XML, and that with XML XPath support, certain blocks of HTML can be found with accuracy and reliability, and then changed.

So, if there were just one place where you could store your “changes”, the rest could be automated and kept in sync with the other servers. And when a new server was provisioned, it could automatically be sync’d with this copy of the changeset. If we only had a SharePoint concept, called, oh, I don’t know…

SPWebConfigModification

The SPWebConfigModification class in SharePoint (as of WSS version 3.0) allows you to specify changes with the following information:

1. What “ownership” group should this settings change go under? (To group certain settings together for easy management.)
2. What is the XML Node path you want to start at? (for example, /configuration/system.web)
3. What is the XPath to the specific node you want to match?
4. What is the block of XML that you wish to go under the matched node?

These changes are stored in a centralized repository per-web application, and are automatically sync’d with all SharePoint web applications in the farm at regular intervals.

Below is a screenshot of what this “central repository” looks like when viewed through an internal tool I developed for this purpose (blurred to exclude possible sensitive info):

How Changes are Applied

Changes are applied one of two ways:

1. Manually (on-demand), through the SharePoint object model.
2. Automatically, at a regular interval.

When a web.config changeset synchronization occurs, the following things happen:

1. A timer job is created to synchronize web.config settings, specified to run on all farm servers.
2. Within 30 seconds, each server picks up its timer job to synchronize its own web.config.
3. Each server reads its own web.config, compares the XML nodes with the central modification repository entries, and ensures that all values match up. If a repository entry doesn’t exist in the web.config, it will be created, and if it does exist in the web.config with a different value, the web.config value will be updated.
4. Depending on what was changed, this can cause a recycle of the SharePoint web application.

WARNING: SharePoint, by default, schedules this to happen automatically at regular intervals (at least once a day). This is why it is important NOT to make “manual” changes to web.config files (i.e. by editing them in a text editor). If your manual changes happen to match a repository entry’s XPath, your manual change will be overwritten as soon as this synchronization occurs.

Additionally, if the XPath is only ‘halfway’ correct, it can cause multiple entries of the same node. For an example of this, please read the previous post MOSS Web.Config Modification – Beware the XPath.

Managing Web.Config Changes for Structured Deployments

Unfortunately, there is not a good way within SharePoint’s out of the box Solution Deployment to handle web.config change deployments. As far as I know, you will need to roll your own. However, the team working on this particular project came up with a pretty clever way of managing these changes, and pushing them out in a structured manner.

First, a XML file that is more or less referred to as a web.config Changeset File:

<?xml version="1.0" encoding="utf-8" ?>
<WebConfigChanges xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <!-- This change set is for unit test web applications -->
  <ChangeSet Name="UnitTesting" Status="Active" CanUpdate="True">
    <ChangeItem Name="trust" Path="configuration/system.web" Sequence="0" Environment="Development">
      <trust level="Full" originUrl="" />
    </ChangeItem>
  </ChangeSet>

  <!-- This change set is for development web applications -->
  <ChangeSet Name="Development" Status="Active" CanUpdate="True">
    <ChangeItem Name="compilation" Path="configuration/system.web" Sequence="2" Environment="Development">
      <compilation batch="false" debug="true">
        <assemblies>
          <add assembly="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </assemblies>
        <expressionBuilders>
          <remove expressionPrefix="Resources" />
          <add expressionPrefix="Resources" type="Microsoft.SharePoint.SPResourceExpressionBuilder, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
          <add expressionPrefix="SPHtmlEncodedResources" type="Microsoft.SharePoint.SPHtmlEncodedResourceExpressionBuilder, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
          <add expressionPrefix="SPSimpleFormattingEncodedResources" type="Microsoft.SharePoint.SPSimpleFormattingEncodedResourceExpressionBuilder, Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
          <add expressionPrefix="SPUrl" type="Microsoft.SharePoint.Publishing.WebControls.SPUrlExpressionBuilder, Microsoft.SharePoint.Publishing, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </expressionBuilders>
      </compilation>
    </ChangeItem>
  </ChangeSet>

  <!-- Used to configure error log page-->
  <ChangeSet Name="ErrorLogging" Status="Active" CanUpdate="True">
    <ChangeItem Name="SafeMode" Path="configuration/SharePoint" Sequence="0">
      <SafeMode MaxControls="300" CallStack="true" DirectFileDependencies="10" TotalFileDependencies="50" AllowPageLevelTrace="false">
        <PageParserPaths>
        </PageParserPaths>
      </SafeMode>
    </ChangeItem>
    <ChangeItem Name="customErrors" Path="configuration/system.web" Sequence="1">
      <customErrors mode="On" defaultRedirect="~/_layouts/error.aspx" />
    </ChangeItem>
  </ChangeSet>
</WebConfigChanges>

This file encapsulates what will go into a SPWebModification repository entry, and specifies a couple critical things:

1. The changeset name (translated into the “Owner” in SPWebConfigModification)
2. The environment that should be applied to each item (this provides a way to specify different settings for Development, Staging, Production, All, etc, for example, connection strings).
3. The XPath and Sequence of the modification entry, directly translated into the SPWebModification entry.

A straightforward deployment tool parses this XML, takes a Web Application as an argument, identifies the current environment (via a MOSS Farm-level Property Bag setting), and translates relevant entries into SPWebModification entries for the specified web application.

This provides a structured way to deploy and manage web.config changes, specific to the environment. Although you can still get in a great deal of trouble if XPaths are not absolutely perfect, this provides a great way to eliminate human error to deploy web.config changes in a similar fashion as normal MOSS .WSP package deployments.

Demystifying IIS Smooth Streaming / Live Smooth Streaming

UPDATE: We now offer a product that takes away the complexities of Smooth Streaming as a cloud-based platform, and gives your content instant access to the #1 CDN in the world… if you are considering implementing Smooth Streaming, consider Nimbus (www.nimbushd.com). Now on to the post…

Lately I’ve been trying to solve a few specific problems with Smooth Streaming, and I noticed as I go through forums, blogs, and Microsoft web sites, the only thing clear is that this topic is vague, and there are lots of unanswered questions out there. In addition, the way Microsoft uses the term “Live Smooth Streaming”, it almost seems intentionally misleading. So, with that in mind, I’d like to clear up a few things.

How does Smooth Streaming work? (the quick and straightforward version)
Smooth Streaming (or, as I will call it, On-Demand Smooth Streaming) relies on having the same stream encoded at varying bitrates, with one complete streaming file per bitrate. So you will typically have individual files that look like this in a web server directory:

• MyStream_230.ismv
• MyStream_305.ismv
• MyStream_403.ismv
• MyStream_534.ismv
• MyStream_708.ismv
• MyStream_937.ismv
• MyStream_1241.ismv
• MyStream_1644.ismv

These files all contain special timecode marks in them, and are all tied together by two “manifest” files, for example:

• MyStream.ism
• MyStream.ismc

These files simply contain a mapping of which bitrates correspond with each stream (for example, the 230k stream = MyStream_230.ismv, 708k stream = MyStream_708.ismv).

The Smooth Streaming player makes tiny HTTP requests requesting a second or two of video at a certain bitrate. For example, “give me 230k @ 00:01:06, give me 708k @ 00:01:08”, etc. These requests are no different than how your browser downloads images on a web page. The “smooth” part comes in because with this architecture, it’s extremely easy for your computer to switch bitrates and adjust to your network connection.

IIS Media Services
An IIS7 feature called IIS Media Services is what makes it possible for the server to understand these player requests, timecode, and a few special URL constructs.

An example of this is the Manifest request; if you had a smooth stream (with MyStream.ism) in the root folder of your IIS web server, you can open up your browser and go to:

http://server/MyStream.ism/Manifest

IIS Media Services will generate the XML manifest file that the player would consume, and you can view it in your browser.

What is “Live Smooth Streaming”?
Live Smooth Streaming uses the same technique of on-demand Smooth Streaming, except encoded in real-time for live events by hardware encoders. Although it is similar in player concept, it is quite different in implementation.

IIS Features: Smooth Streaming Presentations vs. Live Smooth Streaming Publishing Points
Once you install IIS Media Services, and navigate to a folder under an IIS web site, you will see these options:

• Live Smooth Streaming Publishing Points – Similar to Windows Media Services, you can create publishing points that live hardware encoders can use.
• Smooth Streaming Presentations – This is the feature that detects ISM and ISMC files (smooth streaming manifests) within a web site directory and interprets them as Smooth Streams.

How can you encode content for On-Demand Smooth Streaming?
Expression Encoder, with the IIS Live Smooth Streaming option.

Clarification Point: Expression Encoder is offered in 2 versions, a free version without the ability to do this, and a paid version with IIS Smooth Streaming encoding capabilities.

The IIS Smooth Streaming version will:

• Encode your content into separate bitrate files, and….
• Create the manifest files (.ISM, .ISMC) automatically

How can you encode content for Live Smooth Streaming?
Let me be clear, since this is the most confusing and vague part of Smooth Streaming to date: As of this writing, there is no software encoder that can do Live Smooth Streaming. You will have to be prepared to drop anywhere between $5,000 to $20,000 on a hardware IP video encoder (or a series of hardware encoders). For example, the Inlet Spinnaker Encoder.

The reason for this is the amount of sheer processing power that it requires to encode a smooth stream – it’s fairly intensive for a regular computer to encode one stream in realtime of acceptable quality; for a normal smooth streaming event, it’s eight realtime streams, ranging from low-quality to extremely high quality.

Again, Expression Encoder does NOT do Live Smooth Streaming. Expression Encoder only prepares content for On-Demand Smooth Streaming.

How do you play a Smooth Stream with Silverlight?
Just as you would play another media file in a Silverlight media player, but with a twist or two.

• Set the MediaSource property of your playlist item to the HTTP path of your .ISM file, appended with /Manifest – for example:

  http://server/path-to-mystream.ism/Manifest

• Make sure to also set the playlist item property IsAdaptiveStreaming to true. Without this, Silverlight won’t understand that it’s a smooth streaming source, and will be confused.
• If your Silverlight player is not located on the same host / relative path of your Smooth Stream, make sure that the root directory of your IIS site has a CrossDomain.xml file present, with contents that look like this:

<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
  <allow-http-request-headers-from domain="*" headers="*"/>
</cross-domain-policy>

This simply tells Silverlight that it’s okay to contact this domain for content.

• If you’re making your own Silverlight application with a player, Silverlight somehow needs to reference the AdaptiveStreaming.dll assembly. The easiest way to do this is to take the stock boilerplate player that Expression Encoder creates for you, and look for the SmoothStreaming.xap file in your output directory. Rename it to SmoothStreaming.zip (XAP files are really Zip files), and extract it. Inside are the assemblies you need to reference in your custom player (including AdaptiveStreaming.dll).

 

Hopefully this will clear up some understanding when working with Smooth Streaming, or at least making the total picture a little easier. Good luck!

Enterprise MOSS 2007: Event Receivers and threads don’t mix

MOSS provides an Event Receiver model you can use to perform actions when something happens (i.e. a list item is updated). This is somewhat described here:

http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.speventreceiverbase.aspx

It can be easy to end up in a loop here; for example, if you handle the ItemUpdated event, and you update the item again inside your event receiver, then your event receiver is going to fire again, causing you to update the item again, which will cause the event receiver to fire, causing you to … well, you get the picture.

SharePoint does have built-in protection for this problem, though – it will only allow a “depth” of 10 recursive events to fire.

Additionally, to solve this problem, the SharePoint object model provides two methods that can prevent this:

• DisableEventFiring()
  http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.speventreceiverbase.disableeventfiring.aspx
• EnableEventFiring()
  http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.speventreceiverbase.enableeventfiring.aspx

This all works great, but the solution I was working with on this problem required a somewhat “mass processing” of a bunch of list items. To do this, the event receiver spawns a few new threads for processing list items. We started getting some very weird errors – my favorite?

WebPartPageUserException: Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED))

In any case, what we actually found was that DisableEventFiring and EnableEventFiring, quite frankly don’t do jack when it comes to other threads. Why?

They set a property in the SPEventManager class:

Which, in turn, flags some data on the current thread!

On one hand, this makes sense, from the HTTP model perspective (you don’t want to disable events globally, because other updates coming from a browser wouldn’t fire the initial events while you were still processing yours), but at the same time, it’s far from obvious that asynchronous multithreading is not supported in the event model.

Lessons learned:

• The SharePoint event model is NOT friendly to multi-threading. If possible, offload asynchronous operations to a timer job or another method that will take place in a single thread / SharePoint request context.
• For multi-threaded operations, make sure the threads you spawn disable events before updating any items, to avoid an infinite loop.

Enterprise MOSS 2007 Performance Gotcha: ConsoleNode / Editing Toolbar Customization

There is a severe problem with the SharePoint 2007 ConsoleNode. In particular, you can run into an astonishing performance loss when attempting to build a dynamic menu that can change depending on page state. The Microsoft.SharePoint.Publishing.WebControls.ConsoleNode class is a base class that you inherit to customize the SharePoint Editing Toolbar.

In the above image, the “Page”, “Workflow”, and “Tools” options, for example, are ConsoleNodes.

The high level implementation goes like this:

1. Create a class – let’s say, MyConsoleNode – that inherits from Microsoft.SharePoint.Publishing.WebControls.ConsoleNode.
2. Override the GET property accessor for ChildConsoleNodes, shown below.

   

3. When SharePoint requests the ChildConsoleNodes property, you build a few child nodes that will be displayed in the dropdown menu, and add them to the base.ChildConsoleNodes collection.

This is the pattern that Microsoft uses for their Out-of-the-Box console nodes as well.

The Problem

This appears to be a fairly straightforward pattern, and works well, until you start doing any significant calculations to build your Child Node collection. The reason is that, in my testing, this property gets called 12+ times per page load. Hang on, because this journey is not over…

Most normal developers would solve this problem with a simple boolean flag, say in the current HTTP request, that indicates whether or not you have already built your child nodes, and if so, don’t do it again. Do NOT do this! You will most likely end up with a completely blank child menu, and start to question why you became a SharePoint developer.

This is the way that MyConsoleNode is used by SharePoint when it renders the page; my best guess is that it’s done this way to calculate permissions. For the purposes of my example, we’ll assume that 2 child console nodes are built.

1. HTTP request begins, and SharePoint starts to build the page.
2. A new instance of MyConsoleNode is instantiated (we’ll call this Instance A).
3. GET ChildConsoleNodes is called (Instance A), no existing child nodes.
4. GET ChildConsoleNodes is called (Instance A), 2 child nodes are already present.
5. A new instance of MyConsoleNode is instantiated (Instance B).
6. GET ChildConsoleNodes is called (Instance B), no existing child nodes.
7. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
8. GET ChildConsoleNodes is called (Instance B), 0 child nodes already present.
9. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
10. GET ChildConsoleNodes is called (Instance B), 3 child nodes already present [don’t ask – I don’t know].
11. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
12. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
13. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
14. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
15. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present.
16. Page is rendered.

It’s fairly obvious why a simple boolean flag will not work, for two reasons:

• The class is instantiated twice (once in Step 2, again in Step 5).
• Even when the class is not instantiated again, in Step 8, the previously added child nodes are gone.

There’s another mystery of why there are 3 child nodes in Step 10, but I didn’t really care to research that one.

The Solution

If you want to solve this problem, you will, unfortunately, have to use reflection to get an internal property value. I am not going to post the code to do this, because it is not guaranteed to survive a MOSS Service Pack upgrade or translate into future versions of SharePoint, and you will have to make some decisions in your code as to what to do if this property disappears at some point.

The key to solving this problem is a boolean internal property of ConsoleNode called ReadyToCreateChildNodes.

If you look at a Microsoft out-of-the-box Console Node (for example, let’s check out ModifyWebPartsNode) you’ll see a method called EnsureChildNodes():

In the code of this method, Microsoft internally checks for this condition:

As a side note, in my testing, I did also duplicate the ConsoleNode.IsValidStateForNode conditions, and they were always true; however, this depends on the permissions you’ve applied to the node, and I was using EmptyMask. But for the purposes of solving this problem, all you really need to care about is ReadyToCreateChildNodes.

Let’s look at the request again, now considering the state of this property.

1. HTTP request begins, and SharePoint starts to build the page.
2. A new instance of MyConsoleNode is instantiated (we’ll call this Instance A).
3. GET ChildConsoleNodes is called (Instance A), no existing child nodes. [ReadyToCreateChildNodes = FALSE]
4. GET ChildConsoleNodes is called (Instance A), 2 child nodes are already present. [ReadyToCreateChildNodes = FALSE]
5. A new instance of MyConsoleNode is instantiated (Instance B).
6. GET ChildConsoleNodes is called (Instance B), no existing child nodes. [ReadyToCreateChildNodes = FALSE]
7. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
8. GET ChildConsoleNodes is called (Instance B), 0 child nodes already present. [ReadyToCreateChildNodes = TRUE]
9. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
10. GET ChildConsoleNodes is called (Instance B), 3 child nodes already present. [ReadyToCreateChildNodes = FALSE]
11. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
12. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
13. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
14. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
15. GET ChildConsoleNodes is called (Instance B), 2 child nodes already present. [ReadyToCreateChildNodes = FALSE]
16. Page is rendered.

Occasionally, in my testing, ReadyToCreateChildNodes in step #9 would also be in a “true” state, but for the most part, it was only true for one call. And once I built my child nodes in Step 8, they remained populated for the rest of the request.

So, to sum this all up, the solution to this problem is: Use reflection to check the internal boolean ReadyToCreateChildNodes property before you build your child nodes.

Code Access Security Policy (CASPOL.exe) GUI Utility

I find it extraordinarily difficult (okay… inconvenient?) to make Code Access Security Policy changes in the .NET Framework. After .NET 2.0, outside of the SDK, the graphical tools for doing this were no longer installed on most client PC’s. Usually, when I have to break out caspol.exe, it’s possibly because:

1. I downloaded a third party DLL from a web site, and it’s not trusted, with no unblock option, and Visual Studio is complaining about it when I’m actually trying to get some real work done
2. I want to run some .NET apps off a network share, and the machine I’m using doesn’t have .NET 3.5 SP1.

… and it’s used so infrequently that I have to look up all the command line options and re-learn the thing every time, not to mention when I finally get the command right, I’m not in Elevated Privileges mode (Vista / Win 7).

So, for those like me, who like GUIs, and just want to get an assembly or folder trusted and move on with life, I present to you – a GUI CASPOL command generator:

This supports a subset of the available CASPOL commands:

• Single file / assembly
• Folder path / network share
• Different scopes (Machine / User / Enterprise)
• Trust Level (FullTrust by default)
• Code Groups
• Run CASPOL command in Elevated mode

For the small executable – Download It Here (9K)

… or, here’s the source project (VS2008 / C#).

Exchange 2003: Deleting Mailboxes doesn’t free up disk space… why?

So, your Exchange 2003 server is running out of disk space, and you notice that no matter how many mailboxes you delete and purge, no matter if it’s backed up or not, no matter how short you set your deleted items retention period, you aren’t getting any disk space back… why?

To actually free up disk space, you will need to run an Exchange Database Utility called eseutil to defragment the Exchange datastore. It is located in the Exchange \bin folder, usually C:\Program Files\Exchsrvr\bin.

Also, look in your MDBDATA folder for your .edb and .stm files – you’re going to need their names. Most likely if you haven’t changed anything since installing Exchange, they are named priv1.edb and priv1.stm.

Now, before you go off to command line land, hang on a second – most likely if you’re doing this for the purpose of freeing up disk space, you don’t have enough disk space left to defragment the drive. You’ll need something, whether it’s another drive with available space or a USB stick.

Step 1: Stop the Exchange Information Store service.

Step 2: Run defragment command, and prepare to wait for a while. This is not something you do on the way out to lunch in the middle of a working day. For a 15GB database, this can take 1-2 hours.

Here’s the command:

eseutil /d “C:\Program Files\Exchsrvr\MDBDATA\priv1.edb” /f “E:\TempStreamFile.stm” /t “E:\TempDBFile.edb”

Replace the red path to priv1.edb with your actual path to your Exchange database.

The green .stm file is the temporary Exchange Streaming File to use during defragmentation, and the blue .edb file is the temporary database to use during defragmentation. Change the paths to point to your temporary disk with enough space to hold at least the size of your original EDB.

Happy defragmenting…

Microsoft Office 2007 XML Patent Infringement Update – No reason to panic

As of January 1, 2010, Microsoft had to stop selling copies of Microsoft Office 2007 (in particular, Word) that uses a certain XML feature. That feature will no longer be supported, and I got a couple questions this morning on the impact of this change. Some X9 systems generate reports in Office 2007 format, and I imagine lots of other systems around the world do, too, so obviously it’s important to pay attention and check.

The feature in question is actually Custom XML Markup, and this is not a huge change in Office functionality. Unless your company uses this specific feature to integrate Word documents into other business systems, you’ll probably never notice any difference. So, let’s break this down:

Geek speak, if you please (Microsoft KB 978951): http://support.microsoft.com/default.aspx/kb/978951

This is what the feature looks like (pink representations of XML):

What versions of Office are affected?
Only Office 2007.

Once this update is applied to my system, can I still open documents created by someone else, or documents I created a while ago?
Yes. This update will not affect your ability to open or use previously created documents.

Will the .DOCX format remain the same?
Yes, according to Microsoft’s KB, the Open XML format (what you know as the .docx, .xlsx, .pptx, etc file extensions) is not affected.