Wednesday, October 27, 2010

Hotpatching: Great idea Microsoft, but a terrible implementation

While doing research on minimizing planned downtime within a datacenter, I re-examined the idea of hotpatching. Microsoft introduced this idea with Windows 2003 Service Pack 1 as part of a “reboot reduction initiative.”

Microsoft reboot reduction initiative

Hotpatching is part of the Microsoft reboot reduction initiative, which seeks to help minimize the need for a full system reboot after installing updates. Reducing reboots is important because IT departments in many organizations implement a time-consuming test cycle every time an update is installed and the system is rebooted. This results in loss of productivity and revenue to the organization until their system is fully verified and operational. Hotpatching allows customers to deploy important updates and patches in a timely, transparent manner without requiring a full system shutdown and restart. This reduces their rollout time.

The following examples demonstrate possible savings from reboot reduction:

  • Of the 22 updates that shipped for Windows Server 2003 RTM between April 2005 and August 2005, 15 of them required a reboot. Eight of these could have been hotpatched. This would have reduced the number of reboots by 53%.
  • Of the 14 updates that shipped for Windows Server 2003 Service Pack 1 (SP1) prior to August 2005, ten of them required a reboot. Four of these could have been hotpatched. This would have reduced the number of reboots by 40%.

Source: http://technet.microsoft.com/en-us/library/cc781109(WS.10).aspx

So how did it work out?

I did some research by using Google to find security bulletins that contained the word hotpatching and then refined those results. I found only a few bulletins that supported the switch. Here is an example of one search:

hotpatching -does-not-support-HotPatching site:http://www.microsoft.com/technet/security/bulletin

I would estimate less than 1% of patches released support hotpatching—which leaves me pretty disappointed given Microsoft’s example.

The Linux equivalent, ksplice seems to have a much better track record—though I have no experience with its impact on system stability.

Update:

In an effort to find additional statistics showing that hotpatching was basically unused by Microsoft, I extracted one of the few hotfixes I could find that supported hotpatching (WindowsServer2003-KB917159-x86-ENU) using the /x switch. According to Microsoft’s documentation, a hotfix that supports hotpatching will contain a file with a “.hp” extension. Within the directory structure I did indeed find the hotpatching file “svr.hp.sys”.

I then extracted and searched through a collection of 200 post Windows 2003 service pack 2 hotfix files for other filenames containing the ‘hp’ string.

Number found: zero.

No comments:

Post a Comment