August 09, 2007

InstallShield 2008 HOTFIX: COM Extraction Causes System Corruption during Uninstall

About a month ago I was working on an Installer for a North-Texas company when I did something I would normally never, ever do.... install/uninstall an untested prototype install on my dev box. The results were horrific... my entire network stack was trashed, all kinds of services weren't running and I actually got a message from Windows Vista saying something to the effect that a certain error that should never occur, occurred.

I looked at my package to see what could have happened and I noticed some TCP/IP settings had been picked up as part of a COM Extraction. I grabbed another laptop, transcribed the data and rebooted and all was well. Whew... bad setup developer!

I didn't think much more about this because at my day job we have almost no COM in our solution. The Server Installs ( Web Services and WebUI ) are pure .NET and the client only has some third party controls that we interop with. But all of those COM servers were authored in InstallShield 12 with the Right-Click Extract COM pattern. They never had a chance to be butchered by IS2008.

Then I came across a KB article today that described the exact situation that made me lose a few heartbeats.

I'll let you read the article, but basically it described a situation where the COM Extraction didn't filter well enough and that there is Hotfix available. I'd pick this one up for sure....

6 comments:

Erik Garcia said...

That is a particularly NASTY bug. Thanks for mentioning it!

Markus S. said...

We're currently trying to deploy an application which contains a COM server with IS 2008. We use the 'Extract COM Data for Key File' method and we've applied the hotfix. Whenever we execute the setup, TCP/IP is corrupted in terms of being not able to perform a simple "ping localhost". Although pinging other sites is possible. We stripped the setup down to one simple COM component and the problem persists. Can you confirm that behaviour?

markus s. said...

We're currently trying to deploy an application which contains a COM server with IS 2008. We use the 'Extract COM Data for Key File' method and we've applied the hotfix. Whenever we execute the setup, TCP/IP is corrupted in terms of being not able to perform a simple "ping localhost". Although pinging other sites is possible. We stripped the setup down to one simple COM component and the problem persists. Can you confirm that behaviour?

Christopher Painter said...

When my was corrupted, you didn't have a TCP stack at all so you could resolve or ping anything.

Did you author this package or is it third party and you are just deploying it? I'd have to see the package in question to confirm the problem.

markus s. said...

The striped down package contains only our product. It would be nice if you could have an eye on it.

Maybe I could send you an email.

markus s. said...

We finally found the cause of our problem, which is indeed a bug in InstallShield:
Our COM server implements an Asynchronous Pluggable Protocol which is used for internal communication and registered ony for the server process. During the 'Extract COM Data for Key File' process, IS detects that protocol, extracts its keys and tries to install them on a destination computer. This leads to a corrupted TCP/IP protocol.
The workaround is to remove those protocol keys from the registry tables in IS.
Maybe you'd like to inform your readers about this issue.