I've been giving thoughts on Custom Actions lately and Nates feedback has prompted me to start publishing them. I'll probably keep updating this entry as my thoughts change.
First of all, I'm sure we all try to avoid custom actions in the first place! It is often best to handle a problem using native MSI standard actions for several reasons which I won't name here. But we all probably also know by now sometimes CA's are needed. So what is my list of requirements when choosing a solution for a CA?
1) Capable: The CA must be able to get the job done. It needs to be able to call Win32 API and COM. Possibly it needs to be able to call .NET services also, although at this point I havn't needed to. Some on the web have argued that alot of the .NET services are just wrappers of existing COM and Win32 interfaces, but I really don't know yet. It also must be able to access the MSI handle for doing things like getting/setting properties and posting messages back to MSI logs and subscribers.
2) Reliable: The implementation needs to have as few points of failure as possible. The less moving parts and dependencies the better.
3) Debugable: When things aren't working you MUST be able to step through line by line and get meaningful data to aid in troubleshooting. You'll pull your hair out if you can't.
4) Ease of Development: This is a subjective requirement. The language used by the CA should be easy to program in. We shouldn't be loads of time developing CAs. This is one reason I love InstallScript over C++. Unfortunatly InstallScript is failing requirement #2 and hence C++ is beggining to trump InstallScript in my mind.
5) Supportable: You won't always be working at Acme Software House. What languages are well understood by your fellow developers who will be stuck with your project when you leave?
6) Compiled: What goes in a CA, stays in a CA! This is important in terms of protecting source. It also aids in requirement #2 since a truely compiled language doesn't require a runtime intepreter.
7) Integrated: Ideally the CA's source would be integrated into the MSI development IDE. This is one thing I really like about InstallScript. It's very easy to author, compile, build and debug the CA from one IDE. Unfortunatly refer to rule #2.
Obviously runtime requirements (gets the job done reliably ) are more important then design time requirements ( easy for the developer) . So where does this leave me? Seriously considering dumping InstallScript and going to a language that can compile to a Native Win32 DLL ( not COM ). Unfortunatly where I work there aren't many C++ programmers and a whole lot of VB programmers. So I'm also considering using the DLL as a wrapper to register and invoke a VB COM object where the real work gets done. But this creates more moving parts and starts to violate rule #2. There are some programs that claim to be able to create true DLL's out of VB6 projects. This also looks very tempting to me. VBAdvance, VBExport and VisualDLL seem promising, but in truth all of this is a hack to get around doing C++ CA's.