DLL Hell with a VB6 client

See: DLL Hell (wikipedia)

Overview

The app I admin is written in VB6 which uses a ton of ActiveX controls and COM+ DLL libraries. These libraries and controls are registered into the Windows registry which generates a GUID in HKEY_CLASSES_ROOT\CLSID along with entries of the filename and path, Company, etc.

To register the libraries/controls, you use regsvr32 with the file path. For my app’s case, it’s best to use the SysWow64 version of regsvr32 (which means you must run it from that directory) since most of our stuff is 32bit. Good news is that GUID will stay the same as long as Binary Compatibility is enabled for the VB6 project.

Problem

If binary compatibility is switched off while compiling, or it’s broken at some point, regsvr32 will generate a new GUID as to not replace the original registered component since it’s trying to retain the compatibility for old projects that would require the old ocx/dll. This is usually a good thing since projects compiled with the old OCX/DLL will continue using it and you could have a newer version side by side. The bad part for us is with all the different builds going on for development or on the Bamboo build server, we get a ton of junk registered into the registry. If one of those builds failed, which happens a lot, it could potentially leave partial entries in the registry. Those partial entries will be for broken dll’s or ocx’s and when compiling new versions of the application, it may (and in my case, did) reference the junk entries, rather than any of the good entries.

The Fix

One way to fix this is to browse over EVERY GUID in HKEY_CLASSES_ROOT\CLSID and try and find all of the DLL’s or OCX’s. This approach would take weeks worth of time, and obviously not feasible.

The better approach is a 3rd party tool called ActiveX Helper (axhelper).

This tool that basically parses the registry entries for all the registered OCX’s and other dll’s under HKEY_CLASSES_ROOT\CLSID . After that, the app lists everything registered on the machine in a datagrid. This allows you to sort by their file path and company.

To fix any number of issues, such as the client randomly freezing, or projects not compiling, perform the following operations.

  1. Open AxHelper.exe AS ADMIN. This must be opened as admin, otherwise it wont let you remove the entries.
  2. Sort by Company in the datagrid, and find all of the items for your target company: axhelper1.png
  3. Highlight all of the entries for your target organization. Note: you may want to confirm filepaths or Product names to make sure you don’t remove something you don’t intend to. Good news is this usually doesn’t matter because we’ll reregister everything to make sure we have the right ones installed.
  4. Once all the affected entries are highlighted, go to File > Unregistered DLL/OCX Files axhelper2.png
  5. Confirm you want to remove these entries by clicking YES axhelper3.png
  6. ActiveXHelper will refresh and load all the entries in your Windows registry again. Scroll back down to your target organization and confirm the removal worked.
    • If it worked, there shouldn’t be any more entries for your target organization!
  7. If it didn’t work FIRST check and make sure you’re running AxHelper as admin, or you may have a few stragglers left behind, so…
    • right-click on the affected row(s) and choose Open In RegEdit. NOTE: In my case, I needed to have RegEdit already opened in the background, but once that was done it worked. axhelper4.png
  8. This will bring you to the CLSID entry in RegEdit which you can right-click and choose delete: axhelper5.png
  9. After you’ve deleted the affected row(s), refresh ActiveXHelper and confirm that the manual deletion was successful.
  10. Next, re-register all of your affected ocxs/dlls/etc using regsvr32 and you should have all the correct entries in your Windows registery.

Note, if you accidentally remove some extra OCX’s, I had to re-register some of these Microsoft controls:

  • sssplt30.ocx
  • msflxgrd.ocx
  • sqaote32.ocx
  • sstbars.ocx
Written on February 17, 2021