DEP Data-Execution-Prevention Windows 2003 Server "Joy"

I just had the "fun" of dealing with a difficult to track down (the underlying cause of) COM Surrogate error in Windows 2003 Server Web Edition with SP2 on it. The error was showing up constantly in the event logs on reboot, and a web-application that used COM+ components was working and failing intermittently because of something going wrong in the COM system.

Well, after a lot of hunting through event logs and examining system-settings and such, I found the issue was due to Windows DEP (Data Execution Prevention) junk that was introduced into Windows Server 2003 with Service Pack 1. More than coincidentally, it was at the time of the SP1 install that the web-server program / software in question (using COM Components) began to exhibit strange symptoms and seemingly random failures.

It seems that Windows DEP interpreted data being sent to/from my COM+ Component (Remote Procedure Call to another server where data was being pulled from a SQL-Server 2005 database) as a potential threat - i.e., as code embedded in the data stream that may be an attempt to execute malicious logic. The DEP function would only fail PART of the time, when a certain stream of data (which was actually just large PDF reports) would "look like" potentially malicious code embedded in a a data stream... at least, that is my theory.

DEP was introduced back with Service Pack 1 for Windows 2003 Server, which I installed on the web-server in question back in July of 2005, which shortly after is when I started seeing random COM Surrogate errors (I checked my logs / notes). I found this on the Microsoft support site (http://support.microsoft.com/kb/899298/en-us) Rather interesting bulletin from October, 2006, after SP1 was already installed on the webserver / web-site in question that was experience the COM Surrogate Error under Windows 2003 Server.

SYMPTOMS

The "Understanding Data Execution Prevention" help topic in Microsoft Windows Server 2003 with Service Pack 1 (SP1) contains the following incorrect entry:

By default, DEP is only turned on for essential Windows operating system programs and services. To help protect more programs with DEP, select Turn on DEP for all programs and services except those I select.

By default, in Windows Server 2003 SP1, DEP is turned on for all programs and services except those that the administrator selects. By default, the "Turn on DEP for all programs and services except those I select" OptOut policy is already selected.

Gee, isn't that special!

I never knew Microsoft officially announced that the original help file was BACKWARDS from reality. This whole time, the fix (Which I presume it is a FIX now), may have been just a "checkbox away"! Arghhhhhhhh.

So, here's my updated note to myself about what symptoms to look out for with COM Surrogate Errors under Windows 2003 Server (with SP1 or SP2 or SP3, etc)...

COM Surrogate Error Fix

IMPORTANTNOTE: DATA-EXECUTION-PREVENTION (DEP) SETTING in Windows Server 2003 SP1+ may need altered to allow the COM SURROGATE to run or function!!!

Where: Control Panel, System, Advanced, Performance (Setting button), DEP -- there should be a radio-button (2nd one from top) that is clicked and says "Turn on DEP for all programs and services except those I select", with a check-listbox below that shows "COM Surrogate" --> that item needs checked, to ALLOW custom COM object(s) to run on WebServer!!

I don't know if this will help anyone else, but debugging a COM Surrogate error isn't fun, and this is something to look into!