Troubleshooting an x64 com interop marshaling issue

Question

I have a C++ COM Server which I have recently recompiled to 64 bit. This COM server has a method that contains a struct parameter which contains some ints and a BSTR and another structure. Now, I am trying to call this COM server from a 64 bit .Net C# application. I can successfully load my COM server and call this method as long as I don't try to populate any of the string parameters. If I try to pass valid values in the int members, they end up corrupted by the time the end up at the COM object implementation. It appears as though the way the structure is marshaled is just wrong. This code worked just fine in 32 bit applications.

The following is the general way the idl is defined on the C++ side: (ignore the goofy typedefs, it's some legacy code)

[helpstring("method Method1")] HRESULT Method1([in] STRUCT1* pStruct, [in, out] DWORD* inparm1, [out]USHORT* outparm2);


typedef struct _Struct2
{
    USHORT  p1;
    BSTR  s1;
    BSTR  s2;
    BSTR  s3;
    BSTR  s4;
    DWORD       p2;
    DWORD       p3;
} STRUCT2;


typedef struct _Struct1
{
    DWORD p1;
    DWORD p2;
    BSTR s1;
    BOOL p3;
    STRUCT2 struct2;
}STRUCT1;

Attempting to populate the members in STRUCT2 causes undefined behavior and crashes. Can anyone see why this would be a problem in 64 bit verses 32 bit code? Is there some marshaling magic I need to make happen? Additionally, I don't seem to have the tools to troubleshoot marshaling issues. Any suggestions on a good way to troubleshoot what the marshaler is doing under the covers?

Answer 1

A struct is COM's Achilles heel. The actual layout of the members of the struct is highly compiler dependent. They were not support in COM automation for quite a while until they came up with the IRecordInfo hack. Not getting used here.

In unmanaged code, the #pragma pack directive is very important. For type libraries, the /pack argument to midl.exe is essential. If it isn't 8 or you don't use the 64-bit version of midl then you'll definitely have this kind of problem. The BSTR members are the ones that throw it off, they are either 32-bit or 64-bit pointers, depending on the bitness. And align on a multiple of 4 for the 32-bit version of midl, a multiple of 8 for the 64-bit version. You can possible rescue it by passing /pack 4, as long as the struct doesn't contain any doubles. But try the 64-bit version of midl.exe first. Or get rid of structs and replace them with interface pointers, that's the real fix.