C++ COM Object Hotpatching?

Question

Background:

I'm hooking on windows COM object.

The method used is vtable modification. say we have an instance of interface A named instance, it contains oldmethod in the interface, I replaced with newmethod. However, in my newmethod I need to know the address of oldmethod so that I can call oldmethod after doing my own thing.

It is not safe to store the address of oldmethod in a global variable, since there might be more than one implementation behind interface A, let's say there are two implementations, class A1 and class A2. Thus my newmethod needs to store both A1->oldmethod and A2->oldmethod, and call appropriate function based on the instance type.

• One way to accomplish this is that I keep a map, which stores the (address of vtable -> oldmethod). Since the address of vtable can act as a distinguisher of class A1 and class A2. In my newmethod, the map is checked for the correct oldmethod for current instance. However, this will make the program check the map every time, which imposes a cost, and thread safety on the map will increase the cost.

• Another way is to make a closure, I allocate a chunk of executable memory, and write the binary code of my newmethod inside(which can be reduced to the minimum size, so size is not a problem). I modify the address of oldmethod in the binary code for each instance. In this case there is no searching on the map cost.

Question 1:

Is the second way a safe way to do this, or is the first way better? Is there any potential safety problems in either of them?

Question 2:

In the second way, the closure I created contains class specific data, which is the oldmethod pointer. If I need to store instance specific data in my newmethod is there any strategy other than keeping a (this pointer -> data) map? I tried my best and couldn't find a way.

Answer 1

You may not have the source to class A1, but do you control when it gets instantiated (either by "new", CoCreateInstance, or some other factory function)? If so, then just implement a class that implements interface A and just forwards all the calls on interface A to the real object and intercept the method(s) you care about.

In the example below, we show an example of replacing

class InterfaceA : public IUnknown
{
public:

    virtual int M1() = 0;
    virtual int M2(int x, int y) = 0;
    virtual int M3() = 0;
};


class CMyWrapperClass : public InterfaceA
{
public:

    int _refcount;
    InterfaceA* _pInner;

    CSomeClass2(InterfaceA* pInner)
    {
        _pInner = pInner;
        _pInner->AddRef();
        _refcount = 1;
    }

    ~CSomeClass2()
    {
        _pInner->Release();
    }

    virtual int M1() {return _pInner->M1();}
    virtual int M2(int x, int y)  {printf("CSomeClass2::M2(x=%d, y=%d)\n", x, y); return _pInner->M2(x,y);  }
    virtual int M3() {return _pInner->M3();}

    // not shown - addRef, release, queryinterface
};


   // example instantiation
   hr = CoCreateInstance(CLSID_A1, NULL, CLXCTX_ALL, IID_InterfaceA, (void**)&pInterfaceA);

   // now do the wrap
   pInterfaceA = new CMyWrapperClass(pInterfaceA);

If you don't have control of the instantiation of the class you are trying to hotpatch, I do have code to share for that. But it's obivously a bit more complicated. If this doesn't work, I'll post another answer directly related to hotpatching a COM vtable.

Answer 2

It took me quite a while to understand this problem. I actually wrote out a good chunk of code that I thought to demonstrate how to patch a vtable and then invoke the original method in a wrapper class. Patching the vtable is easy.

And then I discovered the problem you are referring to. And that is when the patched vtable method is called (newmethod), even if it defined within another class, "this" is the original object and you don't have any context of which instance is being invoked. So you can't easily just reference a member variable to get back to the "oldmethod" you have saved off.

So after some thought, I think the global map is the safest approach for doing this. You likely only need a lock (critical_section) to guard inserting, removing, or looking up the function pointers in the map. You likely won't need to hold the lock while invoking the old method after you have safely retrieved it from the map. As such, the runtime overhead of doing this operation is very negligible.