Why doesn't this keyboard intercepting kernel extension work?

Question

my fellow developers! I hope very much that at least some of you will not get frightened by the amount of text this question contains (I simply did my best to be as descriptive as humanely possible). :)

To those who think I've asked this question to write malware or something. I want to write an application that will allow users to select applications to be launched after the OS will finish launching. The whole idea is to allow user to select these apps BEFORE the OS finished launching by pressing hotkeys previously binded to the apps. For example user turns on his Mac, types SMTV and goes away, when the system finishes launching my app recovers the input and launches Safari, Mail, Tweetie and Vuze. I'm new to SO but I do my best to help others by answering their questions - I think I can expect the same in return. Check my profile and my activity and after that start screaming about malware.

This question is a follow-up to the question Is it possible to recover keyboard input that was done while Mac OS was starting up?.

Guided by Pekka's advice, I've stumbled upon an article Intercepting Keyboard Events by Christian Starkjohann that describes how he and the Objective Development team succeeded in reassigning iBook's CDROM eject key from F12 to Shift+F12. The main part is that they actually intercepted keyboard events, which is what I need. In the end Christian has written this article exactly for developers like me to use the idea of iJect as a prototype for similar functionality.

To start with, I decided to create a simple kernel extension to simply log the user's keyboard input to /var/log/kernel.log. I've started a new Generic Kernel Extension project in XCode, followed the instructions of the Hello Kernel: Creating a Kernel Extension With Xcode tutorial found in Mac Dev Center's Kernel Extension Concepts to create a Hello World project and then stuffed it with code taken from iJect sources. Here are the results:

TestKEXT.c

#include <sys/systm.h>
#include <mach/mach_types.h>


extern int HidHackLoad(void);
extern int HidHackUnload(void);


kern_return_t MacOSSCKEXT_start (kmod_info_t * ki, void * d) {
    return HidHackLoad() == 0 ? KERN_SUCCESS : KERN_FAILURE;
}


kern_return_t MacOSSCKEXT_stop (kmod_info_t * ki, void * d) {
    return HidHackUnload() == 0 ? KERN_SUCCESS : KERN_FAILURE;
}

HIDHack.h

#ifdef __cplusplus
extern "C" {
#endif

#include <mach/mach_types.h>
#include <sys/systm.h>

 extern int HidHackLoad(void);
 extern int HidHackUnload(void);

#ifdef __cplusplus
}
#endif

#include <IOKit/system.h>
#include <IOKit/assert.h>
#include <IOKit/hidsystem/IOHIDSystem.h>


class HIDHack : public IOHIDSystem {
public:
 virtual void keyboardEvent(unsigned   eventType,
          /* flags */            unsigned   flags,
          /* keyCode */          unsigned   key,
          /* charCode */         unsigned   charCode,
          /* charSet */          unsigned   charSet,
          /* originalCharCode */ unsigned   origCharCode,
          /* originalCharSet */  unsigned   origCharSet,
          /* keyboardType */     unsigned   keyboardType,
          /* repeat */           bool       repeat,
          /* atTime */           AbsoluteTime ts);

 virtual void keyboardSpecialEvent(unsigned   eventType,
           /* flags */        unsigned   flags,
           /* keyCode  */     unsigned   key,
           /* specialty */    unsigned   flavor,
           /* guid */         UInt64     guid,
           /* repeat */       bool       repeat,
           /* atTime */       AbsoluteTime ts);
};

HIDHack.cpp

#include "HIDHack.h"


static void *oldVtable = NULL;
static void *myVtable = NULL;


int HidHackLoad(void) {
 IOHIDSystem *p;
 HIDHack *sub;

 if (oldVtable != NULL) {
  printf("###0 KEXT is already loaded\n");
  return 1;
 }
 if (myVtable == NULL) {
  sub = new HIDHack();
  myVtable = *(void **)sub;
  sub->free();
 }
    p = IOHIDSystem::instance();
    oldVtable = *(void **)p;
    *(void **)p = myVtable;

 printf("###1 KEXT has been successfully loaded\n");

    return 0;
}

int HidHackUnload(void) {
 IOHIDSystem *p;

    if (oldVtable != NULL) {
        p = IOHIDSystem::instance();
  if (*(void **)p != myVtable) {
   printf("###2 KEXT is not loaded\n");

   return 1;
  }
        *(void **)p = oldVtable;
        oldVtable = NULL;
    }

 printf("###3 KEXT has been successfully unloaded\n");

 return 0;
}

void HIDHack::keyboardEvent(unsigned   eventType, unsigned   flags, unsigned   key, unsigned   charCode, unsigned   charSet, unsigned   origCharCode, unsigned   origCharSet, unsigned   keyboardType, bool repeat,
       AbsoluteTime ts) {
 printf("###4 hid event type %d flags 0x%x key %d kbdType %d\n", eventType, flags, key, keyboardType);

    IOHIDSystem::keyboardEvent(eventType, flags, key, charCode, charSet, origCharCode, origCharSet, keyboardType, repeat, ts);
}

void HIDHack::keyboardSpecialEvent(   unsigned   eventType,
          /* flags */        unsigned   flags,
          /* keyCode  */     unsigned   key,
          /* specialty */    unsigned   flavor,
          /* guid */         UInt64     guid,
          /* repeat */       bool       repeat,
          /* atTime */       AbsoluteTime ts) {
 printf("###5 special event type %d flags 0x%x key %d flavor %d\n", eventType, flags, key, flavor);

 IOHIDSystem::keyboardSpecialEvent(eventType, flags, key, flavor, guid, repeat, ts);
}

The resulting kernel extension gets successfully loaded/unloaded by kextload/kextunload programs, but doesn't actually intercept any of the keyboard events. I've tried doing lots of things to get it working, but without any errors or other problems with it in the way I can't google anything useful and ask your help.

Answer 1

The problem is not with how you are overriding the existing IOHIDSystem instance. That works just fine.

The problem is that when IOHIKeyboard is opened, it is passed a callback function to the IOHIDSystem for processing events. The callback is a static private function of IOHIDSystem, called _keyboardEvent:

    success = ((IOHIKeyboard*)source)->open(this, kIOServiceSeize,0,
                (KeyboardEventCallback)        _keyboardEvent, 
                (KeyboardSpecialEventCallback) _keyboardSpecialEvent,
                (UpdateEventFlagsCallback)     _updateEventFlags);

The callback then calls keyboardEvent function in the IOHIDSystem instance:

    self->keyboardEvent(eventType, flags, key, charCode, charSet,
                            origCharCode, origCharSet, keyboardType, repeat, ts, sender);

It does not call the ten parameter one, which is virtual (and which you are overriding). Instead, what is being called is the 11 parameter non-virtual one. So even if you tried to override the 11 parameter one, it would not work as the call never goes through the vtable.