Intercepting syscalls in Android kernel — device reboots when module is removed

Question

I have been trying to intercept the read syscall in Android kernel (3.0.72 for maguro). I am using kernel module for such purpose. An example is as follows:

#include <linux/module.h>
#include <linux/unistd.h>    

MODULE_LICENSE ("Dual BSD/GPL");

asmlinkage long
  (*orig_call_open) (const char __user * filename, int flags, int mode);

asmlinkage long
  (*orig_call_read) (unsigned int fd, char __user * buf, size_t count);

#define SYS_CALL_TABLE_ADDR 0xc0058828

void **sys_call_table;

asmlinkage long
new_sys_open (const char __user * filename, int flags, int mode)
{
  printk ("Calling my open\n");
  return orig_call_open (filename, flags, mode);
}

asmlinkage long
new_sys_read (unsigned int fd, char __user * buf, size_t count)
{
  printk ("Calling my read\n");
  return orig_call_read (fd, buf, count);
}

/* Module initialization and cleanup functions */

int
init_module ()
{
  sys_call_table = (void *) SYS_CALL_TABLE_ADDR;

  // save original function ptrs
  orig_call_open = (void *) sys_call_table[__NR_open];
  orig_call_read = (void*) sys_call_table[__NR_read];

  // replace existing functions with ours
  sys_call_table[__NR_open] = (unsigned long *) new_sys_open;
  sys_call_table[__NR_read] = (unsigned long *) new_sys_read;

  printk ("Initializing.\n");

  return 0;
}

void
cleanup_module ()
{
  sys_call_table[__NR_open] = (unsigned long *) orig_call_open;
  sys_call_table[__NR_read] = (unsigned long *) orig_call_read;
  printk ("Cleaning up.\n");

}

I can normally insert the module by using insmod. However, when I try to remove it (with rmmod), the kernel breaks and the device reboots.

[...]
[   80.512054] Unable to handle kernel paging request at virtual address bf000040
[   80.512145] pgd = c6d98000
[   80.512237] [bf000040] *pgd=85edc811, *pte=00000000, *ppte=00000000
[   80.512634] Internal error: Oops: 80000007 [#1] PREEMPT SMP
[   80.512725] Modules linked in: [last unloaded: privacy_capsules]
[   80.513061] CPU: 0    Not tainted  (3.0.72-gfb3c9ac-dirty #4)
[   80.513214] PC is at 0xbf000040
[   80.513336] LR is at sys_read+0x6c/0x78
[   80.513427] pc : [<bf000040>]    lr : [<c01533ec>]    psr: 200f0013
[...]

I also tested with other syscalls (for example, only for sys_open and sys_write -- and no sys_read) and it works ok (insmod and rmmod). However, the problem seems to happen only with sys_read.

Any idea? many thanks in advance!

EDIT:

These are the addresses for the function pointers:

orig_call_read : 0xc0153380
new_sys_read   : 0xbf000000

And I took a piece of the assembly code generated from the module code:

00000000 <new_sys_read>:
   0:   e1a0c00d    mov ip, sp
   4:   e92dd878    push    {r3, r4, r5, r6, fp, ip, lr, pc}
   8:   e24cb004    sub fp, ip, #4
   c:   e1a04000    mov r4, r0
  10:   e3000000    movw    r0, #0
  14:   e3400000    movt    r0, #0
  18:   e1a06001    mov r6, r1
  1c:   e1a05002    mov r5, r2
  20:   ebfffffe    bl  0 <printk>
  24:   e3003000    movw    r3, #0
  28:   e3403000    movt    r3, #0
  2c:   e1a00004    mov r0, r4
  30:   e1a01006    mov r1, r6
  34:   e5933000    ldr r3, [r3]
  38:   e1a02005    mov r2, r5
  3c:   e12fff33    blx r3
  40:   e89da878    ldm sp, {r3, r4, r5, r6, fp, sp, pc}

So, as @ChrisStratton suggested, a blocked read() returns (in this case, after the blx r3), but cannot find the next address (0xbf000040).

Answer 1

This does indeed look like a blocking or otherwise delayed read call trying to return through your code after it is unloaded.

I think you swapped the addresses for the two functions in your report of their address.

The failure happens when the original tries to return to the last line of your replacement at [<bf000040>] but this is no longer in memory since your module has been unloaded. I'd expect a system such as this could easily have a lot of read calls that block for long periods of time.

Instead of unloading your module, you might need to implement an interface in sysfs or similar which you can use to disable new redirections while leaving it in memory.

Another option would be to see if it is workable to "jump" rather than "call" to the original, so that the original's return skips you and goes straight back to your caller. In ARM speak that would be a branch without link. Looking at the code, it seems like you would have to clean up your local stack first, restoring the registers to their state at the start of your code, and then make the jump.