In matlab it is possible to write matlab objects, or even the entire workspace, to a file using the matlab save() call. I would like to intercept the bytestream and postproc
Couldn't you encrypt the content of variables instead ?
With whos, you get a list of all your variables in alphabetic order. For each one, you generate a mask of the same size with your encryption algorithm and you replace the "true" value by itself XOR the mask. To finish, you save the encrypted variables using save. The name and size of your variables are visible but that's probably not critical (if necessary, you can crypt names too).
Proceed the same way to load.