In matlab it is possible to write matlab objects, or even the entire workspace, to a file using the matlab save() call. I would like to intercept the bytestream and postproc
Your best bet is probably to write the mat file to a tmpfs/ramdisk and then encrypt it before saving it to disk. You sacrifice portability and rely on the OS to provide secure virtual memory, but if you can't even trust the local disk, you're probably not going to be able to achieve satisfactory security.
By the way, why exactly are you unable to trust the local disk at all, even to the extent that you can't put your temporary file in a directory with permissions set to only allow access for the user owning the matlab process (and root)? Are you trying to implement a DRM system?