Is it possible to efficiently insert or remove bytes from the middle of a large file, and if so how? Or am I stuck rewriting the entire file after the point
If it is a flat file, you have to rewrite the portion after the edits. If it is a file with logical structure (e.g., pointers to other parts of the file), then updates can be very efficient.