Python write to file based on offset

问题:

I want to create a python program which splits up a files into segments of specified width, and then a consumer program takes the segments and creates a duplicate of the original file. The segments might be out of order so I intent to use the offset value to write to the file. Is there a way I can achieve this with without creating a local array to hold all the data on the receiving end?

for example,

f = open(file, "wb") f.seek(offset) f.write(data) 

The idea behind this is that the program that sends the file might not be able to finish sending the file, and will resume again once it has started. I have a sample code below which the "combine_bytes" function throws an exception when I try placing data in the buffer location.

import sys import os  def SplitFile(fname, start, end, width):     t_fileSize = os.path.getsize(fname)     buffData = bytearray(t_fileSize)     for line, offset in get_bytes(fname, int(start), int(end), int(width)):      combine_bytes(buffData, offset, line, width)                 nums = ["%02x" % ord(c) for c in line]         print " ".join(nums)      f = open("Green_copy.jpg", "wb")     f.write(buffData)     f.close()   def combine_bytes(in_buff, in_offset, in_data, in_width):     #something like memcpy would be nice     #in_buff[in_offset:in_offset + in_width] = in_data      #this works but it's the mother of inefficiency      i = in_offset     for c in in_data:         in_buff.insert(i, c)         i  = i + 1   def get_bytes(fname, start, end, width):     t_currOffset = start     t_width = width     f = open(fname, "r+b")      if end != 0:     while t_currOffset < end:         f.seek(t_currOffset)         if (t_currOffset + t_width) > end:             t_width = end - t_currOffset         t_data = f.read(t_width)         yield t_data,t_currOffset         t_currOffset += t_width     else:        f.seek(t_currOffset)     t_data = f.read(t_width)     while t_data:         yield t_data, t_currOffset         t_currOffset += t_width         f.seek(t_currOffset)         t_data = f.read(t_width)      f.close()   if __name__ == '__main__':     try:     SplitFile(*sys.argv[1:5])     except:     print "Unexpected error:", sys.exc_info()[0] 

回答1:

I still could nt figure out what is your intent - but this version of combine_bytes will get rid of your "mother of your inefficiency" part (which actually is exactly that)

def combine_bytes(in_buff, in_offset, in_data, in_width):     #something like memcpy would be nice     #in_buff[in_offset:in_offset + in_width] = in_data      in_buff = in_buff[:in_offset] + in_data + in_buff[in_offset:]      return in_buff 

Of course this creates a new (larger) buffer for each call, and you have to replace your buffer on the caller scope with the one returned:

buffData = combine_bytes(buffData, offset, line, width)

回答2:

Found it. here is a better way which produces the what I wanted and is faster. _buffData[t_offset:t_offset + len(t_data)] = bytearray(t_data)

文章来源: Python write to file based on offset