stdin stdout python: how to reuse the same input file twice?

Question

I am quite new to Python and even newer to stdin stdout method. Nevertheless I need to make my script usable for UNIX commands, in order to make it possible for example to process 2 input files at once with my script. This script works perfectly well with command line arguments:

newlist = []
def f1()
 .... 
def f2(input_file):
  vol_id = sys.argv[3]
  for line in input_file:
  if ... :
    line = line.replace('abc','def')
    line = line.replace('id', 'id'+vol_id)
  ....
  newlist.append(line)
return newlist

def main():
 if len(sys.argv) < 4:
   print 'usage: ./myscript.py [file_in... file_out... volume_id]'
   sys.exit(1)

 else:

    filename = sys.argv[1]
    filename_out = sys.argv[2]


    tree = etree.parse(filename)
    extract(tree)

    input_file = open(filename, 'rU')
    change_class(input_file)

    file_new = open(filename_out, 'w')
    for x in newlist:

        if '\n' in x:                   
           x = x.replace('\n', '')                
        print>>file_new, x

When I tried to add stdin stdout to it, I first had a problem with reading the same input file first, and for this reason made some chages so that it would be actually open only once. Here is my modified main():

            filename = sys.argv[1]
            filename_out = sys.argv[2]

            if filename == '-':
               filename = sys.stdin
            else:
                input_file = open(filename, 'rU')


            if filename_out == '-':
                filename_out = sys.stdout
                file_new = filename_out
            else:
                file_new = open(filename_out, 'w')

            input_file = open(filename, 'rU')
            tree = etree.fromstring(input_file)
            extract(tree)

            change_class(input_file)

            for x in newlist:

                if '\n' in x:                   
                   x = x.replace('\n', '')                
                print>>file_new, x

Then I ran my script like this:

./myscript.py - - volumeid < inputfile > outputfile

And I got this error message:

Traceback (most recent call last):
  File "./myscript.py", line 191, in <module>
    main()
  File "./myscript.py", line 175, in main
    input_file = open(filename, 'rU')
TypeError: coercing to Unicode: need string or buffer, file found

What am I doing wrong?

Answer 1

You are trying to use an open file object as a filename:

filename = sys.stdin

# ...

input_file = open(filename, 'rU')

You cannot re-read from sys.stdin anyway; you need to read all of the file into memory, then process it twice:

if filename == '-':
    input_file = sys.stdin
else:
    input_file = open(filename, 'rU')

input_data = input_file.read()

tree = etree.fromstring(input_data)
extract(tree)

change_class(input_data)

mwhere you'll have to alter change_class to handle a string, not an open file object.