progress bar for reading lines in text file

Question

I'm reading lines form in a text file and then performing actions per line. Due to the size of the text file and the time of each action 500 => seconds. I would like to be able to view the progress but not sure where to start.

Here is an example script I'm using, how would write that for this?

import os

tmp = "test.txt"
f = open(tmp,'r')

for i in f:
    ip = i.strip()
    os.system("ping " + ip + " -n 500")

f.close()

test.txt:

10.1.1.1
10.1.1.2
10.2.1.1
10.2.1.1

Answer 1

Here's a handy module: progress_bar.

It's short and simple enough; read the source for ideas on implementing your own.

Here's a very simple piece of code that, I hope, makes things clearer:

import time, sys

# The print statement effectively treats '\r' as a newline, 
# so use sys.stdout.write() and .flush() instead ...
def carriage_return_a():
    sys.stdout.write('\r')
    sys.stdout.flush()

# ... or send a terminal control code to non-windows systems
# (this is what the `progress_bar` module does)
def carriage_return_b():
    if sys.platform.lower().startswith('win'):
        print '\r'
    else:
        print chr(27) + '[A'

bar_len = 10
for i in range(bar_len + 1):
    # Generate a fixed-length string of '*' and ' ' characters
    bar = ''.join(['*'] * i + [' '] * (bar_len - i))

    # Insert the above string and the current value of i into a format
    # string and print, suppressing the newline with a comma at the end
    print '[{0}] {1}'.format(bar, i),

    # Write a carriage return, sending the cursor back to the beginning
    # of the line without moving to a new line. 
    carriage_return_a()

    # Sleep
    time.sleep(1)

As others have observed, you still need to know the total number of lines in the file in order to have a very meaningful progress bar. The most straightforward way to do that is to read the whole file to get a line count; but that's rather wasteful.

Incorporating this into a simple class isn't too hard... now you can create the progress bar and update() it whenever the value of interest changes.

class SimpleProgressBar(object):
    def __init__(self, maximum, state=0):
        self.max = maximum
        self.state = state

    def _carriage_return(self):
        sys.stdout.write('\r')
        sys.stdout.flush()

    def _display(self):
        stars = ''.join(['*'] * self.state + [' '] * (self.max - self.state))
        print '[{0}] {1}/{2}'.format(stars, self.state, self.max),
        self._carriage_return()

    def update(self, value=None):
        if not value is None:
            self.state = value
        self._display()

spb = SimpleProgressBar(10)
for i in range(0, 11):
    time.sleep(.3)
    spb.update(i)

Answer 2

Another starting point could be the progressbar module.

You can also download the source code [here], inside the tar.gz there's an example.py file with some good examples.