Threading in python using queue

前端 未结 3 1486
执笔经年
执笔经年 2020-12-01 05:14

I wanted to use threading in python to download lot of webpages and went through the following code which uses queues in one of the website.

it puts a infinite while

3条回答
  •  -上瘾入骨i
    2020-12-01 06:13

    I don't think Queue is necessary in this case. Using only Thread:

    import threading, urllib2, time
    
    hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
    "http://ibm.com", "http://apple.com"]
    
    class ThreadUrl(threading.Thread):
        """Threaded Url Grab"""
        def __init__(self, host):
            threading.Thread.__init__(self)
            self.host = host
    
        def run(self):
            #grabs urls of hosts and prints first 1024 bytes of page
            url = urllib2.urlopen(self.host)
            print url.read(1024)
    
    start = time.time()
    def main():
        #spawn a pool of threads
        for i in range(len(hosts)):
            t = ThreadUrl(hosts[i])
            t.start()
    
    main()
    print "Elapsed Time: %s" % (time.time() - start)
    

提交回复
热议问题