How do I unit test a module that relies on urllib2?

Question

I\'ve got a piece of code that I can\'t figure out how to unit test! The module pulls content from external XML feeds (twitter, flickr, youtube, etc.) with urllib2. Here\'s

Answer 1

I think the easiest thing to do is to actually create a simple web server in your unit test. When you start the test, create a new thread that listens on some arbitrary port and when a client connects just returns a known set of headers and XML, then terminates.

I can elaborate if you need more info.

Here's some code:

import threading, SocketServer, time

# a request handler
class SimpleRequestHandler(SocketServer.BaseRequestHandler):
    def handle(self):
        data = self.request.recv(102400) # token receive
        senddata = file(self.server.datafile).read() # read data from unit test file
        self.request.send(senddata)
        time.sleep(0.1) # make sure it finishes receiving request before closing
        self.request.close()

def serve_data(datafile):
    server = SocketServer.TCPServer(('127.0.0.1', 12345), SimpleRequestHandler)
    server.datafile = datafile
    http_server_thread = threading.Thread(target=server.handle_request())

To run your unit test, call serve_data() then call your code that requests a URL that looks like http://localhost:12345/anythingyouwant.