How can one mock/stub python module like urllib

Question

I need to test a function that needs to query a page on an external server using urllib.urlopen (it also uses urllib.urlencode). The server could be down, the page could cha

Answer 1

Probably the best way to handle this is to split up the code, so that logic that processes the page contents is split from the code that fetches the page.

Then pass an instance of the fetcher code into the processing logic, then you can easily replace it with a mock fetcher for the unit test.

e.g.

class Processor(oject):
    def __init__(self, fetcher):
        self.m_fetcher = fetcher

    def doProcessing(self):
        ## use self.m_fetcher to get page contents

class RealFetcher(object):
    def fetchPage(self, url):
        ## get real contents

class FakeFetcher(object):
    def fetchPage(self, url):
        ## Return whatever fake contents are required for this test

Answer 2

Adding onto Clint Miller's answer, to do this I had to create a fake class that implements a read method like this:

class FakeURL:
    def read(foo):
        return '{"some":"json_text"}'

Then to stub out urllib2.open:

# Stub out urllib2.open.
def dummy_urlopen(foo, bar, baz):
  return FakeURL()
urllib2.urlopen = dummy_urlopen