Posting another web query during render_GET or render_POST processing

问题

I have a small web server written using Twisted. One of the things I want to do is have it return a result from another web server as the response to loading a page. That is, the response to render_GET() at server A (via http://A.com/resource) should be the content of a different URL at server B (via http://B.com/resource2). The content returned by server B is dynamic, so I can't just cache it.

Right now, server A can render pages just fine, it just can't render this remote resource. I've tried with Agent(), but I can't seem to get the response from B let alone forward it to A. I know that somewhere I have to take that request from the render_GET and later write() and finish() it. That's done in the cbBody callback, which get called but can't get to the original request to populate it.

Here's a piece of the code for server A's resource handler:

def render_GET(self,request):
    # try with canned content just to test the whole thing
    bmpServer = BMPServer(ServerBURL,
                          "xyzzy",
                          "plugh")
    d= bmpServer.postNotification({"a":123},request)
    print "Deferred", d
    return NOT_DONE_YET

And this is the other code at server A:

theRequest = None

def cbRequest(response,args):
    print "response called"
    print response
    print args
    print 'Response version:', response.version
    print 'Response code:', response.code
    print 'Response phrase:', response.phrase
    print 'Response headers:'
    print pformat(list(response.headers.getAllRawHeaders()))  
    d = readBody(response)
    d.addCallback(cbBody)
    return d

def cbBody(body):
    print "Response body:"
    print body
    theRequest.write(body)
    theRequest.finish()
    theRequest = None

def cbError(failure):
    print type(failure.value), failure # catch error here
    print failure.value.reasons[0].printTraceback()

class BMPServer(object):
    def __init__(self,url,arg1,arg2):
        self.url = url
        self.arg1 = arg1
        self.arg2 = arg2

    def postNotification(self,message,request):
        theRequest = request
        bmpMessage = {'arg1':self.token,
                      'arg2':self.appID,
                      'message':message}
        print "Sending request to %s"%self.url

        print "Create agent"
        agent = Agent(reactor)
        print "create request deferred"
        print "url = %s" % self.url
        d = agent.request('POST', self.url,
                          Headers({'User-Agent': ['Twisted Web Client Example']}),
                          MessageProducer(bmpMessage))
        print "adding callback"
        d.addCallbacks(cbRequest,cbError)
        print "returning deferred"
        return d

When I run this as a standalone code (outside of the resource, using react() for example), it works fine. However, when I try to include it as shown above it just never seems to receive the data. I've got WireShark running so I can see the response is being returned from Server B, but the data never shows up in cbRequest().

For example, here's the output I see:

Sending request to http://localhost:8888/postMGCMNotificationService
Create agent
create request deferred
url = http://serverB:8888/postService
Message producer: body = {"arg2": "plugh", "arg1": "xyzzy", "message": {"a": 1}}
adding callback
returning deferred
testAgent: returning deferred
<Deferred at 0x10b54d290>
Writing body now
response called
<twisted.web._newclient.Response object at 0x1080753d0>
Response version: ('HTTP', 1, 1)
Response code: 200
Response phrase: OK
Response headers:
Response body:
{"result": false}
^CUnhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/Twisted-13.1.0_r39314-py2.7-macosx-10.8-intel.egg/twisted/web/_newclient.py", line 1151, in _bodyDataFinished_CONNECTED
    self._bodyProtocol.connectionLost(reason)
  File "/Library/Python/2.7/site-packages/Twisted-13.1.0_r39314-py2.7-macosx-10.8-intel.egg/twisted/web/client.py", line 1793, in connectionLost
    self.deferred.callback(b''.join(self.dataBuffer))
  File "/Library/Python/2.7/site-packages/Twisted-13.1.0_r39314-py2.7-macosx-10.8-intel.egg/twisted/internet/defer.py", line 382, in callback
    self._startRunCallbacks(result)
  File "/Library/Python/2.7/site-packages/Twisted-13.1.0_r39314-py2.7-macosx-10.8-intel.egg/twisted/internet/defer.py", line 490, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/Library/Python/2.7/site-packages/Twisted-13.1.0_r39314-py2.7-macosx-10.8-intel.egg/twisted/internet/defer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "AServer.py", line 85, in cbBody
    print theRequest
exceptions.UnboundLocalError: local variable 'theRequest' referenced before assignment

Looking at this a little more, it seems that if I could figure out a way to get the request over to cbBody() this would all work just fine.

回答1:

You can pass extra arguments to callbacks on a Deferred:

d.addCallback(f, x)

When d fires, the result is f(result of d, x). You can pass as many positional or keyword arguments as you like this way. See the API documentation for Deferred for more details.

来源：https://stackoverflow.com/questions/18222953/posting-another-web-query-during-render-get-or-render-post-processing

标签

twisted