How to match a result to a request when sending multiple requests?

问题

A. Summary

As its title, Guzzle allows to send multiple requests at once to save time, as in documentation.

$responses = $client->send(array(
    $requestObj1,
    $requestObj2,
    ...
));

(given that each request object is an instance of
Guzzle\Http\Message\EntityEnclosingRequestInterface)

When responses come back, to identify which response is for which request, we can loop through each request and get the response (only available after executing the above command):

$response1 = $requestObj1->getResponse();
$response2 = $requestObj2->getResponse();
...

B. Problem

If the request object contains the same data. It's impossible to identify the original request.

Assume we have the following scenario where we need to create 2 articles: A and B on a distance server: something.com/articles/create.json

Each request has same POST data:

subject: This is a test article

After created, the Guzzle responses with 2 location come back:

something.com/articles/223.json
something.com/articles/245.json

Using the above method to link response-to-its-request, we still don't know which response is for which article, because the request object is exactly the same.

Hence in my database I cannot write down the result:

article A -> Location: 245.json
article B -> Location: 223.json

because it can be the other way arround:

article A -> Location: 223.json
article B -> Location: 245.json

A solution is to put some extra parameter in the POST request, e.g.

subject: This is a test article
record: A

However, the distance server will return error and does not create article because it does not understand the key "record". The distance server is a third party server and I cannot change the way it works.

Another proper solution for this is to set some specific id/tag on the request object, so we can identify it afterwards. However, I've looked through the documentation but there is no method to uniquely identity the request like

$request->setID("id1")

or

$request->setTag("id1")

This has been bugging me for months and still cannot resolve this issue.

If you have solution, please let me know.

Many many thanks and you've saved me!!!!

Thanks for reading this long post.

回答1:

I was having the same problem with this.

I solved it by adding a custom query parameter with a unique id generated for each request and add it to the request url (you will need to remember this ids for each one of them to address it after).

After $responses = $client->send($requests) you could iterate through the responses and retrieve the effective url $response->getEffectiveUrl() and parse it (see parse_url and parse_str) to get the custom parameter (with the unique id) and search in your array of requests which one has it.

回答2:

I've found a proper way to do it, Guzzle allow to add callback once a request is completed. So we can achieve this by setting it on each request in the batch

Each request by default can be created like this

$request = $client->createRequest('GET', 'http://httpbin.org', [
    'headers' => ['X-Foo' => 'Bar']
]);

So, to achieve what we want:

$allRequests = [];
$allResults = [];

for($k=0; $k<=10; $k++){
    $allRequests['key_'.$k] = $client->createRequest('GET', 'http://httpbin.org?id='.$k, [
        'headers' => ['X-Foo' => 'Bar'],
        'events' => [
            'complete' => function ($e) use (&$allResults, $k){
                $response = $e->getResponse();
                $allResults['key_'.$k] = $response->getBody().'';
            }
        ]
    ]);
}

$client->sendAll(array_values($allRequests));

print_r($allResults);

So now the $allResults has result for each corresponding request.

e.g. $allResults['key_1'] is the result of $allRequests['key_1']

回答3:

I found a much better answer.

I was sending batches of 20 requests at a time, 4 concurrently, and used the pooling technique where I got fulfilled, and rejected back, as in the documentation.

I found that I could add this code to the end of my requestAsync() function calls, when yielding / building the array (I do both in different places).

$request = $request->then(function (\GuzzleHttp\Psr7\Response $response) use ($source_db_object) {
            $response->_source_object = $source_db_object;
            return $response;
});

And then in the clousures on the pool, I can just access the _source_object on the response normally, and it works great. I find it a little hacky, but if you are just sure to use a name that NEVER clashes with anything in Guzzle, this should be fine.

Here is a full example:

use GuzzleHttp\Client;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Response as GuzzleResponse;

$client = new Client();
$requests = [];

// Simple set-up here, generate some random async requests
for ($i = 0; $i < 10; $i++) {
    $request = $client->requestAsync('GET', 'https://jsonplaceholder.typicode.com/todos/1');
    // Here we can attach any identifiable data
    $request->_source_object = $i;
    array_push($requests, $request);
}

$generator = function () use($requests) {
    while ($request = array_pop($requests)) {
        yield function() use ($request) {
            return $request->then(function (GuzzleResponse $response) use ($request) {
                // Attach _source_object from request to the response
                $response->_source_object = $request->_source_object ?? [];

                return $response;
            });
        };
    }
};

$requestPool = new Pool($client, $generator(), [
    'concurrency' => 5,
    'fulfilled' => function ($response) {
        // Then we can properly access the _source_object data once response has arrived here!
        echo $response->_source_object . "\n";
    }
]);

$requestPool->promise()->wait();

回答4:

I do it this way :

// create your requests
$requests[] = $client->createRequest('GET', '/endpoint', ['config' => ['order_id' => 123]]);
...
// in your success callback get 
$id = $event->getRequest()->getConfig()['order_id']

回答5:

An update related to the new GuzzleHttp guzzlehttp/guzzle

Concurrent/parallel calls are now run through a few different methods including Promises.. Concurrent Requests

The old way of passing a array of RequestInterfaces will not work anymore.

See example here

    $newClient = new  \GuzzleHttp\Client(['base_uri' => $base]);
    foreach($documents->documents as $doc){

        $params = [
            'language' =>'eng',
            'text' => $doc->summary,
            'apikey' => $key
        ];

        $requestArr[$doc->reference] = $newClient->getAsync( '/1/api/sync/analyze/v1?' . http_build_query( $params) );
    }

    $time_start = microtime(true);
    $responses = \GuzzleHttp\Promise\unwrap($requestArr); //$newClient->send( $requestArr );
    $time_end = microtime(true);
    $this->get('logger')->error(' NewsPerf Dev: took ' . ($time_end - $time_start) );

In this example you will be able to refer to each of the Responses using $requestArr[$doc->reference] . In short give an index to your array and run the Promise::unwrap call.

回答6:

I also had come across this issue. This was the first thread coming up. I know this is a resolved thread, but I have eventually come up with a better solution. This is for all those who might encounter the issue.

One of the options is to use Guzzle Pool::batch.

What batch does is, it pushed the results of pooled requests into an array and returns the array. This ensures that the response and requests are in the same order.

$client = new Client();

// Create the requests
$requests = function ($total) use($client) {
    for ($i = 1; $i <= $total; $i++) {
        yield new Request('GET', 'http://www.example.com/foo' . $i);
    }
};

// Use the Pool::batch()
$pool_batch = Pool::batch($client, $requests(5));
foreach ($pool_batch as $pool => $res) {

    if ($res instanceof RequestException) {
         // Do sth
        continue;
    }

    // Do sth
}

来源：https://stackoverflow.com/questions/22649888/how-to-match-a-result-to-a-request-when-sending-multiple-requests