Asynchronous cURL using POST

Question

I am making a command line application. I need to send out multiple POST requests via cURL simultaneously after I have performed log in procedures - meaning outgoing requests must send session id etc.

The chain of events is as follows:

1. I open cURL connection with curl_init
2. I log in to remote site sending POST request with curl_exec and get returned HTML code as response
3. I send multiple POST requests to same site simultaneously.

I was thinking of using something like that:

// Init connection

$ch = curl_init();

// Set curl options

curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, 1);

// Perform login

curl_setopt($ch, CURLOPT_URL, "http://www.mysite/login.php");
$post = array('username' => 'username' , 'password' => 'password');
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post));
$result = curl_exec($ch);

// Send multiple requests after being logged on

curl_setopt($ch, CURLOPT_TIMEOUT_MS, 1);

for($i = 0 ; $i < 10 ; $i++){
    $post = array('myvar' => 'changing_value');
    curl_setopt($ch, CURLOPT_URL, 'www.myweb.ee/changing_url');
    curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post));
    curl_exec($ch);
}

But this doesn't seem to work as only the first request in loop seems to be sent.

Using curl_multi_init would probably one solution but I don't know if i can pass it the same cURL handle multiple times with changed options for each.

I don't need any response from server for those simultaneous requests but it would be awesome if it also can be done somehow.

It would be perfect if someone could push me in the right direction how to do it.

Answer 1

You'll need to create a new curl handle for every request, and then register it with http://www.php.net/manual/en/function.curl-multi-add-handle.php

here is some code i ripped out and adapted from my code base, have in mind that you should add error checking in there.

function CreateHandle($url , $data) {
    $curlHandle = curl_init($url);

    $defaultOptions = array (
        CURLOPT_COOKIEJAR => 'cookies.txt' ,
        CURLOPT_COOKIEFILE => 'cookies.txt' ,

        CURLOPT_ENCODING => "gzip" ,
        CURLOPT_FOLLOWLOCATION => true ,
        CURLOPT_RETURNTRANSFER => true ,
        CURLOPT_POST => 1,
        CURLOPT_POSTFIELDS => $data
    );

    curl_setopt_array($curlHandle , $defaultOptions);

    return $curlHandle;
}

function MultiRequests($urls , $data) {
    $curlMultiHandle = curl_multi_init();

    $curlHandles = array();
    $responses = array();

    foreach($urls as $id => $url) {
        $curlHandles[$id] = CreateHandle($url , $data[$id]);
        curl_multi_add_handle($curlMultiHandle, $curlHandles[$id]);
    }

    $running = null;
    do {
        curl_multi_exec($curlMultiHandle, $running);
    } while($running > 0);

    foreach($curlHandles as $id => $handle) {
        $responses[$id] = curl_multi_getcontent($handle);
        curl_multi_remove_handle($curlMultiHandle, $handle);
    }
    curl_multi_close($curlMultiHandle);

    return $responses;
}

Answer 2

There's a faster, more efficient option ... that doesn't require that you use any curl at all ...

http://uk3.php.net/manual/en/book.pthreads.php http://pthreads.org

See github for latest source, releases on pecl ....

I will say this, file_get_contents may seem appealing, but PHP was never designed to run threaded in this manner, it's socket layers and the like give no thought to consumption you might find that it's better to fopen and sleep inbetween little reads to conserve CPU usage ... however you do it it will be much better ... and how you do it depends on what kind of resources you want to dedicate the task ...