How to use curl PHP in loop with 100 time? [closed]

问题

How to use curl in loop 100 times with send request and store response in an array?

For example : when curl uses in loop first time and gets 500 records and store in an array and again the same process with the second loop and gets 500 records form response and store in the same array without any issues. Finally, I need store 50K records in the array and I will use for insert records in my database.

I am working last 2 days but not getting any solution for this so please help me.

<?php
$final_data = array();
for($d=1;$d<=100;$d++)
{   
    $data = '{"request": {"header": {"username": "xxx","password": "xxx"},
    "body": {
    "shapes": [],
    "size_to": "",
    "size_from": "",
    "color_from": "",
    "color_to": "",
    "clarity_from": "",
    "clarity_to": "",
    "cut_from": "",
    "cut_to": "",
    "polish_from": "",
    "polish_to": "",
    "symmetry_from": "",
    "symmetry_to": "",
    "labs": [],
    "price_total_from": "",
    "price_total_to": "",
    "page_number": "1",
    "page_size": "50",
    "sort_by": "price",
    "sort_direction": "ASC"
    }}}';

    $json = json_decode($data,true);
    $json['request']['body']['page_number'] = $d;
    $data = json_encode($json); 

    $curl = curl_init();
    curl_setopt($curl, CURLOPT_POST, 1);
    curl_setopt($curl, CURLOPT_POSTFIELDS, $data);

    curl_setopt($curl, CURLOPT_URL, 'http://technet.rapaport.com/HTTP/JSON/RetailFeed/GetDiamonds.aspx');
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
    $result = curl_exec($curl);
    $dd = json_decode($result,true);

    foreach($dd['response']['body']['diamonds'] as $key)
    {       
        array_push($final_data,$key);
    }   
    curl_close($curl);
}
?>

回答1:

You could use curl_multi, it is more efficient when having several requests to perform.

$mh = curl_multi_init();
$handles = array();

for($i = 0 ; $i < 100 ; $i++){
    $ch = curl_init();
    $handles[] = $ch;

    curl_setopt($ch, CURLOPT_URL, 'http://technet.rapaport.com/HTTP/JSON/RetailFeed/GetDiamonds.aspx');
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

    curl_multi_add_handle($mh,$ch);
}

$running = null;
do {
    curl_multi_exec($mh, $running);
} while ($running);

foreach($handles as $ch){
    $result = curl_multi_getcontent($ch);

    $dd = json_decode($result,true);

    foreach($dd['response']['body']['diamonds'] as $key){       
        array_push($final_data,$key);
    }   

    curl_multi_remove_handle($mh, $ch);
    curl_close($ch);
}

来源：https://stackoverflow.com/questions/38967821/how-to-use-curl-php-in-loop-with-100-time

标签

php

arrays

curl