Bulk Parameterized Inserts

Question

I'm trying to switch some hard-coded queries to use parameterized inputs, but I've run into a problem: How do you format the input for parameterized bulk inserts?

Currently, the code looks like this:

$data_insert = "INSERT INTO my_table (field1, field2, field3) ";
$multiple_inserts = false;
while ($my_condition)
{
    if ($multiple_inserts)
    {
        $data_insert .= " UNION ALL ";
    }

    $data_insert .= " SELECT myvalue1, myvalue2, myvalue3 ";
}

$recordset = sqlsrv_query($my_connection, $data_insert);

A potential solution (modified from How to insert an array into a single MySQL Prepared statement w/ PHP and PDO) appears to be:

$sql = 'INSERT INTO my_table (field1, field2, field3) VALUES ';
$parameters = array();
$data = array();
while ($my_condition)
{
    $parameters[] = '(?, ?, ?)';
    $data[] = value1;
    $data[] = value2;
    $data[] = value3;
}

if (!empty($parameters)) 
{
    $sql .= implode(', ', $parameters);
    $stmt = sqlsrv_prepare($my_connection, $sql, $data);
    sqlsrv_execute($stmt);
}

Is there a better way to accomplish a bulk insert with parameterized queries?

Answer 1

Well, you have three options.

1. Build once - execute multiple. Basically, you prepare the insert once for one row, then loop over the rows executing it. Since the SQLSERVER extension doesn't support re-binding of a query after it's been prepared (you need to do dirty hacks with references) that may not be the best option.

2. Build once - execute once. Basically, you build one giant insert as you said in your example, bind it once, and execute it. This is a little bit dirty and misses some of the benefits that prepared queries gives. However, due to the requirement of references from Option 1, I'd do this one. I think it's cleaner to build a giant query rather than depend on variable references.

3. Build multiple - execute multiple. Basically, take the method you're doing, and tweak it to re-prepare the query every so many records. This prevents overly big queries and "batches" the queries. So something like this:

   $sql = 'INSERT INTO my_table (field1, field2, field3) VALUES ';
   $parameters = array();
   $data = array();
   
   $execute = function($params, $data) use ($my_connection, $sql) {
       $query = $sql . implode(', ', $parameters);
       $stmt = sqlsrv_prepare($my_connection, $query, $data);
       sqlsrv_execute($stmt);
   }
   
   while ($my_condition) {
       $parameters[] = '(?, ?, ?)';
       $data[] = value1;
       $data[] = value2;
       $data[] = value3;
       if (count($parameters) % 25 == 0) {
           //Flush every 25 records
           $execute($parameters, $data);
           $parameters = array();
           $data = array();
       }
   }
   if (!empty($parameters))  {
       $execute($sql, $parameters, $data);
   }
   

Either method will suffice. Do what you think fits your requirements best...

Answer 2

Why not just use "prepare once, execute multiple" method. I know you want it to either all fail or all work, but it's not exactly hard to handle that with transactions:

http://www.php.net/manual/en/pdo.begintransaction.php

http://www.php.net/manual/en/pdo.commit.php

http://www.php.net/manual/en/pdo.rollback.php