MySQL query error “Illegal hour value” causing loop and write issues in Google Apps Script

问题

I'm not well-versed in either language, so please bear with me on this. I'm trying to pull a full table with 100 rows from a remote MySQL database into a Google Sheet. I've managed to sort all the issues I've been having with this, but finally feel stuck. It seems the "illegal hour value" error I get from the SQL query during the loop is the main problem.

One of the columns in the MySQL database is "duration", and unfortunately, it can contain durations longer than 23:59:59. I only have access to call the procedure and cannot make any changes to the table. I get the

"illegal hour value"

error when a row hits a duration longer than 24 hours (e.g., 70:00:00). I tried to simplify things by using a try-catch to skip the error and continue writing on the Google Sheet, but then I get the

"number of columns in the data does not match the number of columns in the range.The data has X but the range has Y"

error in the final sheet.getRange() line.

I'm also unable to figure out how to pass multiple statements when executing the MySQL query. I tried to understand addBatch and a couple of other things, but it becomes too complicated for me. Maybe there's a simpler solution with the query, or maybe that's the only solution, because it just might work if I can also add a CONCAT query after the CALL query to convert the "duration" column to string before the data goes into the loop.

The code below has been updated to include the solution:

function getData3(query, sheetName) {

  //MySQL (MariaDB) connection and statements.
  var user = '';
  var userPwd = '';
  var url = 'jdbc:mysql://remote.server.example.com/database_name';
  var conn = Jdbc.getConnection(url, user, userPwd);
  var stmt2 = conn.createStatement();
  stmt2.setMaxRows(100);
  var rs2 = stmt2.executeQuery('CALL spdAdminGetPIREPS(api_key)');
  //Logger.log(rs2)

  //Function to convert raw binary data to string to be used for durations >23:59:59.
  function byteArrToString(byteArr){
    return Utilities.newBlob(byteArr).getDataAsString();
  }

  //Setting up spreadsheet, results array, and cell range.
  var doc = SpreadsheetApp.openById("id");
  var sheet = doc.getSheetByName("sheet_name");
  var results = [];
  var cell = doc.getRange('a1');
  var row = 0;

  //Loop to get column names.
  cols = rs2.getMetaData();
  colNames = [];
  for (i = 1; i <= cols.getColumnCount(); i++ ) {
    //Logger.log(cols.getColumnName(i));
    colNames.push(cols.getColumnName(i));
  }
  results.push(colNames);

  //Loop to get row data, catch type errors due to duration >23:59:59 and fix it.
  var rowCount = 1;
  while(rs2.next()) {
    curRow = rs2.getMetaData();
    rowData = [];
    for (i = 1; i <= curRow.getColumnCount(); i++) {
      try {
      rowData.push(rs2.getString(i));
      } catch (e){
      var bytes = rs2.getBytes(i);
      rowData.push(byteArrToString(bytes)); //Pushes converted raw binary data as string using function defined above.
      //Logger.log(JSON.stringify(rs2.getBytes(i))); //To see the raw binary data returned by getBytes() for durations >23:59:59 that throw an error.
        continue;
      }
    }
    results.push(rowData);
    rowCount++;
  }

  //Write data to sheet.
  sheet.getRange(1, 1, rowCount, cols.getColumnCount()).clearContent();
  sheet.getRange(1, 1, rowCount, cols.getColumnCount()).setValues(results);

  //Close result set, conn, and statement.
  //Logger.log(results);
  rs2.close();
  stmt.close();
  conn.close();
}

I know the two separate statements and all look ridiculous, but it seems they work, because I don't get the "no database" error with the query anymore. The simpler, single-line JDBC connector did not work for me, hence the current format for connecting to the MySQL server (Mariadb).

If there are no durations in the table longer than 24 hours, the code works and successfully writes the entire table into the Google Sheet.

To sum:

If I don't use try-catch, the loop stops with the error. If I use try-catch and continue, I get the number of columns mismatch error.

The end goal is to call the procedure and write the entire table onto a Google Sheet. Skipping problematic cells I can probably work with, but I'd definitely love to grab all of the data. I might be missing something trivial here, but any direction or help would be appreciated. Thanks in advance!

UPDATE: I found this question answered here, but cannot figure out how to utilize it in my case. I think that if I can pass multiple queries, I should be able to send a CONCAT query following the CALL query to convert the "duration" column from Datetime (I believe) to string.

UPDATE 2: @TheMaster's solution for try-catch helped with skipping the problematic cell and continue writing the rest. I'd love to find a way to convert all durations (the entire column or the ones >23:59:59) to string to capture all the data.

UPDATE 3: Based on @TheMaster's suggestions, using getInt() instead of getString() partially works by returning the hour but not the minutes (e.g., returns 34 if duration is 34:22:00). Need a way to convert when getString() is used.

UPDATE 4 (edited): Using getBytes(), the values returned are:
[51,52,58,52,56,58,48,48] for 34:48:00
[51,55,58,48,48,58,48,48] for 37:00:00
[56,55,58,48,48,58,48,48] for 87:00:00
[49,53,49,58,51,53,58,48,48] for 151:35:00
Which means:
[48,49,50,51,52,53,54,55,56,57,58] corresponds to [0,1,2,3,4,5,6,7,8,9,:]. How can I incorporate this conversion?

Using getLong(), the values returned are:
34 for 34:48:00, converted to duration -> 816:00
37 for 37:00:00, converted to duration -> 888:00
87 for 87:00:00, converted to duration -> 2088:00

UPDATE FINAL: @TheMaster's modified answer solved the problem by getting raw binary data and converting to string for durations >23:59:59. Code above has been updated to reflect all modifications; it works as written above.

回答1:

If you want to skip failed getString(), it should be easy:

try {
      rowData.push(rs2.getString(i));
      } catch (e){
      rowData.push("");//Keeps array straight
        continue;
      }

If you want to convert the time column to string, you need to use CAST(time AS char) or CONCAT('',time) on the CREATE_PROCEDURE query used to create spdAdminGetPIREPS

Alternatively, You can get the raw binary data using resultSet.getBytes() and change it back to string through blob using Utilities:

function byteArrToString(byteArr){
 return Utilities.newBlob(byteArr).getDataAsString();
}

Use it as

var bytes = rs2.getBytes();
rowData.push(byteArrToString(bytes));

If you could directly get blob, it will be easier to get appsScriptBlob

var jdbcBlob = rs2.getBlob();
var blob = jdbcBlob.getAppsScriptBlob();
rowData.push(blob.getDataAsString());
jdbcBlob.free();

回答2:

You currently use MySQL connector, and even if TIME value can be from '-838:59:59.999999' to '838:59:59.999999', MySQL driver throws an exception when getting a value of a TIME type not in the 0-24 hour range. This can make sense when using Resultset.getTime(i) when not really but doesn't when using Resultset.getString(i).

This can be disabled using noDatetimeStringSync=true, so changing URL to jdbc:mysql://remote.server.example.com?noDatetimeStringSync=true

Disclaimer: I am one of the maintainers of MariaDB java driver, but I would recommend to use MariaDB driver with MariaDB server (and MySQL one with MySQL server). You would have avoid this issue :)

btw, you can directly set database in URL jdbc:mysql://remote.server.example.com/database?noDatetimeStringSync=true avoiding a query.

来源：https://stackoverflow.com/questions/58161183/mysql-query-error-illegal-hour-value-causing-loop-and-write-issues-in-google-a