Gulp: call an async function which provides its own callback from within a transform function

问题

I want to create a function for use in a pipe() call in Gulp that'll enable conversion of xlsx files to json.

I had this working with NPM package 'excel-as-json' for gulp 3, however Gulp 4 has forced me to actually understand what it is doing ;-)

Six hours in, and my incapacity of getting this to work due to a lack of js/async/streaming knowledge is punching my curiosity.

Code is as follows:

paths = {excel_sourcefiles: "./sourcefiles/*.xls*", excel_targetdir_local_csvjson: "./targetfiles_local/*.*"}


var replaceExt = require('replace-ext');
var PluginError = require('plugin-error')
var gulpFunction = require('gulp-function').gulpFunction // default ES6 export
var through = require("through2")
var convertExcel = require('excel-as-json').processFile;
var changed = require('gulp-changed');
var assign = Object.assign || require('object.assign');
var notify = require('gulp-notify');
var gulpIgnore = require('gulp-ignore');
var rename = require('gulp-rename');

gulp.task('excel-to-jsoncsv', function() {
  return gulp.src(paths.excel_sourcefiles)
    // .pipe(debug())
    .pipe(gulpIgnore.exclude("*\~*")) // Ignore temporary files  by Excel while xlsx is  open
    .pipe(gulpIgnore.exclude("*\$*")) // Ignore temporary files  by Excel while xlsx is  open
    .pipe(plumber({errorHandler: notify.onError('gulp-excel-to-jsoncsv error: <%= error.message %>')}))
    .pipe(changed(paths.excel_targetdir_local_glob, { extension: '.csv' }))
    .pipe(GulpConvertExcelToJson()) // This is where the magic should happen
    .pipe(rename({ extname: '.csv' })) // Saving as .csv for SharePoint (does not allow .json files to be saved)
    .pipe(gulp.dest(paths.excel_targetdir_local))
});

function GulpConvertExcelToJson() {
  return through.obj(function(chunk, enc, callback) {
    var self = this
    if (chunk.isNull() || chunk.isDirectory()) {
      callback() // Do not process directories
      // return
    }

    if (chunk.isStream()) {
      callback() // Do not process streams
      // return
    }

    if (chunk.isBuffer()) {
      convertExcel(chunk.path, null, null, // Converts file found at `chunk.path` and returns (err, `data`) its callback.
        function(err, data) {
          if (err) {
            callback(new PluginError("Excel-as-json", err))
          }
          chunk.contents = new Buffer(JSON.stringify(data))
          self.push(chunk)
          callback()
          // return
        })
    } else {
      callback()
    }
  })
}

I (now) know there are other excel > json gulp modules that could allow me to solve this problem without writing my own module, yet I'd like to understand what I should do different here.

The error returned is 'Did you forget to signal Async completion?', which I tried to not do. However, probably my attempt to fix the error 'this.push is not a function' with a var self = this is not what I was supposed to do.

Looking at examples like gulp-zip introduced me to unfamiliar notations, making me unable to autodidact my way out of this.

In summary:

how would I go about calling an async function during a through.obj function call, where the chunk's contents is updated with the (not under my control) async function that only provides me with a callback (err, data)?

回答1:

This problem is coming deep from the excel library. The end event should be finish.

Unsuccessful Try

I did the following, to install the modified excel.js:

rm -rf node_modules/excel
git clone https://github.com/bdbosman/excel.js node_modules/excel
cd node_modules/excel && npm i && cd ../..

This is not enough since the excel-as-json uses an old version of excel.

Either you have to modify the excel-as-json module, to use excel@1.0.0 (after the Pull Request is merged) or manually edit the file in node_modules. I will describe the second way here.

Temporary Solution

Edit the excel file after installing the modules.

I used:

sed -i "s/'end'/'finish'/g" node_modules/excel/excelParser.js

And then I ran:

$ gulp excel-to-jsoncsv
[19:48:21] Using gulpfile ~/so-question-gulp-async-function-call-xlsx/gulpfile.js
[19:48:21] Starting 'excel-to-jsoncsv'...
[19:48:21] Finished 'excel-to-jsoncsv' after 126 ms

And it apparently worked.

来源：https://stackoverflow.com/questions/59154091/gulp-call-an-async-function-which-provides-its-own-callback-from-within-a-trans

标签

javascript

node.js

asynchronous

callback

gulp