Asynchronous code in custom ESLint rules

Question

The Story and Motivation:

We have a rather huge end-to-end Protractor test codebase. Sometimes it happens that a test waits for a specific fix to be implemented - usually as a part of a TDD approach and to demonstrate how a problem is reproduced and what is the intended behavior. What we are currently doing is using Jasmine's pending() with a Jira issue number inside. Example:

pending("Missing functionality (AP-1234)", function () {
    // some testing is done here
});

Now, we'd like to know when we can rename the pending() back to it() and run the test. Or, in other words, when the issue AP-1234 is resolved or sent to testing.

The Current Approach:

At the moment, I'm trying to solve it with a custom ESLint rule, jira NodeJS module, and Q. The custom ESLint rule searches for pending() calls with at least one argument. Extracts the ticket numbers in format of AP- followed by 4 digits and uses jira.findIssue() to check its status in Jira. If status is Resolved - report an error.

Here is what I've got so far:

"use strict";

var JiraApi = require("jira").JiraApi,
    Q = require('q');
var jira = new JiraApi("https",
    "jira.url.com",
    "443",
    "user",
    "password",
    "2");

module.exports = function (context) {
    var jiraTicketRegex = /AP\-\d+/g;

    return {
        CallExpression: function (node) {
            if (node.callee.name === "pending" && node.arguments.length > 0) {
                var match = node.arguments[0].value.match(jiraTicketRegex);

                if (match) {
                    match.forEach(function(ticket) {
                        console.log(ticket);  // I see the ticket numbers printed
                        getTicket(ticket).then(function (status) {
                            console.log(status);  // I don't see statuses printed
                            if (status === "Resolved") {
                                context.report(node, 'Ticket {{ticket}} is already resolved.', {
                                    ticket: ticket
                                })
                            }
                        });
                    });

                }
            }
        }
    }
};

Where getTicket() is defined as:

function getTicket(ticket) {
    var deferred = Q.defer();

    jira.findIssue(ticket, function(error, issue) {
        if (error) {
            deferred.reject(new Error(error));
        } else {
            deferred.resolve(issue.fields.status.name);
        }
    });

    return deferred.promise;
}

The problem is: currently, it successfully extracts the ticket numbers from the pending() calls, but doesn't print ticket statuses. No errors though.

The Question:

The general question is, I guess, would be: can I use asynchronous code blocks, wait for callbacks, resolve promises in custom ESLint rules? And, if not, what are my options?

A more specific question would be: what am I doing wrong and how can I use Node.js jira module with ESLint?

_{Would appreciate any insights or alternative approaches.}

Answer 1

The short answer is - no, you can't use asynchronous code inside of the rules. ESLint is synchronous and heavily relies on EventEmitter when it walks AST. It would be very hard to modify ESLint code to be async, but at the same time guarantee that events will be emitted in the right order. I think your only choice might be to write a sync rule that outputs enough information into the error message, then use one of the parsable formatters like JSON or UNIX and then create another application that you can pipe ESLint output to and do a async lookup in Jira based on the error message.

Answer 2

These answers remain valid in 2018.

For some insights from the eslint devs, see this conversation we had on their mailing list.

For a working example, in my "pseudo eslint plugin" I opted to use expensive but synchronous APIs and warn users about how best to use the "plugin" in their CI process.

Answer 3

Note: it does not answer original question about support of async code in ESLint custom rules, but provides an alternative solution to the issue.

I personally would not use ESLint in this case, it is supposed to be used to check if your code is written correctly and if you follow style guides; from my point of view missing tests is not the part of code check, it's more like your team internal processes. Also, this kind of requests may slow your ESLint executions significantly, if anyone runs it in real-time in their editor, calls will be made very often and will slow down the entire check. I would make this JIRA check a part of Protractor flow, so if the ticket is resolved, you will get a failed Protractor spec. (copied from the comment to make the answer complete)

Jasmine allows to mark specs as pending using xit(). I am not sure about pending() though, it works weird in Protractor. Also, Jasmine allows to call pending() inside a spec, so it will be marked as pending, but it is not implemented for Protractor yet (see issue). Knowing that, I would use a custom helper to define "pending specs", which should be checked for JIRA issue status. I guess you can still use Q to work with promises, I'll just post an alternative using WebDriver promises without external dependencies. Here is a modified version of getTicket():

function getTicketStatus(ticket) {

    // Using WebDriver promises
    var deferred = protractor.promise.defer();

    jira.findIssue(ticket, function(error, issue) {
        if (error) {
            deferred.reject(new Error(error));
        } else {
            deferred.fulfill(issue.fields.status.name);
        }
    });

    return deferred.promise;
}

Then there is a custom helper function:

function jira(name) {
    // Display as pending in reporter results, remove when pending() is supported
    xit(name);

    // Using Jasmine Async API because Jira request is not a part of Control Flow
    it(name, function (done) {

        getTicketStatus().then(function (status) {
            if (status === 'Resolved') {
                done.fail('Ticket "' + name + '" is already resolved.');
            } else {
                done();
                // pending() is not supported yet https://github.com/angular/protractor/issues/2454
                // pending();
            }
        }, function (error) {
            done.fail(error);
        });
    });

}

Usage example:

jira('Missing functionality (AP-1234)', function () {
    //
});

jira('Missing functionality (AP-1235)');

In case if request to JIRA fails or issue has a status Resolved, you will get a failed spec (using Jasmine async API). In all situations you will still have this spec duplicated as pending in reporter results. I hope it can be improved, when pending() functionality inside a spec is implemented.