ANTLR4 DefaultErrorStrategy fails to inject missing token

问题

I'm trying to run in TestRig the following grammar:

grammar COBOLfragment;

// hidden tokens
WS : [ ]+ -> channel(HIDDEN);
NL : '\n' -> channel(HIDDEN);

// keywords
PERIOD : '.';

DIVISION : 'DIVISION';
SECTION : 'SECTION';

DATA : 'DATA';
WORKING_STORAGE : 'WORKING-STORAGE';

FILE : 'FILE';

FD : 'FD';

EXTERNAL : 'EXTERNAL';
GLOBAL : 'GLOBAL';

BLOCK : 'BLOCK';
CONTAINS : 'CONTAINS';
CHARACTERS : 'CHARACTERS';

// data
INTEGER : [0-9]+;
ID : [A-Z][A-Z0-9]*;

dataDivision :
    DATA DIVISION PERIOD
    fileSection?
    workingStorageSection?
;

fileSection :
    FILE SECTION PERIOD
    fileDescription*
;

fileDescription :
    FD fileName=ID
//    (IS? GLOBAL)?          // 1. IS GLOBAL clause
//    (IS? EXTERNAL)?        // 2. IS EXTERNAL clause
    blockClause?
    PERIOD
;

blockClause :
    BLOCK CONTAINS? blockSize=INTEGER CHARACTERS
;

workingStorageSection :
    WORKING_STORAGE SECTION PERIOD
;

with the following input:

DATA DIVISION.
FILE SECTION.
FD FD01
WORKING-STORAGE SECTION.

Clearly the third line of input ("FD FD01") is missing the terminator PERIOD asked for in fileDescription rule.

The DefaultErrorStrategy correctly acknowledges this and conjures up the missing token:

On stderr the correct report is displayed: line 4:0 missing '.' at 'WORKING-STORAGE'.

But if the fragments commented out are enabled (that is, the clauses 'IS EXTERNAL' and 'IS GLOBAL' are brought in the grammar again), then single token insertion fails:

On stderr the misleading report is displayed: line 4:0 no viable alternative at input 'WORKING-STORAGE'

How to enable the full grammar (with IS EXTERNAL and IS GLOBAL clauses) retaining the ability to correct the missing PERIOD?

Side note 1: if I enable either IS EXTERNAL or IS GLOBAL, but not both clauses, then the DefaultErrorStrategy works nicely and injects the missing token.

Side note 2: the code generated for a grammar with both clauses enabled has the following extra code (compared to a grammar with just one of them enabled):

public final FileDescriptionContext fileDescription() ... {
    ...
    try {
        ...
        switch ( getInterpreter().adaptivePredict(_input,4,_ctx) ) {
        case 1:
            {
            setState(31);
            _la = _input.LA(1);
            if (_la==IS) {
                {
                setState(30); match(IS);
                }
            }

            setState(33); match(GLOBAL);
            }
            break;
        }
        ...
    }
    catch (RecognitionException re) {
        ...

And the adaptivePredict() call is the culprit, because it throws no viable alternative at input 'WORKING-STORAGE' before the parser has a chance to match(PERIOD) (in the generated code, not pasted here).

回答1:

I've managed to solve it adding a new clause for both IS clauses:

(here just the fragments changed)

...
fileDescription :
    FD fileName=ID
    isClauses?
    blockClause?
    PERIOD
;

isClauses :
    IS? GLOBAL (IS? EXTERNAL)?
|   IS? EXTERNAL
;
...

Now the DefaultErrorStrategy does its work and injects the missing PERIOD.

Why not isClauses : (IS? GLOBAL?) (IS? EXTERNAL)?; Well, I tried that first, of course. But got a warning (warning(154): rule 'fileDescription' contains an optional block with at least one alternative that can match an empty string) and no missing PERIOD injected.

来源：https://stackoverflow.com/questions/29934794/antlr4-defaulterrorstrategy-fails-to-inject-missing-token

标签

antlr4