Alternative between parser rule and lexer rule

问题

The question is buried in the update section of another question, now specifically ask it.

I am using antlr3.4.

I have a simple grammar trying to parse 2 types of text, the line started with "#include" and others. Here is my grammar:

cmds
    : cmd+
    ;

cmd
    : include_cmd |  other_cmd
    ;

include_cmd
    : INCLUDE  DOUBLE_QUOTE  FILE_NAME  DOUBLE_QUOTE
    ;

other_cmd
    : (~INCLUDE)+
    ;


INCLUDE
    : '#include'
    ;

DOUBLE_QUOTE
    : '"'
    ;

FILE_NAME
    : ('a'..'z' | 'A'..'Z' | '0'..'9' | '_')+
    ;

New_Line 
    : ('\r' | '\n')+   
    ;

WS 
    : ('\t' | ' ')+  {$channel = HIDDEN;}
    ;

But I get such warning:

Decision can match input such as "{DOUBLE_QUOTE..FILE_NAME, New_Line..WS}" using multiple alternatives: 1, 2

As a result, alternative(s) 2 were disabled for that input

I guess this is because a double quote can match both other_cmd rule and DOUBLE_QUOTE rule, but I am wondering here, one is parser rule and the other is lexer rule, does this warning make sense?

Any help to clear this warning?

A side question - the warning message just says alternative 1,2, but it is not immediately clear to me what is 1 and what is 2, is there a way to render antlr to give more direct alternatives?

回答1:

I guess this is because a double quote can match both other_cmd rule and DOUBLE_QUOTE rule, ...

No, that is not the issue, since include_cmd starts with something that other_cmd cannot match.

Decision can match input such as "{DOUBLE_QUOTE..FILE_NAME, New_Line..WS}" using multiple alternatives: 1, 2

The warning means that input like foo" (a FILE_NAME followed by a DOUBLE_QUOTE) can be matched by the parser in more than one way:

1. greedy

2. ungreedy

ANTLR will choose the greedy parse, but since an ungreedy is possible, a warning is generated. If you explicitly tell the parser to match greedily, the warning would not be issued anymore:

other_cmd
 : (options {greedy=true;} : ~INCLUDE)+
 ;

A side question - the warning message just says alternative 1,2, but it is not immediately clear to me what is 1 and what is 2, is there a way to render antlr to give more direct alternatives?

No, not as far as I know. This warning is indeed rather cryptic. Alternatives usually denote the branches the parser can follow:

parser_rule
 : alternative_1 
 | alternative_2
 | alternative_3
 ;

But in your case, it seems ANTLR is talking about token ranges being the alternatives: DOUBLE_QUOTE..FILE_NAME being an alternative and New_Line..WS being the 2nd.

来源：https://stackoverflow.com/questions/14657396/alternative-between-parser-rule-and-lexer-rule

标签

antlr