ANTLR4: Unrecognized constant value in a lexer command

问题

I am learning how to use the "more" lexer command. I typed in the lexer grammar shown in the ANTLR book, page 281:

lexer grammar Lexer_To_Test_More_Command ;

LQUOTE : '"'        -> more, mode(STR) ;

WS   : [ \t\r\n]+   -> skip ; 

mode STR ;

STRING : '"'    -> mode(DEFAULT_MODE) ;

TEXT : .        -> more ;

Then I created this simple parser to use the lexer:

grammar Parser_To_Test_More_Command ;

import Lexer_To_Test_More_Command ;

test: STRING EOF ;

Then I opened a DOS window and entered this command:

antlr4 Parser_To_Test_More_Command.g4

That generated this warning message:

warning(155): Parser_To_Test_More_Command.g4:3:29: rule LQUOTE contains a lexer command with an unrecognized constant value; lexer interpreters may produce incorrect output

Am I doing something wrong in the lexer or parser?

回答1:

Combined grammars (which are grammars that start with just grammar, instead of parser grammar or lexer grammar) cannot use lexer modes. Instead of using the import feature¹, you should use the tokenVocab feature like this:

Lexer_To_Test_More_Command.g4:

lexer grammar Lexer_To_Test_More_Command;

// lexer rules and modes here

Parser_To_Test_More_Command.g4:

parser grammar Parser_To_Test_More_Command;

options {
  tokenVocab = Lexer_To_Test_More_Command;
}

// parser rules here

¹ I actually recommend avoiding the import statement altogether in ANTLR. The method I described above is almost always preferable.

来源：https://stackoverflow.com/questions/28384215/antlr4-unrecognized-constant-value-in-a-lexer-command

标签

parsing

antlr

antlr4

lexer