lexer

I use the normal whitespace separation into the hidden channel but I have one rule where I would like to include any whitespace for later processing but any example I have found requires some very strange manual coding. Is there no easy option to read from multiple channels like the option to put the whitespace there from the beginning. Ex. this is the WhiteSpace lexer rule WS : ( ' ' | '\t' | '\r' | '\n' ) {$channel=HIDDEN;} ; And this is my rule where I would like to include whitespace raw : '{'? (~('{'))*; Basically it's a catch all rule to capture any content that does not match other

I am a complete ANTLR4 newbie, so please forgive my ignorance. I ran into this presentation where a very simple arithmetic expression grammar is defined. It looks like: grammar Expressions; start : expr ; expr : left=expr op=('*'|'/') right=expr #opExpr | left=expr op=('+'|'-') right=expr #opExpr | atom=INT #atomExpr ; INT : ('0'..'9')+ ; WS : [ \t\r\n]+ -> skip ; Which is great because it will generate a very simple binary tree that can be traversed using the visitor pattern as explained in the slides, e.g., here's the function that visits the expr : public Integer visitOpExpr(OpExprContext

Is it a lexer's job to parse numbers and strings? This may or may not sound dumb, given that fact that I'm asking whether a lexer should parse input. However, I'm not sure whether that's in fact the lexer's job or the parser's job, because in order to lex properly, the lexer needs to parse the string/number in the first place , so it would seem like code would be duplicated if the parser does this. Is it indeed the lexer's job? Or should the lexer simply break up a string like 123.456 into the strings 123 , . , 456 and let the parser figure out the rest? Doing this wouldn't be so

Javascript has a tricky grammar to parse. Forward-slashes can mean a number of different things: division operator, regular expression literal, comment introducer, or line-comment introducer. The last two are easy to distinguish: if the slash is followed by a star, it starts a multiline comment. If the slash is followed by another slash, it is a line-comment. But the rules for disambiguating division and regex literal are escaping me. I can't find it in the ECMAScript standard . There the lexical grammar is explicitly divided into two parts, InputElementDiv and InputElementRegExp, depending on

I am trying realize python like indent-depending grammar. Source example: ABC QWE CDE EFG EFG CDE ABC QWE ZXC As i see, what i need is to realize two tokens INDENT and DEDENT, so i could write something like: grammar mygrammar; text: (ID | block)+; block: INDENT (ID|block)+ DEDENT; INDENT: ????; DEDENT: ????; Is there any simple way to realize this using ANTLR? (I'd prefer, if it's possible, to use standard ANTLR lexer.) I don't know what the easiest way to handle it is, but the following is a relatively easy way. Whenever you match a line break in your lexer, optionally match one or more

I have a project where a user needs to define a set of instructions for a ui that is completely written in javascript. I need to have the ability to parse a string of instructions and then translate them into instructions. Is there any libraries out there for parsing that are 100% javascript? Or a generator that will generate in javascript? Thanks! Stobor Something like http://jscc.phorward-software.com/ , maybe? JS/CC is the first available parser development system for JavaScript and ECMAScript -derivates. It has been developed, both, with the intention of building a productive compiler