lexer

I'm migrating a C#-based programming language compiler from a manual lexer/parser to Antlr. Antlr has been giving me severe headaches because it usually mostly works, but then there are the small parts that do not and are incredibly painful to solve. I discovered that most of my headaches are caused by the lexer parts of Antlr, rather than the parser. Then I noticed parser grammar X; and realized that perhaps I could have my manually written lexer and then an Antlr generated parser. So I'm looking for more documentation on this topic. I guess a custom ITokenStream could work, but there appears

For all you compiler gurus, I wanna write a recursive descent parser and I wanna do it with just code. No generating lexers and parsers from some other grammar and don't tell me to read the dragon book, i'll come around to that eventually. I wanna get into the gritty details about implementing a lexer and parser for a reasonable simple language, say CSS. And I wanna do this right. This will probably end up being a series of questions but right now I'm starting with a lexer. Tokenization rules for CSS can be found here . I find my self writing code like this (hopefully you can infer the rest

What are and how to use the "island grammar" in antlr3? An island grammar is one that treats most of a language as a blob of text ("water") and picks out the part of the langauge of interest to parse using grammar rules ("island"). For instance, you might choose to build an island grammar to pick out all the expressions found in a C# program, and ignore the variable/method/class declarations and the statement syntax (if, while, ...). The real question is, "Should you use island grammars at all?". The positive benefits: you don't have to write a full, complete grammar for the language you want

I'm writing my own scripting language and I need a software tool which generates C++ code for parsing my language. I need a lexical analyzer and a parser generator which generates C++ code. It would be nice for me to be able also to generate a Visual C++ 2010 project. Suggestions? Try with Flex and Bison. They are good lexical analizers and parser generator usefull to define new languages. http://en.wikipedia.org/wiki/Flex_lexical_analyser http://en.wikipedia.org/wiki/Comparison_of_parser_generators for C/C++: http://epaperpress.com/lexandyacc/ Or look at: Boost.Spirit: "Spirit is a set of C++

I wrote a grammar with antlr 4.4 like this : grammar CSV; file : row+ EOF ; row : value (Comma value)* (LineBreak | EOF) ; value : SimpleValueA | QuotedValue ; Comma : ',' ; LineBreak : '\r'? '\n' | '\r' ; SimpleValue : ~(',' | '\r' | '\n' | '"')+ ; QuotedValue : '"' ('""' | ~'"')* '"' ; then I use antlr 4.4 for generating parser & lexer, this process is successful after generate classes I wrote some java code for using grammar import org.antlr.v4.runtime.ANTLRInputStream; import org.antlr.v4.runtime.CommonTokenStream; public class Main { public static void main(String[] args) { String source

Is there a good parser generator (think lex/yacc or antlr) for .NET? Any that have a license that would not scare lawyers? Lot’s of LGPL but I am working on embedded components and some organizations are not comfortable with me taking an LGPL dependency. I've heard that Oslo may provide this functionality but I'm not sure if it's a build time dependency or also a runtime dependency. Can anyone clarify what Oslo will provide? UPDATE What I would really like is a parser generator that is a build time only dependency. It looks like ANTLR has a runtime component. I just discovered that F# ships