boost-spirit-lex

Sorry if this is a newbie question, but I need to know which token definition produced a certain token. When I print the token ID, I just get an integer. I need to know which regex generated this token. Edit: Here's how I define my tokens: template <typename LexerT> class Tokens: public lex::lexer<LexerT> { public: Tokens(const std::string& input): lineNo_(1) { using boost::spirit::lex::_start; using boost::spirit::lex::_end; using boost::spirit::lex::_pass; using boost::phoenix::ref; using boost::phoenix::construct; // macros this->self.add_pattern ("EXP", "(e|E)(\\+|-)?\\d+") ("SUFFIX", "

I have a very simple path construct that I am trying to parse with boost spirit.lex. We have the following grammar: token := [a-z]+ path := (token : path) | (token) So we're just talking about colon separated lower-case ASCII strings here. I have three examples "xyz", "abc:xyz", "abc:xyz:". The first two should be deemed valid. The third one, which has a trailing colon, should not be deemed valid. Unfortunately the parser I have recognizes all three as being valid. The grammar should not allow an empty token, but apparently spirit is doing just that. What am I missing to get the third one

Let's consider following code: #include <boost/phoenix.hpp> #include <boost/spirit/include/lex_lexertl.hpp> #include <boost/spirit/include/qi.hpp> #include <algorithm> #include <iostream> #include <string> #include <utility> #include <vector> namespace lex = boost::spirit::lex; namespace qi = boost::spirit::qi; namespace phoenix = boost::phoenix; struct operation { enum type { add, sub, mul, div }; }; template<typename Lexer> class expression_lexer : public lex::lexer<Lexer> { public: typedef lex::token_def<operation::type> operator_token_type; typedef lex::token_def<double> value_token_type;

I'm working on a compiler and I would like to improve its performances. I found that about 50% of the time is spent parsing the source files. As the source file are quite small and I do quite a lot of transformations after that, it seems to me that it is perfectible. My parser is a Boost Spirit parser with a lexer (with lexer::pos_iterator) and I have a middle sized grammar. I'm parsing the source into an AST. My problem is that I have no idea what takes the most time during the parsing: copies of AST nodes, lexer, parser rules or memory. I don't think that it is I/O problem since I'm working

I try learning to use boost::spirit. To do that, I wanted to create some simple lexer, combine them and then start parsing using spirit. I tried modifying the example, but it doesn't run as expected (the result r isn't true). Here's the lexer: #include <boost/spirit/include/lex_lexertl.hpp> namespace lex = boost::spirit::lex; template <typename Lexer> struct lexer_identifier : lex::lexer<Lexer> { lexer_identifier() : identifier("[a-zA-Z_][a-zA-Z0-9_]*") , white_space("[ \\t\\n]+") { using boost::spirit::lex::_start; using boost::spirit::lex::_end; this->self = identifier; this->self("WS") =