I was recently working on a piece of C++ code for a side project (the cpp-markdown library, for the curious), and ran into a coding question that I\'d like some opinions on.
Fun stuff. The dynamic_cast in the tokenizer implies that you want to actually split Token into something that creates the Token based on the current position in the text stream and the logic in Token. Each token will either be self-contained or have to create a Token just like above to process text correctly.
That way you pull out the generic stuff and still can parse based on the hierarchy of tokens. You can even make this whole parser data-driven without using dynamic_cast.