Looking for a clear definition of what a “tokenizer”, “parser” and “lexers” are and how they are related to each other and used?

前端 未结 4 1931
隐瞒了意图╮
隐瞒了意图╮ 2020-11-29 15:12

I am looking for a clear definition of what a \"tokenizer\", \"parser\" and \"lexer\" are and how they are related to each other (e.g., does a parser use a tokenizer or vice

4条回答
  •  感情败类
    2020-11-29 15:35

    (adding to the given answers)

    • Tokenizer will also remove any comments, and only return tokens to the Lexer.
    • Lexer will also define scopes for those tokens (variables/functions)
    • Parser then will build the code/program structure

提交回复
热议问题