I am trying to build a static program analyzer for a proprietary progamming language for a school project, and am currently trying to implement the parser from scratch.
I was wondering, what are the pros and cons of
- Parsing in 2 passses (tokenization followed by parsing to create an AST) VS
- Parsing in 1 pass (tokenization + construction of AST done together)?
For 1, my current understanding of the benefits is that it follows the rule for seperation of concerns, making the code easier to extend, maintain and test over the course of the project. However are these the only benefits for tokenizing/lexing before parsing and constructing the AST?