I'm thinking of making a Python-JavaScript converter/compile. I've been wondering about the pros and cons of the following methods to parse the Pythonic code, into JavaScript equivalent.

  1. Working with the output of ast.dump(ast.parse(code))
  2. Working with tokens.

I've been trying weigh the pros and cons of these methods, but well I'm unable to think of any major one, as I've never used them..
So, Basically could someone help me by telling me what the pros and cons of the two methods I've mentioned above are?
Also, if there is something better that I can use for this purpose, notify me about it..

PS: I'll be using Python to write the converter..


1 Answer 1


Big con of tokens: You'll have to parse, and this is both unnecessary and nontrivial.

Big con of ast.dump(ast.parse(code)): It's pointlessly stupid. ast.parse gives you a perfectly good AST, structured in an appropriate data structure. ast.dump gives you a string representation of that data structure, so you'd have to parse that to get back what ast.parse already gave you!

Leaving out ast.dump and using the AST from ast.parse, there are two big pros:

  • You don't have to worry about parsing Python code at all (and doing so is moderately complex).
  • ASTs are really the minimum you need to look at for any compiler-ish project at this scale. While the ast AST may have some complexity you don't need (e.g. line information, or Python features you don't support), I assume that simplifying a Python AST to your custom AST is simpler than generating your custom AST from tokens.
  • A minor benefit: There's already a visitor class built into ast, so you don't have to create one (a simple but somewhat boring task) for easy AST traversal.

Cons? Not so much. Go for ast, but don't be afraid to use additional data structures beyond that.

  • So use ast!! Thx.. Any cons?
    – pradyunsg
    May 11, 2013 at 14:23
  • As I said, it may have some fluff that gets in your way, so you may want to immediately turn the AST into a different, simpler AST.
    – user7043
    May 11, 2013 at 14:25
  • I was thinking of using ast.dump as I wanted to parse a string, maybe implement some functions, and then evaluate the string..
    – pradyunsg
    May 11, 2013 at 14:58
  • Token have flexibility, I can have the user type $, for using jquery..
    – pradyunsg
    May 12, 2013 at 8:20
  • @Schoolboy Yes, although then you aren't parsing Python but a dialect of it, which would have been a useful piece of info to include in the question. Assuming you're using tokenize, you don't get unlimited flexibility either: $ produces an "error" token (and I wouldn't be surprised if the next refactoring of the tokenizer turns it into an actual error), and indentation is checked during tokenizing. Of course, if you don't use tokenize and write your own lexer, you're facing even more work...
    – user7043
    May 12, 2013 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.