What I want to do:

Parse source code, search for a beginning and closing tag of my own definition (one that does not conflict with any defined patterns in the programming language), and then replace then everything from the beginning of the opening tag to the end of the closing tag with some function of the data within. (For context: This is part of sort of building a language ontop of an existing language.)

So, say my pattern is @#$.

and I have a line of code:

int x = @#$42@#$;

I would like to replace


with f(42), which is some return value of some function f.

I know how to search for the tags and replace everything and that works just fine.

Something to the effect of:

//filePointer points to the beginning of a file
while(filePointer != EOF)
    //find index of first character of next opening tag
    //find index last character of next ending tag
    //infer data between tags
    //calculate new data from data between tags
    //replace index_first to index_last with new data

However, where I run into trouble, is that I do not want to perform this replacement if the tags lie within a string literal and can not think of an elegant way to handle this case.


int x = @#$42@#$;

would have a replacement.

String x = "@#$42@#$";

would not have a replacement.

Naive algorithm considered:

Iterate through file, track pairs of quotes indicating string literals.

Iterate through file again, find my tags, check to see if they are inside of any of the string literals, if so, do nothing.

I just feel that there is a better way that is escaping me.

Any suggestions would be most appreciated.

  • 6
    What you describe is a lexical preprocessor. Have you considered using an off-the-shelf preprocessor? Nov 29, 2014 at 12:13
  • I was not aware that there are off-the-shelf preprocessors. I will look into this. Thank you very much.
    – Zéychin
    Nov 29, 2014 at 20:08
  • 1
    @Zéychin cpp, and m4 come to mind first.
    – user40980
    Nov 30, 2014 at 0:16


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.