I'd like to match a regex pattern on a stream, but I am not sure what algorithm to use. I certainly don't want to load the entire file into memory.
I tried to figure out how to do this, but I have only very basic ideas. For example I could concat the chunks into some sort of aggregate string as long as I find a match. After that I cut the aggregate at the end of the match and continue. But if there is no match I would end up loading the entire string into memory.
I could give a maximum size to the aggregate, but I might lose matches that way. Another issue that if there is a longer match starting in the same position, then I would miss that. So I guess this algorithm is incompatible with possessive quantifiers too.
Is there a better solution for this problem?