This is a very bad situation, you need to refactor this ASAP - this is technical debt in it's worst - you don't even know how important the code really is - only speculate that it's important.
As to solutions ASAP:
Something that can be done is adding a custom compilation step. If you use Maven that is actually fairly simple to do, other automated build systems are likely to cope with this as well. Write a file with a different extension than .java and add a custom step that searches your source for files like that and regenerates the actual .java. You may also want to add a huge disclaimer on the auto-generated file explaining not to modify it.
Pros vs using a once-generated file: Your developers will not get their changes to the .java working. If they actually run the code on their machine before committing they will find that their changes have no effect (hah). And then maybe they will read the disclaimer. You are absolutely right in not trusting your teammates and your future self with remembering that this particular file has to be changed in a different way. It also allows automatic testing as wel, as JUnit will compile your program before running tests (and regenerate the file as well)
EDIT
Judging by the comments the answer came off as if this is a way to make this work indefinitely and maybe OK to deploy to other performance critical parts of your project.
Simply put: it is not.
The extra burden of creating your own mini-language, writing a code-generator for it and maintaining it, not to mention teaching it to future maintainers is hellish in the long run. The above only allows a safer way to handle the problem while you are working on a long-term solution. What that will take is above me.