I'm thinking about using the basic mechanics behind the D20 system royalty-free, however, I'm unsure if the OGL allows that. Any thoughts appreciated. And if I can't use the D20 system royalty-free, then any suggestions on other well tested systems appreciated as well.
From what I understand this means the software itself does not need to be open, but the methods/data it uses that are OGL need to be available and must be the same data the program uses.
That explains that interactive games may be made using the OGL, but the d20 System is very different.
More information can be found in the Software FAQ for the d20 system.
I believe you are OK to do that, but I am not a lawyer. You should be able to contact the WotC and get clarification directly from them, which it what I would recommend in this particular case.
Personally, I wouldn't bother. Assuming you already know the d20 rules pretty well yourself, you shouldn't have too much trouble coming up with your own set of rules that work "like" the d20 system, but is independent of the d20 system.
Only a die-hard D&D fan would ever notice that your rules are actually different, and you completely avoid any problems you might run into with WotC.
You absolutely can. You need to have all software based on the d20 SRD follow the license such as clearly marking what is and is not Open Game Content and PI.
Here is software that is actively using the OGL : http://www.dundjinni.com/eula.htm