Building a template engine - starting point

Question

We're building a Django-based project with a template/generator component. This component will be separate from the project as such and can be Django/Python, Node, Java or whatever works. The idea is to generate questions based on a pattern. For eg., I would want to generate questions like "What is 2 + 3", "What is 5 + 8" based on the pattern "What is X + Y ; where X,Y < 10". In this sense, it's like a template engine.

The templates will contain references to objects with properties that are defined in the DB, say, a Bus. For eg, it could be something like

[object type="vehicle" weight="heavy"]

and it would have to pull a random object from the DB fulfilling the criteria : type="vehicle" weight="heavy" (bus/truck/jet) and then substitute that tag with an image, say, of a Bus. Also it would have to be able to handle some processing. Eg:

What is [X type="integer" lte="10"] + [Y type="integer" lte="10"]

[option X+Y correct_ans="true"]

[option X-Y correct_ans="false"]

[option X+y+1 correct_ans="false"]

The engine would be expected to fill in a random integer value <= 10 for X and Y and show radioboxes for each of the options. Would also have to store the fact that the first option is the correct answer.

Does it to make sense to write something from the scratch? Or is it better to use an existing templating system (like Django's own templating system) as a starting point? Any suggestions on how I can approach this?

edit : Clarification of use case

Answer 1

First of all, I think having the template system pull items from the database directly is a bad idea. It introduces a tight coupling between your templates (presentation) and database (data storage), with no way to add any logic between the two. This means that any change to your database schema has a potential impact on your templates, and vv.

Instead, the template engine should be able to consume arbitrary data structures, as long as they implement a dictionary-like interface. If you throw in the ability to call functions / methods, you can have the same functionality, only on top of objects rather than the naked database. If your objects have lazy-loading properties, then you can even trigger trips to the database from the template after all (by requesting such a property), only that the request still goes through your normal application logic before hitting the database. Not only does this mean you can change your database schema and catch the regression through the use of compatibility adapters, it also allows you to:

• add authorization to your data access
• pre-massage data before sending it to the template
• display data from sources other than a database
• implement all sorts of caching
• do i18n by calling a special function (many template engines call it _()) on anything you want translated (and, obviously, implementing that special function to do your translating)

Typically, this is solved either by passing a data structure to the template engine when evaluating a template, or by making the templates so that whatever is currently in scope at the time the template is called, is accessible in the template.

As to whether you should invent your own template engine: Making template engines is great fun, especially if you love building parsers and lexers and tweaking AST's and optimizing rendering code, but I'd first hit the market and see what's already out there. This page (despite the source, not specific to CherryPy) gives a nice starting point. If you know for sure that no template engine exists to fill your needs, by all means make your own, but be advised that it's harder than it looks.