I'm about to take on a new project for a client designing a server-side python program that will poll a number of XML streams at regular intervals and populate a Postgresql database with results. The system will basically act as an XML cacheing daemon and populate a database. That database will then be accessed by a separate program which will serve that data in JSON to a public-facing website.
I'm wondering if anyone has suggestions as to the high-level design of the python caching/polling program. I'm thinking something class-based where a number of classes (one for each XML stream to be polled) descend from a common parent class, check for new content and then grab any if found. I'm thinking to have a table for each of the XML streams on the database, a column for each of the XML fields, and another table to keep track of the "last access" for each stream.
For this I'm planning on using Python 2.7, pyscopg, and ElementTree for XML processing.
Does anyone have any thoughts or criticism on my design or toolchain at this point? Is parsing my XML values to store in a Postgres database more work than is worthwhile? The front-end program that will be serving that data will be turning it into JSON rather than XML by the way.