Framework in which views are defined in the database - design flaw or reasonable?

Question

I'm working with a framework where the fields displayed on each web page are defined in the application database. Administrators can add new fields to pages or hide/completely delete old fields. The logic for each page is in a Java class - sometimes there's a one-to-one relationship between a page and a class, sometimes several pages use the same class.

The issue

This framework design creates a hidden dependency between the page data in the database and the logic in the Java class. If a new field is added to a page, the matching Java class must provide that field - or the page will break. The Java code has no compile-time knowledge of how the page is configured in the database (each customer database may have different configurations), which increases brittleness.

A typical solution has been to SELECT * from one or more database tables, allowing a degree of configuration as to which fields can be displayed. This leads to database-centric Java code with plenty of embedded SQL and little object-oriented design or domain models.

Solution plans

We've introduced a three-tier MVC-like architecture (controller -> service -> repository) with object-oriented models to the framework. It limits the ability to add fields to the pages without changing the underlying code. However, the change makes it possible to program in an object-oriented way and create domain models. Next up, we're planning to do something about the dependency between the database and code.

My long-term plan would be to move the page definitions from the database into the code (your standard HTML&JS views), limiting configurability but reducing brittleness. Any incompatibilities between the pages and Java would then become compiler errors when building the application instead of runtime errors.

Question(s)

1. Do you consider the framework practice of defining the view in the database and the controller in the code as sound design or something that should be addressed?
2. If you think the situation should be addressed, does my long-term plan seem reasonable?

Edit:

In addition to defining the layout, the view definitions in the database define which fields are expected to exist - "get field 'foobar' from the controller and show it here". The views themselves don't really contain logic. The Java classes backing up the views typically contain thousands of rows of UI and business logic.

Answer 1

If your actual software requirement is that views be customizable by the customer, then storing the view definitions in the database is not an unreasonable approach at all.

I think you will find that the conventional Model-View-Controller design breaks down when your requirement is user customization, for all but the simplest of scenarios. Consequently, converting your application into a more orthodox design is going to be an exercise in frustration, unless you carefully weigh the pros and cons.

I envision some sort of hybrid system whereby the SQL and the views are dynamically generated, so that only the required information is passed across the wire. You can also use some sort of Javascript templating system in the browser. But it won't look anything like a conventional MVC system, because MVC is designed to aid the programmer, not the user.

In a perfect world, your customized views are constrained to a small part of the system, and most of the application is actually "hard-wired." In other words, you won't custom generate all of your views, but only some of them, leaving the more orthodox views in MVC's capable hands, and custom generating views only as needed.

Answer 2

I can imagine a database-driven view representation that allows to edit views dynamically, without recompilation of any code. If this is your case, I suspect that the design has some merit.

If the Java code needs to be updated when the data describing a view in the DB has changed, I don't think the design is good; the description is better be held in the code.

In any case, 'plenty of embedded SQL' sounds bad enough. I'd think of some kind of an ORM.

Answer 3

Do you consider the framework practice of defining the view in the database and the controller in the code as sound design or something that should be addressed?

I think that storing user-specific information in the data layer is an excellent design principle, and is perfectly compatible with good programming practices. Especially if the expected audience will be provided some tool, even if only SQL, to manage their "views."

That said, I agree that you do have an issue, although it's more abstract than you stated. Your framework needs to have some clear and sensible way to handle unexpected user choices.

If you think the situation should be addressed, does my long-term plan seem reasonable?

If it's causing problems, it should definitely be addressed. But the appropriate plan depends on what the actual problem is, rather than the issue you've identified.

1. If the problem is how your site behaves if the database is not as expected, you should include tests and exceptions to that effect in your java data layer, and make sure you have a database-construction program, written in Java or whatever, that can create the necessary views and tables in your target database.

2. If the problem is users creating invalid views, you should consider if it's possible to provide a default behavior in your site. It's hard to be specific without knowing if your "views" are database views or the results of database queries, but either handling standard SQL column-types or limiting "field" rows to a list of "field types" should be part of the framework's design. (Gracefully omitting an invalid field is not a terrible solution, fwiw.)

3. If your problem is SQL code in Java, you may not have a problem at all. Such code should be moved to a dedicated database-connection class as part of your refactoring, but the choice between a Java-generated SQL string and a database-stored-procedure is an implementation detail, and you should only take the risk of moving from one to the other if you have a clear expectation of increased performance.

4. Lastly, if your problem is difficulty or delay in creating new pages, you should evaluate the process of how these pages are created. If most work isn't actually being done by your user base but are instead being done by your Java team, moving the necessary code into Java may make sense. Especially if your Java team isn't all familiar with SQL.

---

As I mentioned above, it's difficult to diagnose the appropriateness of a course of action without knowing the specifics of the framework in question. All software, especially application frameworks, is a means to an end and a shifting of complexity from run-time to design time. The specifics of an appropriate course are affected far more by "what are you trying to do" than "what should be done".