After several posts about my work with Turbogears (Python framework for Web applications) and Luca (my general ledger application). Each post presents some problems and some alternatives, none is conclusive. It's time to write a conclusion.
The spark of all my posts was my disappointment with some aspects of Luca development, the main one being the difficulty of introducing changes in UI in a productive way. Every controller has three XML templates, one for AJAX version and two for the "lite" version (Javascript-free, for old or incompatible browsers). So, in a sense, Luca has two separate UIs. So I wrote a template generator and wrote the "meta-templates" using Python syntax instead of XML. Now, every change in UI is leveraged (I only need to change the generator, and the templates will be affected at once).
But then I noted that most of the data that I put in the meta-template, already existed in several other places: the controller and the model. So, if e.g. a table gets a new column, I need to change the model, the controller and the meta-template.
Automated testing was another problem: it is easy to test the controllers in Turbogears (Luca has something around 92% test coverage) but it is almost impossible to test the views, I would have to write a HTML interpreter to do that. And, since there are several XML templates per controller, they certainly have bugs.
First I blamed Model-View-Controller pattern for my woes, then Turbogears, then myself. I found another design pattern called "naked objects", kind of an anthitesis to MVC, that seem to solve all my problems.
During Luca development, a "pattern" emerged in the controllers: the "data transmission line". Different levels of the controller deal with a different data representation, and data goes through a "transformer" when crossing from/to another level. Luca controllers have three explicit data presentations: Web (that go to the XML templates or are coming from the forms), Intermediate (manipulated by all non-trivial validation and business rules), and SQL (ready to be fed into SQLObject). In truth we have two more data representations, that fortunately we don't have to deal with: SQLObject (that translates to/from SQL) and pure HTML/form (that is translated by XML templates and by CherryPy respectively).
The 3 explicit data transformers are inside the controllers, but now I feel that intermediate-SQL transformer belongs to the Model, and the Web-intermediate transformer should belong to View. The controllers are swollen, and yet there is redundant information in model, view and controller.
All cards in the table, my conclusions are:
* Turbogears, as well as all other web frameworks that I have seen, mislead (and sometimes even forces) the developer into using MVC the wrong way.
* They establish that one SQL table == one Model class, making the Model a kind of passive element in their MVC interpretation. Models should be much more intelligent than a simple object-relational mapper. Just because the ORM frees the developer from writing SQL (which I don't perceive as so advantageous), it does not mean that a ORM-mapped table is a full-fledged Model.
The developer can easily avoid this problem by developing more intelligent Model classes that in turn manipulate the ORM classes, and setting up a controller that just forwards the requests to the model.
* Most frameworks establish that a XML template system is the "View", when the View should be a normal and almighty Python class. Since XML is obviously not a programming language, every XML template has a (bad) way to allow Python code into the XML. The result is an underpowered, ugly templating language which makes PHP look pretty in comparison.
This one is more difficult to solve, though most frameworks allow the developer to write their own view class. Turbogears specifies the View as a controller method decorator, so it clearly thinks that View is just a "data transformer", and not an intelligent class that can take intelligent decisions.
Turbogears tries to give more power to the XML templates by "widgets" -- Python classes that can generate XML snippets. Being Python classes, they can be much more intelligent than a XML template. It is a good initial idea, but widgets have some problems: they are INCREDIBLY difficult to understand, and they are sent to the page via the controller, so it is another source of contamination of View businesses into the Controller.
The real solution is to have Views as full-powered Python classes, which *in turn* can use some kind of XML templating system.
* As a result of previous items, the Controller ends up swollen with code. But unfortunately the centralization of activities in Controller does not exempt the developer of writing redundant code in Views and Models.
* Most frameworks map Web pages and Web requests into controllers' methods. This is nice except that Controller must mimic the Web site structure. And site structure should be exclusive business of View. If a different UI must be supported (i.e. a new View must be written), the controller will not be adequate.
It already happened in Luca, just because I support AJAX and non-AJAX pages. The AJAX pages call JSON controller methods, and non-AJAX pages call other methods. It is necessary since data from form submits are different than data sent directly via JSON. (The JSON methods are "more pure" since they are not contaminated by the form -- View -- details, so they can also be called by the mobile client that runs in cell phone).
* The antithesis of MVC -- naked objects -- solves most of the abovementioned problems by compulsory automatic generation of UI code. It is an utopia; it would never suffice everybody's needs, but it is worthwile to purse that utopia, up to a certain extent.
It translates to: we should write Views that derive their behaviour from Model introspection as much as possible. It ensures that all function points will look the same, and more important, an improvement or bugfix in View will yield profits to every function point.
The risk is, of course, going too far and giving our application that boring appearance of the 1980's when database-tied 4th generation languages ruled the world.
But if we go too far, it is easy to fall back and add customizations to selected Views. It is way easier to commit the opposite mistake: NOT to pursue any kind of automatic UI generation, and discover that you cannot fix the mistake anymore because you have hundreds of incompatible Views.
In short, Web frameworks should:
1) review their interpretation of MVC;
2) promote the development of much more powerful Model objects, instead of simple ORM classes;
3) make Views as real objects instead of simple "filters", which in turn can use XML templates;
4) Encourage and supply good tools for at least some degree of automatic UI generation.
