Archive for February, 2009

report from the Berlinale sprint: how we can fix the Plone skin situation

Well! I am back from two wonderful weeks of traveling in Germany, Belgium, and Spain, and it’s high time that I write something about what I got done at the Berlinale sprint which was a highlight of the trip and is already almost two weeks past!

I headed into the sprint with the goal of figuring out how to make Plone 4 not have two different ways of registering templates and resources, like we have in Plone 3. By the end of the week I had written only a little bit of code toward this end, but I was able to spend a lot of time talking to Hanno Schlichting, Laurence Rowe, and Andi Zeidler and figuring out how we’d like things to work. I feel quite good about the progress we were able to make toward a clearer vision of “what good looks like” in this area, and have a better idea now about what further code needs to be written to make that vision a reality.

To summarize our overall conclusion… I’ve heard a number of people say “skin layers should die.” But this is a bit shortsighted, and probably reflects more the fact that people are tired of having two different ways to do this stuff, and that Zope 3 views and resources are a newer technology, rather than any clear superiority in functionality of the Zope 3 approaches over the alternative. So instead of ditching portal_skins, we should try to modernize it and make it compatible with the Zope 3 approaches.

But, this is a complicated thing to get right, and I would like to have feedback on whether we are on the right track and considering all the important use cases, so let me go into some detail on our current challenge and some concrete steps we can take to improve the situation.

The status quo

Currently in Plone 3 we have two different ways to register a resource that will be accessible via a URL in Zope. There are Content Management Framework (CMF) skin layers, which let you create a directory with a bunch of files (such as templates, images, Python scripts, CMF Form Controller objects, etc.) and associate it with a particular “theme” which may be enabled in your Plone site. These files are then magically available via Python attribute access on the portal root, thanks to a magic __getattr__ method in CMF’s Skinnable class, which the portal root class extends.

There are also browser views and browser resources, concepts borrowed from Zope 3. These are generally registered via declarations in the XML-based Zope Configuration Markup Language (ZCML) which result in these resources getting registered as multi-adapters of a context and a request in the global registry (a.k.a. global site manager) of the Zope Component Architecture (ZCA). These adapters are then looked up as one of the steps in the traversal process of the Zope publisher.

Of course, it’s confusing to have two different ways of solving the same problem that are both used in different places. (Just try to explain to a Plone newbie what the difference between portal_skins and portal_view_customizations is and when each should be used. There are answers, but they depend mostly on the technical details of the implementation and the history of Plone’s development, and are not much use to someone actually trying to use these systems to solve the practical question of how to override component X.) We’d really like to have one system for registering resources. But the problem is that neither of the existing approaches is an obvious winner. Each one has benefits and tradeoffs.

Comparison of CMF skin layers vs. Zope 3 views and resources

A. CMF skin layers / portal_skins
Pros Cons
  • Very easy both to add a new resource and to override an existing one
  • The concept of overriding a resource through the web by copying it to a ‘custom’ layer is fairly intuitive
  • Supports TTW customization for many different types of resources (templates, images, CSS, JS, KSS, etc.)
  • Includes a diff feature for comparing customized items to the filesystem version.
  • Includes a Find feature for determining which version of an item is active.
  • Tools exist to aid with exporting TTW customizations back to the filesystem.
  • Supports the concept of multiple themes which may be activated at different times
  • Resources are available globally and can collide with content ids
  • The way resources are traversed is a bit magical and may slow attribute access for anything acquired from the portal
  • A bit cumbersome to register a new layer (requires ZCML and Generic Setup)
B. Zope 3 views & resources / portal_view_customizations
Pros Cons
  • Views and resources can be registered more selectively for a particular interface only, so that two resources of the same name can be used in different contexts.
  • Built on infrastructure that is more “core” to Zope (and thus probably better maintained and optimized)
  • Resources can be namespaced by using resource directories, and thereby separated from content.
  • The portal_view_customizations UI is rather poor and doesn’t support some of the nice features of portal_skins.
  • Adding a new view or overriding an existing one requires a new ZCML registration.
  • There’s no way to override just one item from a resource directory.
  • Through-the-web overrides are only supported for templates.
  • There’s no good way to export through-the-web customizations back to the filesystem.
  • No simple way to handle subsite theming…a number of people have been struggling with this lately.

So what do we do?

So how can we continue our modernization toward Zope 3 approaches, without losing all the good things about CMF skin layers?

Based on the conversations I had at the Berlinale sprint, I think the answer involves three parts…

1. Register skin layer items as Zope 3 views and resources

The big coding task I tackled at the sprint was creating a package called ‘plone.resource’. This is intended to be a reimplementation of the DirectoryInformation class from CMF (the class which handles exposing filesystem items as objects within Zope) but with different behavior behind the scenes, such that each item in the directory gets registered as a browser view.

Some more details:

  • It will be possible to register new resources (including template-based browser views) just by creating a file in the right directory. If Zope is running in debug mode, we can detect new and removed files without needing a restart.
  • All the items in a particular resource directory will be registered for a particular, configurable Zope 3 browser layer.
  • For templates, additional parameters to the view registration (such as “for” and “name”) can be specified either in .metadata files or in <meta> tags in the <head> of the template’s HTML. (Of course, ZCML registrations for browser views not located in a plone.resource-managed directory will continue to work.)
  • It will be possible to configure the resources associated with a directory to show up in the global namespace a la portal_skins (for cases where you really do want a resource accessible anywhere), in a ++resource++ namespace directory a la Zope 3 resources (mostly for backwards compatibility), or in a named path like ‘images/’ (for cases where you want resources separated from content, but want a prettier URL than ++resource++).
  • Because everything gets registered as a browser view in the end, it will be easier for us to create a single UI which can handle customizing all kinds of resources.
  • It should be possible to override existing resources from another package in a fashion similar to z3c.jbot, by creating a file whose name is the full dotted path to the resource being overridden.
  • For now I am reusing the resource classes from CMF; though, as Hanno pointed out, it may be possible to replace these with something more lightweight at some point. Supporting DTML, CMFFormController, and Python scripts in the long term are not priorities for me, although I hope to make plone.resource pluggable enough that support for them and other types of resources could be provided by add-on products.

A lot of this still needs to be built, but the foundation is already in place.

2. Improve our mechanisms for managing how browser layers get activated

Since more things will depend on Zope 3 browser layers, we will need better support in plone.browserlayer for:

  • Making sure that items on a “theme” layer take precedence over items on an “add-on product” layer. (That is, controlling the order in which layer interfaces are applied to the request).
  • Enabling a theme layer only for certain sections of the site (as well as disabling any other theme layer within that section).

I haven’t done much thinking yet about the details of either the UI or the implementation for these enhancements. Suggestions welcome.

3. Revamp the UI for customization

portal_view_customizations is functional, but it needs a new user interface…possibly one that is based in the Plone control panel rather than the ZMI. I envision a listing of views and resources driven by a form that lets you filter by name of resource, type of resource, layer it’s registered on, interface that a view is “for”, etc. (Ideally we could even support a full text search of template contents.) For each resource shown in the results listing, all of the layers providing that resource need to be made evident, along with an indication of which is currently taking precedence. And it should be possible to see a diff of changes for items that have been customized through the web, as well as to export all such items for use in filesystem resource directories.

Alternatively, or in addition, an approach more like gloworm (which operates by the “poke-that-thing-on-the-page-and-tell-me-more-about-it-style” interface) could be taken. As long as plone.resource (or whatever else registers views) takes advantage of zope.configuration’s support for annotating the view registration with info about where the registration originated, and as long as we have some way to expose this when a view’s template is rendered, this shouldn’t be too hard to achieve.

Aside from the new things that need to be built, the main implications this has for the existing implementation are:

  • We need to support other types of resources besides templates, both in terms of finding them and storing persistent copies of them. (I need to investigate further whether it’s possible to adapterize the function of getting file contents somehow, so that we don’t need to have separate persistent and non-persistent versions of each resource class.)
  • When an item gets customized through the web and makes a local copy of the view registration, we need to change the layer to something like ITTWCustomized so that it doesn’t mask the global registration for the purposes of introspection.

A few last words

Okay, I think that’s probably enough to keep me busy for a little while. 🙂 Please don’t hesitate to ask questions; this plan incorporates a lot of small improvements and there are no doubt places where I should have been clearer about what something means or why something is a good idea. Constructive criticism and suggestions on how to make this even better are also welcome.

You’ll no doubt notice that I haven’t said much (if anything) about viewlets and portlets. Don’t worry, we know that understanding the distinction between these and learning how to create and register them is one of the common pain points in learning Plone theming today, and we hope to make things easier in Plone 4. But this problem is mostly orthogonal to the question of how resources get registered and customized, and is being dealt with elsewhere, so I’m not going to spend time on it here. For now, suffice it to say that as long as the approach used in Plone 4 for generating a section of a page is based on a Zope 3 browser view, it should be compatible with the improvements discussed here.

on European adventures

I’m safe and sound back in Seattle after two wonderful weeks of visiting friends in Europe. I posted a bunch of photos on facebook and am too lazy to do a full blog post here as well, but the following links will take you to the albums: