Presentation Layer API

Today I want to think about an API for GUI development. Ideally this API should conform to a public standard independent of computer hardware, operating system, development environment, and even programming language. On the other hand, the development (and maintenance) process must be efficient and the resulting applications successful, which is merely to say that there are pragmatic requirements as well as idealistic.

The choice of API, of course, depends upon the application requirements, which I do not intend to specify here in any detail. Generally, though, the applications I hope to develop exhibit the following patterns:

1. The application (often) leverages a local data store or shared network workspace.


2. There are many associations between items in the data store.


3. The application relies on numerous real-time calculations to provide essential user views.


4. Maintenance of a transparent information flow between applications and data store(s) is integral.


5. Reliability of data access is mission critical to the application.
   
   • (Adapted from Mark Sigal.)
   
   

Points 2 and 3 are especially important. In our applications, many fields (and perhaps some graphics) will be computed in real time from data entered by the user (perhaps using a spin button or slider, for example) or from independent variables in the data store. I would also emphasize that for the most part there will be no reliance on a separate server. The data store *may* be a database system running on a network server, but it may *also* be a file or files on a shared network drive or the local hard drive. Thus my target applications are true "clients" or "thick clients" ("fat clients"). They may or may not be considered "rich clients" or "smart clients" (probably not), and certainly they are not "thin clients".

What this means is that the most technology-neutral open standards are nearly irrelevant in my situation. By this I mean that browser-based user interfaces (and their related W3C standards) will not be acceptable for me. I am open to changing my mind about this, but for the moment I am not planning to explore Web-based application development. I am attracted to the W3C standards such as CSS, DOM, HTML, RDF, SOAP, XForms, XHTML, XML and XSL precisely because they are among the most widely accepted open standards for interoperability. But because they are intended primarily for Web-based technologies, which I have rejected, it is not immediately obvious how to take advantage of these standards.

Now, I am quite aware of the fact that the W3C standards mentioned above are not limited to browser-based applications. XML, in particular, is extremely flexible and widely used in the exchange of a wide variety of data, both on the Web and elsewhere. For example, XRC is wxWidget's "XML Resource Code" API, and XUL is Mozilla's XML User Interface Language. There are, however, several problems associated with using XML for this purpose. First, XML files must be parsed and interpreted by a specific runtime environment (RTE) or rendering engine (RE), and even though cross-platform RE's can be provided, this requirement means that the API is no longer "technology neutral", because it is dependant upon the rendering engine. Second, XML files are not easy to write by hand (and not easily read by people either). The XRC page, for example, states that "It is highly recommended that you use a resource editing tool, since it's fiddly writing XRC files by hand." This means that one not only needs a RTE/RE, but a specialized editor as well. For the sake of completeness, there are at least two other XML-based API's: UsiXML and UIML, both with similar limitations. All four (XRC, XUL, UsiXML and UIML) require a third party RTE/RE because they do not use a Web browser. By contrast, XForms is an API that uses a Web browser as its RE. One advantage of XHTML/XML-based API's is that CSS is designed to work with markup languages so that it is relatively easy to use styles. More on XHTML/XML/CSS later.

Having decided not to develop Web-browser-based applications (for right or wrong and for reasons only partially discussed above), the next question becomes what kind of RTE/RE to use. Regardless of what API we come up with, the UI needs to be rendered by one engine or another. The options available depend, of course, on the programming language one chooses. Each language usually has one or more rendering engines (also called GUI toolkits) available. For example, Java applications typically use Swing, Tcl uses Tk, and C++ applications may use wxWidgets or FLTK, just to name two among many. The programming languages and associated GUI toolkits named also happen to work on multiple operating systems (are cross-platform).

For reasons not stated here but immediately obvious to Python enthusiasts everywhere, I have chosen Python for my programming language. Among the cross-platform toolkits that are available for Python are wrappers around the Tk (Tkinter), wxWidgets (wxPython) and FLTK (pyFLTK) toolkits mentioned above. Incidentally, wxPython includes an interpreter for XRC as well. To my knowledge, there are no Python interpreters (RE's) for XUL, UIML, or UsiXML.

Needless to say these GUI toolkits for Python have widely varying API's. Moreover, some of these toolkits are considered to have long learning curves and to be somewhat difficult to maintain. A few years ago a project, Anygui, was initiated to create a common (and hopefully simpler) API for different backend toolkits, including those mentioned above. Although that project has been abandoned, its API is still of interest as it represents a common denominator among the different toolkits.

More recently Greg Ewing has started a project (PyGUI) to develop a GUI API specifically for Python. Instead of relying on one or another of the standard GUI toolkits, his goal is to provide an implementation of the API that is small and lightweight using the underlying platform's GUI facilities (such as Gtk). Still in early development and not yet truly cross-platform, this API is of interest because of its focus on Python. Finally, I am aware of three other projects with API's that may be instructive: PyUI, PyGame, and PyOpenGL based on OpenGL.

So, what kind of strategy should I adopt in deciding on an API? First of all, it seems unlikely that I will find an API to adopt as is without changes or additions. If I follow a substantial portion of a given API (80%, 90%, 95%, 99% ?), however, I could claim it as a standard, but for my purposes it would be ok to come up with a new API if necessary. That said, I have no desire to reinvent the wheel without good reason, and it only makes sense to take advantage of work that has already been done whenever possible.

Since Mindwrapper uses Python syntax, if I am going to adopt a given standard, it would probably have to be the PyGUI API. Although PyGUI has no official standing, it was written by a respected senior member of the Python community and, by default, would seem to be the de facto Python standard if there is one. It remains to be seen, though, how closely I am willing to adhere to the PyGUI API. In terms of Python syntax, AnyGUI may also a good place to look for ideas.

If I decide to part ways with PyGUI to any degree, it would seem wise to look at the XML-based API's. I could not adopt any one of them as a standard, because I am unwilling to parse XML, but one or more of them could be a valuable source of inspiration. An even more likely source might be XForms. In any case, CSS also bears a careful examination. By itself, CSS cannot be a presentation API, but if the notion of styles is to be considered at all, CSS is the first place to look.

The last place to look for insights would be in the various Python GUI toolkits and their underlying (wrapped) implementations. These are the "last" sources to be considered because for the most part they are competing API's. On the other hand, picking and choosing the best features (assuming they can be implemented by modifying other toolkits) is not necessarily a bad idea. And finally, it might be worth taking a look at PythonCard, Dabo and Wax, frameworks using wxPython.

Lessons Learned

So far, this sprint has turned out to be anything but "lean and mean". The idea of evolving Acorn alongside Oak was not necessarily a bad one. In fact, I still think it was a pretty good one. It certainly led to some helpful insights, but as I might have predicted, the process became too sluggish. In spite of this, I have learned some things: (1) that "abstraction / presentation" is probably closer to "model / view" than "model / view / presenter", (2) that the basic design of Mindwrapper is nonetheless still sound, (3) that current Mindwrapper implementation can probably be simplified, (4) that Maps should be thought of as (and named) Tables, and (5) that except for layout and sizing, using more than one GUI toolkit is not as far-fetched as I once thought.

Model-View (MV) vs. Model-View-Presenter (MVP). What I had been calling 'presenters' I now see are actually 'views'. My so-called presenters are so closely tied to the underlying wxPython (or other toolkit) classes that they are really just views (wrapped views, but views nonetheless). It remains to be seen how far I will be able to go in separating the controller / presenter aspect of the applications from the views, but right now, the control is allowed at least to reside right there with the views. This should not be too surprising, I guess, as I realized long ago that the Mindwrapper applications will not necessarily be loosely coupled, but what coupling there is should be easily managed. YMMV. (See also MVC vs. MVP and Presenter First.)

Underlying design. At the start of the sprint, I had a few nagging concerns about: (1) certain aspects of the API and (2) the complexity of the underlying implementation. I'll talk about these issues a bit later, but I want to say here that despite those doubts, I've come away with a renewed outlook on and better appreciation of the basic design of Mindwrapper. The idea of the entire application being a directed, (mostly) acyclic, labeled graph is solid. It fits the idea of building components and building with components. The assemble / add syntax is sound, as is the notion of allowing indexed (with dot notation) syntax for items. The serialization mechanism seems to work well. The way of using *args for external references and **kwds for configuration traits in both class definitions and add() calls is clean and powerful. And finally, using delegation rather than inheritance for the presentation layer was a good idea.

Implementation. Although the basic design is sound, there is one aspect of the design that eventually led to an implementation that is not just complex but complicated. My goal was to set up hierarchies of named components. For example, let's say we define a patient record as consisting of "info" for relatively stable information about the patient and "care" for patient care related data. And then "info" will have, among other things, an item for "name", which in turn will have "last", "first", "middle" and so on.

The current implementation allows one to assemble such composite
components and then to use:

(1) attribute notation: pt.info.name.last
(2) indexing notation: pt[info][name][last]
(3) extended indexing: pt['info.name.last']

to access any given item. I didn't think all of this through ahead
of time. It sort of evolved. The need for (2) and especially (3)
was because sometimes there would be variables such as pt[key] for
sorting etc, but these notations came after (1). Now I am thinking
about dropping (1) for items and using attribute notation only for
real attributes, but not components of the hierarchy.

I've been reluctant to drop (1) because when it is appropriate, I
find attribute notation cleaner looking, but in a way it is
actually misleading. Subitems are similar to items in a dictionary,
not true attributes of the composite item. Furthermore, a rather
convoluted implementation was required to allow all three (or at
least both (1) and (2)(3)), which in the long run will surely be
more difficult to maintain. On the other hand, even though the
final implementation would be simpler by dropping (1), getting from
here to there would not necessarily be trivial. Maybe I should just
leave well enough alone. ... No, in the end:

Even though:
practicality beats purity;

Simple is better than complex.
There should be one obvious way to do it.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Special cases aren't special enough to break the rules.

Attributes should be attributes, items should be items, period.

Maps vs. Tables. One of the API decisions that became clear this week is that the word Table can mean both (1) rows of similarly structured items and (2) a "table of contents" where each item is different but well-specified. So the trait "rows = " works for the first and the "assemble / add" syntax works for the second. I think this is the only major change in the public API.

GUI toolkits. Yesterday I played around enough with Tkinter and especially pyFLTK to see that setting up delegation and working through the assemble / add syntax would be easy enough for several different tool kits. It will require a separate abstraction layer for each toolkit (although certain common aspects can be factored out). The hard part will be the layout and sizing aspects of the display.

The oak sprint

Starting Monday, August 8, I have four weeks to work on Mindwrapper, not all day every day, of course, but probably around 200 hours total. For reasons to be elucidated below, I am calling these 28 days the oak sprint, and here are my goals.

My highest priority is to develop a useful working application in Mindwrapper, something I've not done before. I have tested most, if not all, of the various components of Mindwrapper, but I have never put them all together in a completed application. Nothing would boost my confidence more than being able to do this. Of course I will have to be careful about the requirements. It wouldn't be enough just to make a trivial toy application, for example, but by the same token I surely don't want to set goals that are unreachable.

In the time available, it would seem feasible to wind up with a program that calculates, stores, and retrieves basic hyperalimentation fluid (HAF) orders. No bells and whistles, and perhaps not even the capability to print the orders at first, but even if the program will need to be extended and enhanced afterward, as long as it can at least store and retrieve the essential data and compute the necessary dependent values, then the sprint will have been a success.

To accomplish this goal in the time allotted, an agile approach will be essential. In particular, I will follow lean thinking principles, unit testing, and iterative development. The iterative development process is more like growing an oak tree from an acorn than building a house. Hence, the code name oak for this sprint.

One benefit of the iterative process is that we can save a snapshot of each iteration and later use them to write a Mindwrapper tutorial. Likewise, these snapshots could become part of the documentation for the resulting Oak application. I say "later" because agile methods value working software over comprehensive documentation. During the sprint, then, the only formal documentation written will be in the source code and unit tests.

A significant hurdle is the fact that at the present time Mindwrapper is neither stable nor complete. Thus, another goal for this sprint is to come up with a stable Mindwrapper API (and an effective implementation). Since it is easier to establish a useful syntax when it is actually being used, the oak sprint would seem to be an ideal occasion for completing this task as well.

Now, let me confess that even now I am thinking about writing a whole new implementation of Mindwrapper someday. I can see ways to implement the emerging Mindwrapper API without resorting to many of the "magic" techniques that I currently use (such as metaclasses and the "universal method" decorator). It would clearly be a mistake, however, to embark upon a rewrite now or anytime soon. First, I need to get some experience using Mindwrapper before refactoring it to any extent, and second, I need to get some practical use out of Mindwrapper as soon as possible.

But wait. Wouldn't it also be a mistake to write a lot of applications in Mindwrapper knowing that it is going to be changed? The response to this challenge is that only the implementation details will change. I am basically happy with the Mindwrapper framework as is. (1) The single-tier, two-layer abstraction-presentation architecture is fine the way it is. (2) The directed, acyclic, labeled graph information model is just what I want as well. Moreover, I am getting closer and closer to having an API that I am happy with. What I realize then, is that the Oak application (and its unit tests) will be a set of "black-box tests" of the essential framework with no dependency on the specific implementation. If and when I get ready to re-implement Mindwrapper, then, the new version can use Oak as its test suite.

So, here are the "deliverables" of the oak sprint.

1. a working application, Oak (HAF order calculator)
2. the basis for a Mindwrapper tutorial
3. early documentation for the Oak application
4. stable API for Mindwrapper
5. black box tests for Mindwrapper

Finally, I am not planning to blog regularly about the sprint, but I may post occasional reports to the Mindwrapper-users mailing list.

Mw: clinician-friendly language?

This week I started writing an article about Prism (for possible publication) and found myself referring to Mindwrapper (Mw) as a "domain specific language". This is not quite accurate, since Mw is certainly not specific to the medical domain, for example, but "very-high-level language" (VHLL) or "ultra-high-level language" (UHLL) are probably not helpful terms either. In any case, this perspective got me thinking again about the Mw API. After all, the goal is to encourage clinicians to become as actively involved in the software development process as they want to be, but there are still a number of idioms that are not intuitive to non-programmers. There are limits, of course, to how simple the (powerful) Mw SDK can be made, but if we are going to change the API very much, now is the time to do so, before there are a substantial number of applications written in it.

Off the top of my head I can think of four kinds of ways to make Mw easier for non-programmers to use, two having to do with the public Mw API: (1) changes to the architecture, and (2) changes in individual terms. The other two have to do with non-API improvements: (3) changes in the private Mw developer syntax, and (4) changes in terminology used in the documentation. Examples of these are: (1) change from "assemble/add" syntax to some kind of list/dict syntax, (2) changing "Unit" to "Document", (3) changing "specify" to some other term (or even another mechanism, I suppose), and (4) eliminating the terms "abstraction-presentation" in favor of, say, "data-view". Over the next few days (on-service) I plan to brainstorm about all four of these possibilities.

Today I will start with the assemble/add syntax that has become such an integral part of Mw. Well, I went back and took another look, and in the first place, I do not find this:

    components = [
        {'type': mw.StatusBar,
            'name': 'sb'},
        {'type': Workbook,
            'name': 'work',
            'ratio': 1,
            'grow': True},
        ]

any easier to read than:

    def assemble(self):
        self.add(
            name = 'sb',
            item = mw.StatusBar)
        self.add(
            name = 'work',
            item = Workbook,
            ratio = 1,
            grow = True)

Furthermore, the first syntax (list/dict) would be harder to implement than the assemble/add syntax. The bottom line is that I don't see a need for any wholesale changes in the assemble/add API.

Spectrum of Software Development

Lately I have been thinking about submitting an article on the rationale and philosophy behind Mindwrapper and Prism to an online medical journal (say JMIR). The problem is that Mindwrapper is a particular implementation written in a particular programming language, while the theory (rationale and philosophy) should be independent of any given computing environment, it seems to me. This week I want to see how far I can go in describing the theory behind Mindwrapper without mentioning aspects that are implementation specific. Today I will begin by describing the current spectrum of software development for medicine.

Briefly, software in medicine can be ranked according to size. Large enterprise systems are developed by commercial vendors and licensed to many institutions. Compared to the total number of users, clinicians have relatively little input in the development of such systems. This software therefore cannot be considered peer-reviewed, and it certainly is not independent. Small standalone applications, on the other hand, are often custom developed for clinicians (if not by them) and so usually have more clinician involvement. They have greater potential, then, to be independent and even peer-reviewed, although for a variety of reasons they often are not.

Mindwrapper event handlers

Prism and Mindwrapper are so intimately related that I could easily write about them both in the same place, but I want to reserve the Prism blog for thoughts about clinical application(s) and related issues. Therefore I've created a separate blog here to discuss issues of a more technical nature.

This posting started out to be just a test to make sure I have set things up correctly, but since I have an active issue on my mind right now, I'll write a little about it now.

The Mindwrapper syntax is essentially declarative in nature. Although it is possible to include procedural style Python code in the assemble() definitions, I have tried to discourage this practice (by making it mostly unnecessary). The user-developer will surely want to put procedural code someplace, though, so the trick is to keep it in well-designated locations. Since procedural code is executed in response to an event, such code may be considered an event handler. In the abstraction layer, this code is found primarily in the forward() (and possibly backward()) method(s) of dependent cells. [Note: when I have time, go back and review what mw.Map.update() does.] The forward() method acts as an event handler, where each event is a change in one of the observables to which the dependent cell is attached. In the abstraction layer there is just one type of event to be handled (the change event). Thus binding the handler to the event source is straightforward (and entirely transparent to the user-developer).

In the presentation layer, however, there are many different kinds of events. For some presenters, such as mw.Field, the expected events (EVT_CHAR and EVT_KILL_FOCUS) are handled the same way in every instance. In these cases, binding events to their handlers is automatic and transparent. On the other hand, some presenters, such as mw.Button, are designed to have custom commands associated with them. This might be called the controller role of a presenter. The user-developer must write the procedural code in a method and bind this code to the proper event (EVT_BUTTON). The question is where is the best place to define this method (the event handler)?

One option would be to define a subclass of mw.Button overriding the on_command() method. Access to outside entities would have to be through a "global" attribute self.frame or self.app. A second option would be to define a named handler in the composite presenter where the button is added and use the "command = " parameter of the add() method. A third option would be to define the handler upstream, say in the enclosing frame, and again use the "command = " parameter.

A fourth option has occurred to me, which is to define a separate class for the command itself, say with a do() method. The class would be passed to the "command = " parameter of the add() method and mw.Button would know to bind the EVT_BUTTON event to the do() method of the instance. Potentially there are two advantages of such an implementation. One is that the command could be given an undo() method as well. And the other is that the command could be used by other presenters, such as a menu item (not that a custom handler would not also be available to both button and menu item). At any rate, I have not implemented this last option, but it might warrant experimentation.