wftk: Design and implementation

[ wftk documentation home ]

Overall code organization

The wftk is a vast and complex system, and its arcane corners are known only to the initiate. (And that would be ... me.) It consists of two central modules, a number of adaptors in various classes, and several wrappers. Most of it is compiled into libraries; these libraries are either statically or dynamically linkable.

It is implemented (mostly) in ANSI C. The source code is actually contained in XML files which are sliced and diced by a Perl script to produce the C code which is compiled. The reason for doing this is because the same files contain implementation documentation. This technique is called literate programming, and it works for me. Your mileage may vary.

The HTML documentation generated during the compilation ends up in the source directories; to make things easy for a documentation distribution, however, all these files are duplicated in the code subdirectory here.

This document is an overview and index; jump to various pieces of it as follows. The list of interfacing systems is largely vaporware at present, but it gives you an idea of what goes where.

Build organization

After long years, the build organization on both Unix and Windows is identical. Of course, under Windows I'm using a 1999-vintage Mingw32 GNU make to call the Microsoft VC++ 5.0 compiler, but the principle is the same.

There are a number of configuration values in the Makefile.conf file. The version of Makefile.conf you retrieve from CVS or from a build will have one of several operating systems selected, depending on which one I was last testing on when I updated CVS. So be sure you select the one you're actually using, first. The same applies for whether various optional adaptors or front-ends are selected or not. Upshot: before you try to build anything, look at Makefile.conf.

Yes, I know about autoconf on Unix. I don't like it, because I'm one of those poor souls who spends a lot of time trying to get Unix open-source stuff to work under Windows, and autoconf projects never do. So it's kind of a personal thing.

Anyway, once you have Makefile.conf properly configured, you're pretty much home free. Run make and things should work.

Main modules

Version 1.0 of the wftk consists of the workflow core (the older of the two central modules), a number of adaptors, and a thin command-line utility which is mainly useful for testing. It is not yet terribly useful unless you're embedding it in your own application code, but despite this handicap, there are a few people here and there managing to make a go of using it. This is pretty gratifying.

After core-v1.0 is released, I'll be polishing up the other central module, the repository manager, as a sorta-kinda separate product. Weirdly, the repmgr is the module I'm using more heavily in production. The repmgr handles data, and I handle a lot of data, so I guess that makes sense. The repmgr will be released as repmgr-v1.0. Once that's done, I'll integrate the two modules and release the whole thing as wftk-v1.5. It will continue to be possible to run the wftk core engine without the repository manager; this may be useful for embedded systems, etc.

The relationship between the two central modules, adaptors, and application or wrapper code is explained and depicted in the wftk overview.

Both central modules rely on some shared code: the XMLAPI (described below) and the context handling functionality, which also includes handling of adaptors and configuration information. Read the code for more information.

Central modules, supporting code and command-line utilities:

Alternative implementations (mileage may vary):

XML-centric design

The wftk uses XML-like internal structures for nearly all data. The library I use to manage these structure is the XMLAPI. The XMLAPI is kind of like libxml, except different. The reason I chose to do things this way is not entirely because I'm an incorrigible nonconformist; much more importantly, I thought it would be easier to do a lightweight library than to understand libxml. Predictably, from what I've taken the time to understand about libxml, I seem to have re-implemented it almost in its entirety. But I had fun doing it.

The XMLAPI uses the expat parser for all parsing (whether from file or from string.)

There are three XMLAPI extension modules: xmlobj uses the XMLAPI to do record management for the repmgr and datasheet management for the wftk core; xml_template does a kind of simple XSLT for the repmgr; and xmlcgi takes the current CGI environment and builds an XML lookup structure for easy C/CGI programming.

XML handling functionality:


I use the catchall term wrapper to mean either a complete program using the wftk, or just a further library for inclusion into a complete program. The latter category is really what a wrapper is in common usage, so please don't be terribly confused by my sloppy language.

There are thus two types of wrapper, the UI wrapper (a complete program, a category with no members currently complete) and the scripting wrapper (an interface for a scripting language, currently represented by Python and Tcl.) Note that there is a lot of hope inherent in this list.

UI wrappers:

Scripting wrappers:


Of course, there are almost unlimited possibilities for talking to other systems, which is what the adaptors are for. There are several adaptor classes; each adaptor class characterizes a set of methods for talking to other systems in a particular range of needs. For each adaptor class, there is one or more implementations -- actual adaptors -- and could be any number of alternate implementations depending on your needs. I find that adaptors tend to take me from one to four days to complete, even if I'm not terribly familiar with the system they talk to. (The Oracle taskindex adaptor, for instance, took me four days even though I'd never used the OCI (Oracle Call Interface) before sitting down to write the adaptor.)

The wftk core engine uses most of the adaptor classes in this list; the repository manager makes due mostly with the "list" class, which represents generalized table-like entities with records in them. Click on each class to get an overview (many of these are out of date, so I apologize in advance) or on any implemented adaptor to see its code.

Read this overview of how local adaptors are managed.

The following classes of adaptors are defined (so far):
ACTION: invoking external APIs (wftk core)

DATASTORE: single value special handling (wftk core) DATATYPE: formatting data values (wftk core, soon repmgr) DSREP: datasheet repository (wftk core) LIST: lists and records (used by repmgr) NOTIFY: sending messages (wftk core) PDREP: procdef repository (wftk core) PERMS: permission rulebase (wftk core) TASKINDEX: index of active processes and tasks (wftk core) USER: user database (wftk core)

Copyright (c) 2002-2005 Vivtek. Please see the licensing terms for more information.