AOLserver front-end tools

[ discussion ]

# The AOLserver front-end to the wftk toolkit (wftk.tcl) is an AOLserver-specific Tcl
# extension which organizes repository and object interactions in a convenient way.
# More information is at
#  Copyright (c) 2004-2005, Vivtek.
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  GNU General Public License for more details.
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

ns_log notice "Registering wftk basic front-end v0.1 (c) 2004-2005, Vivtek. for more info."
The idea of the AOLserver front-end tools is pretty vague at this point. On the one hand, Tcl should have some regular object-oriented schema along with the other OO schemas. This may not be the ideal place to define it, but it's probably a good place to toss around some initial ideas. On the other hand, though, I also need some standard AOLserver tools for working with repositories. This is definitely the right place for that. So... That's the idea.

I really hate the existing Tcl wrapper for the wftk. I never could get the whole hash-table thing to work right, and so the API got deformed as a result. So the first thing I want to do here is to provide a better API, storing things in nsv variables so they're (1) global and (2) persistent. This makes a lot of sense to me.

So this Tcl interface is going to be in the "wftk" command. To get a new handle from the stack, we call [wftk gethandle]. To free a handle, we call [wftk release $handle]. Ideally, we could tie a handle to the current connection, but I don't think I can do that in Tcl alone and I don't want to mess with C extensions at the moment (they're hard enough when I don't have time pressure.) So we'll have to live with releasing, but at least this API will be halfway sensible.

At any time, the list of open handles will be in a list in [nsv_get wftk handlelist]. The wftk command should at some point offer some "info" options to hide that, but we'll work that out later.

There are two sets of functions in the wftk wrapper - the first set works outside the context of a repository, and the second set requires the first parameter to be the repository XML already loaded. You can see how the calls break down in this listing below.
See Initialize stuff
proc wftk {command args} {
   switch $command {
      gethandle  {
                   See Get XML handle for general use
      release    {
                   See Release XML handle
      repository {
                   See Open local repository (or retrieve open repository)
      ns_set2xml {
                   See Get new XML handle with contents defined by n_set
      xml2ns_set {
                   See Get ns_set from xmlobj-style fields

   set repos [wftk repository [lindex $args 0]]

   switch $command {
      list       {
                   See Retrieve list of keys
      listxml    {
                   See Retrieve XML handle with list results
      defn       {
                   See Retrieve XML handle with list definition
      get        {
                   See Get ns_set with fields of object
      getxml     {
                   See Get XML handle containing object
      add        {
                   See Add object from ns_set
      addxml     {
                   See Add object from XML handle
      mod        {
                   See Modify object from ns_set
      modxml     {
                   See Modify object from XML handle
      del        {
                   See Delete object by key
      tasks      {
                   See Retrieve lists of tasks in system or in object
      todo       {
                   See Retrieve list of tasks for auth user
      auth       {
                   See Get XML handle with authorized context
      notify     {
                   See Send notification
      submit     {
                   See Submit document
That defined, the other half of the equation for a useful AOLserver front-end is some generic URL handlers which will provide a set of general-purpose repository manipulation tools. I'm going to switch in a new page for this:
See URL handlers for general-purpose repository interface

Initialize stuff
There's not a lot of initialization to take care of, really. But here it is:
nsv_set wftk handle 0
nsv_set wftk handles [list]

nsv_set wftk critsec [ns_critsec create]

Get XML handle for general use
This is really quite straightforward. We have an nsv variable in which we maintain a count. The nsv module automatically locks for thread-safety when using nsv_incr, so getting a unique number is easy. We maintain a list of valid handles.

The string returned here is guaranteed unique, but as you can see, it's not automatically associated with any XML yet. But no worries; the XMLAPI wrapper won't freak if you try to free a fresh handle.
set handle wftk[nsv_incr wftk handle]
nsv_lappend wftk handles $handle
return $handle

Release XML handle
Once you're done with a handle, you need to free it to avoid memory leaks. Since we're maintaining a list of open handles, we could monitor this periodically. Obviously, it'd also be nice to tie release to the current connection or something -- but I'm pretty sure this would require a C component.

Note the sloppy error handling. Note, actually, the utter absence of error handling. TODO: write error handling. Ahem.
set handle [lindex $args 0]
xml free $handle
ns_critsec enter [nsv_get wftk critsec]
set handles [nsv_get wftk handles]
set i [lsearch -exact $handles $handle]
if {$i >= 0} {
   nsv_set wftk handles [lreplace $handles $i $i]
ns_critsec leave [nsv_get wftk critsec]
return OK

Open local repository (or retrieve open repository)
We identify repositories by the directory (or file) name given to open them with. Once a repository is open, it can stay open until explicitly closed. Since this means its definition is not reread, this is a tradeoff between preserving open connections and easy development. Caveat programmor, as always.

TODO: Fix utter lack of error handling. Sigh.

You can explicitly close a repository by releasing its handle, of course.
set reposloc [lindex $args 0]
if {[lsearch [nsv_get wftk handles] $reposloc] > -1} { return $reposloc }

if [nsv_exists wftk_repositories $reposloc] { set reposloc [nsv_get wftk_repositories $reposloc] }
if [nsv_exists wftk repos($reposloc)] {
   set handle [nsv_get wftk repos($reposloc)]
   if {[lsearch [nsv_get wftk handles] $handle] > -1} {
      return $handle

set handle [wftk gethandle]
set r [open $reposloc r]
xml parse $handle [read $r]
close $r
xml set $handle basedir [file dirname $reposloc]/
repmgr open $handle
nsv_set wftk repos($reposloc) $handle
return $handle

Retrieve list of keys
List calls return lists of keys. It's really just a convenient front-end for listxml, which retrieves a full-XML return, after which "list" simply gets the keys from all of them.
if {[llength $args] < 1} { error "no repository given" "wftk list" }
if {[llength $args] < 2} {
   return [repmgr list $repos] 
} else {
   return [repmgr list $repos [lindex $args 1]]

Retrieve XML handle with list results
List calls, of course, actually return lists of (possibly reduced) records. The listxml command returns that directly. It allocates a new handle to do it with.
set handle [wftk gethandle]
xml parse $handle "<list id=\"[lindex $args 1]\"/>"
repmgr list $repos [lindex $args 1] $handle
return $handle

Retrieve XML handle with list definition
This is easy. For error handling on this stuff, by the way, the caller can use [xml is_element $handle] to check for the NULL error return.
set handle [wftk gethandle]
repmgr defn $repos $handle [lindex $args 1]
return $handle

Get ns_set with fields of object
Get, of course, is just a front for getxml, and releases the handle itself when finished.
set obj [eval wftk getxml $args]
if [xml is_element $obj] {
   set ret [wftk xml2ns_set $obj]
   wftk release $obj
   return $ret
} else {
   wftk release $obj
   return ""

Get ns_set from xmlobj-style fields
Let's go ahead and clear this up (used above). March 1, 2005: I need this to give me unquoted HTML, so it's now using stringcontenthtml instead of stringcontent. Hopefully this isn't going to break anything.
set hdl [wftk gethandle]
set ret [ns_set create]
for {xml firstelem $hdl [lindex $args 0]} {[xml is_element $hdl]} {xml nextelem $hdl $hdl} {
   if [xml is $hdl field] {
      ns_set put $ret [xml attrval $hdl id] [xml stringcontenthtml $hdl]
wftk release $hdl
return $ret

Get XML handle containing object
Getting an object is also quite simple.
set handle [wftk gethandle]
repmgr get $repos $handle [lindex $args 1] [lindex $args 2]
return $handle

Add object from ns_set
Oh, we're on a roll now. To add an object, of course, we first build an XML object by scanning the ns_set, then we add it with addxml. The return value is the new object's key.
set handle [wftk ns_set2xml [lindex $args 2]]
set ret [wftk addxml $repos [lindex $args 1] $handle]
wftk release $handle
return $ret

Get new XML handle with contents defined by n_set
This could be made fancier (like giving it the ability to reuse handles) but it'll get the job done.
set handle [wftk gethandle]
xml create $handle record
set size [ns_set size [lindex $args 0]]
for {set i 0} {$i < $size} {incr i} {
   xmlobj set $handle "" [ns_set key [lindex $args 0] $i] [ns_set value [lindex $args 0] $i]
return $handle

Add object from XML handle
Adding XML objects, of course, interfaces directly with the repmgr. The return value is the new object's key.
return [repmgr add $repos [lindex $args 1] [lindex $args 2]]

Modify object from ns_set
This is a tad trickier, as we can't modify by direct replacement if we only look at fields to build the modified object (a great way to lose all workflow information!) Instead, we merge the constructed object.
set handle [wftk ns_set2xml [lindex $args 2]]
set ret [repmgr merge $repos [lindex $args 1] $handle [lindex $args 3]]
wftk release $handle
return $ret

Modify object from XML handle
And like all these, the XML version is a very thin wrapper indeed. Here, we actually modify by replacement.
return [repmgr mod $repos [lindex $args 1] [lindex $args 2] [lindex $args 3]]

Delete object by key
Boy, this is just too easy.
return [repmgr del $repos [lindex $args 1] [lindex $args 2]]

Retrieve lists of tasks in system or in object
No sense implementing this until it's implemented in the C wrapper.

Retrieve list of tasks for auth user
This, either.

Get XML handle with authorized context
This... I think an "authorized context" is logically a separate entity from the repository itself. I don't know yet. Until I do, there's no sense implementing this, either (not that it would have a point without a "todo" command anyway.)

Send notification
March 21, 2004 (Happy spring!): So notification is the next Big Thing I'm implementing. The wftk notify command can work either from a named notification or from an anonymous notification structure. Hmm. I'm not really yet sure whether it's of any use to have this wrapper around the basic repmgr-C wrapper, but just in case we need it later, here it is.
while {[llength $args] < 7} { lappend args "" }
set obj [lindex $args 3]
set release_obj 0
if [catch {
   set obj [wftk ns_set2xml $obj]
   set release_obj 1
} result] {
   ns_log notice "?? $result"
#                             list             key                   notification     to               subject
set ret [repmgr notify $repos [lindex $args 1] [lindex $args 2] $obj [lindex $args 4] [lindex $args 5] [lindex $args 6]]
if {$release_obj} { wftk release $obj }
return $ret

Submit document
November 17, 2004: Submitting a document is where document management starts. To submit a document, we create an object, then we attach the document to the object. During creation, we have the option of setting arbitrary field values.

If we already have an ns_set for our field values, we pass it as the third argument; otherwise, we use -fld as that argument, and give a name-value pair, followed by an arbitrary number of repetitions. (TODO: extend this to the rest of the API).
  wftk submit my_repos tracking_reports -fld title "This is our title" $body
The default is to attach the value of $body as the document. If the attachment is in a file, we use the alternative form
  wftk submit my_repos tracking_reports $fields -file $filename
set offset 2
if {![string compare [lindex $args $offset] -xml]} {
   set extras ""
   incr offset
   set xml_extras [lindex $args $offset]
   incr offset
} elseif {[string compare [lindex $args $offset] -fld]} {
   set extras [lindex $args $offset]
   incr offset
} else {
   set extras [ns_set create]
   while {![string compare [lindex $args $offset] -fld]} {
      incr offset
      ns_set put $extras [lindex $args $offset] [lindex $args [expr $offset + 1]]
      incr offset
      incr offset
set command submit
if {![string compare [lindex $args $offset] -file]} {
   incr offset
   set command store

if {$extras == ""} {
   return [repmgr $command $repos [lindex $args 1] $xml_extras [lindex $args $offset]]

set handle [wftk ns_set2xml $extras] 
set ret [repmgr $command $repos [lindex $args 1] $handle [lindex $args $offset]]
wftk release $handle

return $ret

This code and documentation are released under the terms of the GNU license. They are copyright (c) 2003-2005, Vivtek. All rights reserved except those explicitly granted under the terms of the GNU license. This presentation was prepared with LPML. Try literate programming. You'll like it.