wftk and SOAP Web services

My initial SOAP adaptor is an action adaptor which represents SOAP services as remote procedure calls. The W3 people make a big deal of SOAP not necessarily being restricted to the definition of RPC conversations, but rather being a more general protocol. I'm not sure what to make of this, and as far as I can see, the rest of the industry isn't either -- it seems to me that the whole Web service concept is indeed simply an implementation of RPC on SOAP. So for wftk's SOAP debut, it seems appropriate to ignore the larger picture, which I'm good at.

The initial version of the ACTION_soap adaptor was underwritten by Startext GmbH in July of 2003, and it's the first intersection of SOAP and wftk. I'm sure it won't be the last. (And boy, wasn't it.)

Timeline/contents for ACTION_soap adaptor:
July 6, 2003: Basic test environment with SOAP.py and overview
July 19, 2003: libcurl as HTTP client library
July 25, 2003: Definition of ACTION_soap actions
August 18, 2003: ACTION_http adaptor functional enough to support SOAP August 19, 2003: ACTION_soap adaptor functional!
August 19, 2003: Work started on Java simple_soap class
August 25, 2003: Java simple_soap class works pretty well
August 27, 2003: Work started on Python SOAP server

• C-language SOAP action adaptor (client)
• Java simple SOAP class (client)
• Python SOAP server

from SOAPpy import SOAPServer
def echo(s):
    print "Called with '%s'" % s
    return s + s # repeats a string twice

server = SOAPServer(("localhost", 8080))
server.registerFunction(echo)
server.serve_forever()

Back to the ACTION_soap adaptor, though: just to see that this is actually doing something, you can also set up an even simpler test client:

import SOAPpy SOAPpy.Config.Debug=1 server = SOAPpy.SOAPProxy("http://localhost:8080/") print server.echo("Hello world")

Needless to say, when called, this returns "Hello worldHello world", while the server informs us "Called with 'Hello world'" and also gives a standard log entry line courtesy of SOAPProxy. Since this is run as a POST, though, the log entry gives us no information about the query, which is why I put the print statement in for a basic sanity check.

A much more interesting sanity check is afforded by setting debug mode on for the SOAP.py client -- this dumps the headers and XML bodies for the conversation in both directions, which makes it a very powerful tool for exploring other Web services and how they actually communicate. For our simple "Hello world" example above, the conversation looks like this (I've cleaned up indentation a tad to make things easier to understand.)

*** Outgoing HTTP headers ********************************************** POST / HTTP/1.0 Host: localhost:8080 User-agent: SOAPpy 0.10.1 (http://pywebsvcs.sf.net) Content-type: text/xml; charset="UTF-8" Content-length: 473 SOAPAction: "" ************************************************************************ *** Outgoing SOAP ****************************************************** <?xml version="1.0" encoding="UTF-8"?> <SOAP-ENV:Envelope SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"> <SOAP-ENV:Body> <echo SOAP-ENC:root="1"> <_1 xsi:type="xsd:string">Hello world</_1> </echo> </SOAP-ENV:Body> </SOAP-ENV:Envelope> ************************************************************************ *** Incoming HTTP headers ********************************************** HTTP/1.? 200 OK Server: <a href="http://pywebsvcs.sf.net">SOAPpy 0.10.1</a> (Python 2.1) Date: Fri, 25 Jul 2003 21:52:20 GMT Content-type: text/xml; charset="UTF-8" Content-length: 508 ************************************************************************ *** Incoming SOAP ****************************************************** <?xml version="1.0" encoding="UTF-8"?> <SOAP-ENV:Envelope SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/1999/XMLSchema" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"> <SOAP-ENV:Body> <echoResponse SOAP-ENC:root="1"> <Result xsi:type="xsd:string">Hello worldHello world</Result> </echoResponse> </SOAP-ENV:Body> </SOAP-ENV:Envelope> ************************************************************************ Hello worldHello world

I'm going to forebear a lot of comment at this point and simply note that for our purposes, most of this stuff is just boilerplate. We're not actually going to interpret any of the namespace information, for instance; we'll just parrot it out to the server, and ignore what comes back. What we really want is that "Result" string. It would appear that the official "return" value may in fact be a "return" element, but it's by no means required. The action spec for ACTION_soap will take the possibility of multiple returns into account, since SOAP explicitly supports it, but in the case where we only have a single one, then probably the best thing to do is simply to take the first value as the return.

libcurl as HTTP client library: (July 19, 2003)
Now that we have a server to call, and we more or less know how to build XML to talk to it and parse the XML coming back to extract an answer, we're ready to start writing an actual adaptor. As I mentioned, the first adaptor I'm writing is an action adaptor because that's what Startext needs. Later we can write other types of adaptors such as notification, list, and so on. Once we've done one, the rest take about an hour apiece, in my experience.

The obvious choice for anything written in C which is supposed to talk via HTTP is, of course, cURL at http://curl.haxx.se/, which rocks. The library version (as opposed to the old command-line program which is how I got to know cURL way back when) is called libcurl, and it's quite easy to use. I'm not going to go into much detail here, as the code speaks for itself.

I ended up separating the HTTP/libcurl handling into the ACTION_http adaptor so that it can be used for more general HTTP access. That finally got written on August 18.

Definition of ACTION_soap actions: (July 25, 2003)
The SOAP action is quite simple. Any action executes in the context of an object/datasheet, of course, and so what the SOAP action needs to do is to define (1) how parameters are generated from fields in the object, and (2) where results are to be placed in the object.

In general, then, the action specifier for a SOAP action has the following attributes and subelements; these are illustrated with the values necessary to call Borland's Babelfish Web service, one of the more entertaining Web services I ran across while resesarching SOAP:

Attribute "handler"	handler="soap"
Attribute "server"	server="http://ww6.borland.com/webservices/BorlandBabel/BorlandBabel.exe/soap/IBorlandBabel"
Attribute "function"	function="BabelFish"
Element "parm"	<parm name="translationmode" xsi:type="xsd:string">Eleet</parm> <parm name="sourcedata" xsi:type="xsd:string" field="msgtext"/>
Element "result"	<result name="return" field="xlatedmsg"/>

Note that parameters for the action are in "parm" elements, and that they may be constant (like "translationmode" is) or may draw values from the current object (like "sourcedata", which looks at the "msgtext" field for its input -- if the field attribute has a square bracket, of course, it will be treated as a format to build a value from different fields.) The "result" element specifies a target field or fields for results; if omitted, any return value from the SOAP call will be discarded. If the "name" attribute is omitted or blank, then the first return value will be taken.

The full action spec for a SOAP call to Babelfish, therefore, will look like this:

<action handler="soap" server="http://ww6.borland.com/webservices/BorlandBabel/BorlandBabel.exe/soap/IBorlandBabel" function="BabelFish"> <parm name="translationmode" xsi:type="xsd:string">Eleet</parm> <parm name="sourcedata" xsi:type="xsd:string" field="msgtext"/> <result name="return" field="xlatedmsg"/> </action>

This action can either be inserted into a procdef or it can be presented directly to the wftk core API.

This brings me to an interesting notion. If this action is invoked from a procdef, then barring any specification to the contrary, the individual user's computer currently running the procdef must perform the action. In the case of the SOAP adaptor, it's probably safe to believe that each such workstation has an Internet connection, and so possibly this isn't a problem here. But you can easily imagine an action adaptor which requires, for instance, specialized hardware (or even which pertains to a particular workstation attached to a particular device). In this case, any such action would have to name a queue, or location, for the processing of the action directive -- and if this spec is included in the action, then the procdef must block on the action. Then the action must be indexed to the taskindex, and the system must include a queue server which performs necessary actions as required. After completion of the action, the process continues.

This queue notion then also allows us to build workflow which controls costly operations or, say, things which only happen at night, or on particular machines, simply by writing an appropriate queue handler to process blocked actions.