Topic: AOLserver -- HTTP client

AOLserver index ]
Our HTTP client is a replacement for ns_httpget and some of the ancillary functions like ns_urlencode. It is a complete HTTP client with some really nice features, and we were working on more before we found cURL, which is already written and does a lot more. (Now the intention is to make cURL into a Tcl extension.)

To see a simple example of use of vv_http, take a look at the dumb proxy, which is a cute little tool.

If you want the vv_http code, email us, because we still (still!) haven't finished our freebie download system.

Our HTTP client is a far more capable HTTP client than ns_httpget and ns_httpopen. (At least we think so.) Eventually it'll be even better.


  • GET, POST, and HEAD supported.
  • Complete access to headers of request and response.
  • Exposes parser functionality so that you can build and parse your own HTTP requests and responses.
  • User authorization.

Features we'd like to add:
  • Proxy support.
  • https support.
  • Support for protocols other than HTTP (like FTP, for instance.)

07/06/98Created in embryonic form for inclusion in the CyberCash integration module for AOLserver.
09/14/98Finished up enough for the CyberCash integration module.
09/15/98Introduced vv_urlencode and vv_urldecode.
04/24/99Introduced vv_http_textget.
08/31/99Introduced vv_base64_[en|de]code, and user authorization (basic schema).

Functions Besides the HTTP client, this file defines some helper functions that are useful in isolation, and also defines a really cheap text client which serves as a very simple example of how to use the HTTP client. For another example, you can see the Vivtek dumb proxy, which is built on this HTTP client.
vv_httpThe HTTP client itself. Documented more fully below.
vv_urlencodeReplaces ns_urlencode. Encodes a URL the way everybody else does it (so for instance a space is a + and not a %20).
vv_urldecodeDecodes the same.
vv_parsequeryParses a query string into an ns_set. Handy little function.
vv_textgetGiven a URL, returns the textualized content of the URL. Useful for text-based quickie apps. Uses vv_html2text.
vv_html2textExtracts text from HTML. Kind of. It's very crude but usually useful.
vv_base64encodeBase64-encodes a string. And you thought this couldn't be done in Tcl!
vv_base64decodeBase64-decodes a string.

More information about vv_http itself: In this implementation, the HTTP request and its response are stored in the same object. The object is stored in an ns_set, which is really a great data structure, I can't get enough of it.

Structure of an HTTP request:

  • Protocol, host, and port. E.g.
  • URL. E.g. /product/AAHT0
  • HTTP method. E.g. GET
  • Query form, with possible repetitions.
  • Header values.

Structure of an HTTP response:

  • Reference to the request that prompted it.
  • Header values returned from the server.
  • Mime type of returned data. (Parsed from header.)
  • Content length.
  • Content.
  • Response time, easy and cheap to calculate and nice to have sometimes.

Later this package or a derivative of this package should support smarter handling of returned HTML (i.e. at least some rudimentary parsing.) Right now, of course, you have ns_hrefs to get a list of links, at least. But it would be nice to get applets, images, and especially parse out forms.

In the HTTP request object, then, we have the following ns_set keys:
Request keys:
You can set these independently or simply write them with host or URL, e.g.
will correctly populate protocol, host, port, and URL.
querya "virtual" field which translates back and forth between the form.* fields and URL-encoded form.
methodGET or POST. (Default is GET.) You can write anything you want here, as long as the target server understands it.
versionHTTP/1.0 by default; you can change it if you want.
form.*all the values in the form to be queried. Again, if this request is a GET, you can include the form info in the URL and these key/value pairs will be written.
client.*all the header values in the request. Some will be set for you by default; you can obviously examine these and change them.

A note on protocol: for now, only http: is supported. Sorry. Talk to our buds at the Defense Department about export restrictions on cryptography if you want https:, but for others just wait a while and we'll get around to them.

Response keys:
server.*all the header values returned by the server.
typeMIME type of the content.
retverReturned version of the response. Generally identical to version.
statusReturned status code. Usually 200.
stattextText of return status message. Just in case it's interesting.
lengthContent length.
contentThe content returned.
timeResponse time of the transaction.
Finally, one status key:
statetwo values are permitted: "ready" and "complete", corresponding to whether the request has been sent or not. A future version of this package should implement a threaded model, in which a further state "processing" would be valid.

Commands on HTTP request objects:
makecreates a new request with certain defaults set.
setsets any of the above-named values with some special processing, for instance for URLs and hosts and stuff.
setform, setclient, setservershorthand for setting form/header values.
setauthgiven userid and password, sets the user authorization line.
getgets any named value.
getform, getclient, getservershorthand for getting form/hdr values.
showreq, showrespreturn text values corresponding to the actual text sent over the network by client and server.
parsereq, parserespparse said text values into the object.
requestactually executes a request against a server. (!)
postexecutes the request but doesn't wait for a reply. Useful when posting info to a CGI which does something more important than the reply you get.

Note that showreq and parsereq together provide a useful way of saving requests for later repeated execution. Parseresp is used internally to interpret the server's response and can be useful for other purposes, too.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.