Topic: AOLserver -- HTTP client

Features:

• GET, POST, and HEAD supported.
• Complete access to headers of request and response.
• Exposes parser functionality so that you can build and parse your own HTTP requests and responses.
• User authorization.

• Proxy support.
• https support.
• Support for protocols other than HTTP (like FTP, for instance.)

History:

07/06/98	Created in embryonic form for inclusion in the CyberCash integration module for AOLserver.
09/14/98	Finished up enough for the CyberCash integration module.
09/15/98	Introduced vv_urlencode and vv_urldecode.
04/24/99	Introduced vv_http_textget.
08/31/99	Introduced vv_base64_[en|de]code, and user authorization (basic schema).

Functions Besides the HTTP client, this file defines some helper functions that are useful in isolation, and also defines a really cheap text client which serves as a very simple example of how to use the HTTP client. For another example, you can see the Vivtek dumb proxy, which is built on this HTTP client.

vv_http	The HTTP client itself. Documented more fully below.
vv_urlencode	Replaces ns_urlencode. Encodes a URL the way everybody else does it (so for instance a space is a + and not a %20).
vv_urldecode	Decodes the same.
vv_parsequery	Parses a query string into an ns_set. Handy little function.
vv_textget	Given a URL, returns the textualized content of the URL. Useful for text-based quickie apps. Uses vv_html2text.
vv_html2text	Extracts text from HTML. Kind of. It's very crude but usually useful.
vv_base64encode	Base64-encodes a string. And you thought this couldn't be done in Tcl!
vv_base64decode	Base64-decodes a string.

More information about vv_http itself: In this implementation, the HTTP request and its response are stored in the same object. The object is stored in an ns_set, which is really a great data structure, I can't get enough of it.

Structure of an HTTP request:

• Protocol, host, and port. E.g. http://www.vivtek.com:80
• URL. E.g. /product/AAHT0
• HTTP method. E.g. GET
• Query form, with possible repetitions.
• Header values.

Structure of an HTTP response:

• Reference to the request that prompted it.
• Header values returned from the server.
• Mime type of returned data. (Parsed from header.)
• Content length.
• Content.
• Response time, easy and cheap to calculate and nice to have sometimes.

Later this package or a derivative of this package should support smarter handling of returned HTML (i.e. at least some rudimentary parsing.) Right now, of course, you have ns_hrefs to get a list of links, at least. But it would be nice to get applets, images, and especially parse out forms.

In the HTTP request object, then, we have the following ns_set keys:
Request keys:

protocol host	You can set these independently or simply write them with host or URL, e.g. http://www.vivtek.com:80/product/AAHT0
port url	will correctly populate protocol, host, port, and URL.
query	a "virtual" field which translates back and forth between the form.* fields and URL-encoded form.
method	GET or POST. (Default is GET.) You can write anything you want here, as long as the target server understands it.
version	HTTP/1.0 by default; you can change it if you want.
form.*	all the values in the form to be queried. Again, if this request is a GET, you can include the form info in the URL and these key/value pairs will be written.
client.*	all the header values in the request. Some will be set for you by default; you can obviously examine these and change them.

A note on protocol: for now, only http: is supported. Sorry. Talk to our buds at the Defense Department about export restrictions on cryptography if you want https:, but for others just wait a while and we'll get around to them.

Response keys:

server.*	all the header values returned by the server.
type	MIME type of the content.
retver	Returned version of the response. Generally identical to version.
status	Returned status code. Usually 200.
stattext	Text of return status message. Just in case it's interesting.
length	Content length.
content	The content returned.
time	Response time of the transaction.

Finally, one status key:

state

two values are permitted: "ready" and "complete", corresponding to whether the request has been sent or not. A future version of this package should implement a threaded model, in which a further state "processing" would be valid.

Commands on HTTP request objects:

make	creates a new request with certain defaults set.
set	sets any of the above-named values with some special processing, for instance for URLs and hosts and stuff.
setform, setclient, setserver	shorthand for setting form/header values.
setauth	given userid and password, sets the user authorization line.
get	gets any named value.
getform, getclient, getserver	shorthand for getting form/hdr values.
showreq, showresp	return text values corresponding to the actual text sent over the network by client and server.
parsereq, parseresp	parse said text values into the object.
request	actually executes a request against a server. (!)
post	executes the request but doesn't wait for a reply. Useful when posting info to a CGI which does something more important than the reply you get.

Note that showreq and parsereq together provide a useful way of saving requests for later repeated execution. Parseresp is used internally to interpret the server's response and can be useful for other purposes, too.