xmlp -- Language definition

Topic: xmlp -- Language definition

As is perhaps obvious from the application's name, xmlp takes a literate programming presentation which is coded in XML. The XML in question is somewhat questionable, perhaps, as the presentation items allow free embedding of arbitrary markup, but I really don't care, as the whole thing does what I need it to do, which is define pieces of code contiguous with nicely formatted documentation. If somebody else knows more about XML and can tell me how I can specify arbitrary markup in an element, then maybe I'll be able to write a DTD. Otherwise I guess we're just kinda out of luck.

The quick and dirty markup tokenizer (qdmt) will really not care one whit about whether all this is valid XML, so maybe I'm just coding myself into some remote corner of the quasi-XML universe.

Anyway, I've defined a few elements which I'm using here. They are:

<litprog>
<object>
<item>
<piece>
<variant>
<insert>

They work as follows:

<litprog>
This is simply the root element. Takes no attributes.
<object name="NAME" language="LANGUAGE" item="START" variant="VARIANT"/>
The object element defines an output file. The list of output files is available to the index page in the [##objectlist##] tag. The file will be written to "NAME" relative to the current directory, and the tangle will start at the item called "START". All pretty obvious.
The language will come later, probably after QDMT is integrated with the application. It will specify the base language of the object as a default language for each item along the line. Knowing the language of a stream will allow tangle to generate indices of defined data structures and so forth which weave will then be able to use to organize the code. But right now it's still blue-sky.
The variant feature isn't done yet either (although I'm working on it.) What this will do is to allow selection of certain pieces and rejection of others when tangling multiple objects from a single presentation. If a piece is shared by two variants, then it won't have a variant tag. I think the variant feature will be pretty cool, but that remains to be seen. Watch this space for further details. I want to develop QDMT in both C and Perl using a variant structure, but again I don't have much time to do that and I do need QDMT in C, and quickly, so that will all have to wait for later.
<item name="NAME" label="LABEL">
Each main item defines a page of documentation; subitems may follow each main item. A subitem is defined as an item whose name has a period in it, so I may have an item "tangle" and a subitem "tangle.tangle_one". Subitems must follow their parents immediately or you will have some odd behavior. The documentation and code for a subitem is appended to its parent's page under an italicized heading, and a link anchor is created using the post-period part of its name, so that in the above example, the page "tangle.html" will have a link anchor <a name="tangle_one">.
The list of items is available to the index page in the [##itemlist##] tag; subitems are in a second level of list. Each item's list heading contains a link to its documentation page, so it's an effective table of contents. Eventually I'd like to do some more formatting of the table of contents, but that's pretty far down on my to-do list at the moment.
<piece variant="VARIANT" add-to="ADDTO">
Items define the documentation, and pieces define the code output. Each piece must be embedded in an item or it gets roundly ignored. (Yes, yes, a DTD would simply make that illegal but remember that the alpha version doesn't really understand XML.)
If a piece has a variant attribute, then it will only be included in objects for which it matches the variant spec. It will be filtered out of others. This will allow a single presentation to produce specific code for, say, Unix and Win32 variants of a single codebase. For finer control over variants, you can also embed a <variant> tag into a piece.
The other application for the variant system is documentation of porting projects. Both variants (even in widely different languages) can thus be tangled from the same presentation, while patterns shared by both may be shared explicitly. Again, once the feature is complete and I've tried it on a certain project I have in mind (involving Common LISP and Java, so we're at least talking about widely different languages) then we'll see how effective the whole idea is.
The add-to attribute is optional; if present, it causes the piece to be added, not to the currently active item, but to the item named. This makes it convenient for presenting cleanup code adjacent to allocation code, for instance. I'm still not entirely convinced it's a good idea, but I'm reserving judgment until I've tried it out in a couple of presentations.
<variant name="VARIANT">
When used within a <piece>, makes a part of the piece subject to suppression. So if the object I'm currently tangling is variant "unix" and I encounter a variant "win32" in the current piece, the contents of that tag will not appear in the object.
<insert name="NAME"/>
Used to do literate programming in the traditional sense, of course. The occurrence of the tag is replaced with the contents of the named item. This operation is recursive and is called tangling if you're not familiar with literate programming. Come to think of it, if you are familiar with literate programming, it's still called tangling.

Well, that's the lot. As I implement more stuff, if it impacts the element set at all, I'll update this page. I hope.