Tensile: A Tool for Literate Programming

I've begun work on a literate programming utility called "Tensile" which is based on the concepts and syntax used in NoWeb. It addresses what I see as some of the major shortcomings of the NoWeb system, which is otherwise excellent:

Tensile 1 has been released on 2010-06-04.

Since Tensile is written in Tensile, you'll need Tensile source which is already tangled into the source code to tangle the current version of Tensile into source code. Just like you need a C compiler to compile a C compiler written in C. Thus, the normal way of getting Tensile is to:

  1. Download the Tensile archive linked below, and extract it.
  2. Check out the Mercurial version, it's in Projects/Tensile.
  3. Use the bootstrap script in the archive to build the CVS Tensile: tensile -noweb-compat -unit tensile tensile.tnsl

You will then be able to reuse the new tensile script to create new versions when the Tensile source is updated.

A large part of Tensile is the ability to typeset the documentation and source code together using LaTeX. This is done using the tensile.sty file; place it where pdflatex or xelatex can find it. This file is included with the archive below and can be generated from the tensile.dtx file in Mercurial (assuming you also have my tcvfaces class) with xelatex.

Documentation | Bootstrap | LaTeX Package | Archive

Editor Support

The level of support in popular editors can make or break tool adoption, and this is one area in which literate programming has always faltered. Although the change in paradigm leads naturally to a change in the nature of the tools that we use to exploit it, the truth is that over many years of non-literate programming, everybody has gotten used to the tools they already use, and that constitutes a long term investment that we should not be quick to overturn. Thus, it's important that I make an attempt to support existing editors and technology when it comes to editing literate source code.

Syntax highlighting, as the Kate manual puts it, “greatly enhances the readability of the text, and thus helps the author to be more efficient and productive.” This presents are particular challenge for Tensile, due to the fact that Tensile encapsulates a wide variety of languages, which are usually supported individually by an editor. The trick is integrating these file types together in a single editing buffer. In some programs this is not hard: Vim, katepart (used by Kate), and gtksourceview (used by Gedit among others) allow syntax highlighting of one language to be included within the syntax highlighting groups of another language. However, other editors such as Emacs do not allow this; Emacs in fact presents a wonderful case study to the alternate, where due to the establishment of first-class major modes (of which only one may be in effect) and second class minor modes (of which any number may be in effect) there have be repeated and usually not very successful attempts to add such nested syntax support. This makes it very difficult to provide good support for Tensile in Emacs.

It is inherently a little difficult to decide what syntax style to use for source chunks, since often there is no way to figure out what kind of file they belong to unless you track them back through their dependencies to a unit which describes the entire file with a name like “IComparable.java” and this is virtually impossible to do in most editors. As a result, I came up with a way of describing the syntax of a unit by encoding it in the TeX which immediately precedes the unit. For example, let's say you are writing some Tcl code:

\section{Some File}

This is a file that does some stuff.

%% BEGIN ZONE: TCL

<<something.tcl>>=
proc foo {} {
    puts "Hello World"
}

The “%% BEGIN ZONE: TCL” line tells the editor support that the next source chunk should be rendered using Tcl syntax highlighting. Note that the line must be typed exactly like that, starting at column zero with the colon and space before the syntax name at the end. The syntaxes available depend on the editor support: in Vim we provide quite a lot whereas the Kate support is not quite there yet. But all editor support systems work in the same way to switch the syntax of source chunks.

Here are the levels of support for Tensile within various editors, and links to the appropriate files.