read


read
or learn more

The Marginalia Manifesto

Jan 5, 2011

source, official site, example output, guide

The importance of documentation

In late 2010 I was motivated by posts by Tom Preston-Werner, Reginald Braithwaite, and Luigi Montanez to create a Clojure clone of the excellent Docco documentation generator written in CoffeeScript. The byproduct of this motivation was Marginalia. The goal of Marginalia was to create a full-blown literate programming system1 with Docco-esque functionality as a way-station along the way.

No one will give a crap about your crap if your documentation is crap

– fogus

I have a very strong opinion regarding the importance of clear and complete documentation. My reaction to poorly documented code, products, and services is visceral to the point that I often refuse to release even the most humble library without code comments, examples, tests, invariant definitions, a logo, and an “official website”. However, time is not always on my side for personal projects, so I am constantly looking for ways to minimize the amount of work required to generate well-documented software without sacrificing quality. Marginalia is a step in that direction. However, I piddled around with a comment parser and created the code that builds a tree of code-lines associated with comment lines. Having scratched that particular itch I then took a detour into researching literate programming proper, leaving behind a half-baked mess.

Enter Zachary Kim

You may already know Zachary Kim as the creator of the excellent ClojureDocs website; an invaluable resource for finding Clojure API examples. Zachary resurrected my original source for Marginalia and single-handedly molded it into a tool worth using. Marginalia would be nothing without his vision and hard-work. I would also like to thank Justin Balthrop and Brenton Ashworth for their support and code contributions.

If you find Marginalia useful (or even just cool), then please do visit ClojureDocs and donate some of your time to enhance its existing examples and/or fill in the holes as you find them.

Current features

Before I dive into a diatribe, I’ll take a moment to go over some of the details of Marginalia. You can always see the current state of the output of Marginalia. A high-level features list currently stands at:

  • HTML generation from Clojure source
  • Markdown support within comments and docstrings
  • Latex formatting support (via MathJax)
  • Leiningen and Cake support

Usage

Currently Marginalia can be used in a number of ways as described below. In either case, the output document will be generated in $PWD/doc/uberdoc.html.

Command line

You can download the Marginalia 0.3.2 jar including packaged dependencies from Github.

Running Marginalia given the jar file linked above is as easy as:

java -jar marginalia-0.3.2-standalone.jar

This will search the $PWD for a src directory which it will then traverse looking for Clojure source files to parse and generate documentation for. Marginalia also takes specific locations and files to generate docs for:

java -jar marginalia-0.3.2-standalone.jar <files...>

Arguments can be specific files or directories.

Leiningen

To use Marginalia in your own projects simply add the following to your project.clj file in the :dev-dependencies section:

[marginalia "0.3.2"]

After executing lein deps you can generate your complete source documentation with the following command:

lein marg

More information regarding usage scenarios (e.g. Cake support, programmatic use, etc.) will be provided on the official Marginalia website.

TODO features

Marginalia is useable today, but more work is needed. At the moment the parser is based entirely on regular expressions and as a result might miss some documentation in code with non-standard formatting. Fixing the parser is the most glaring need, but other notable plans include:

  • Paragraph anchors
  • Smart code-parser
  • Modularized documents
  • Maven support
  • Alternative output formats
  • Special handling of :pre and :post
  • Plugins

Now that you know something about Marginalia, why should you want to use it?

A new way to think about programs

What if your code and its documentation were one and the same?

Much of the philosophy guiding literate programming is the realization of the answer to this question. However, if literate programming stands as a comprehensive programming methodology at one of end of the spectrum and no documentation stands as its antithesis, then Marginalia falls somewhere between. That is, you should always aim for comprehensive documentation, but the shortest path to a useful subset is the commented source code itself.

The art of Marginalia

If you’re fervently writing code that is heavily documented, then using Marginalia for your Clojure projects is as simple as running it on your codebase. However, if you’re unaccustomed to documenting your source, then the following guidelines2 will help you make the most out of Marginalia for true-power documentation:

  • Start by running Marginalia against your code
  • Cringe at the sad state of your code commentary
  • Add docstrings and code comments3 as appropriate (I find smallish functions with descriptive names and comprehensive docstrings to be ideal — very often docstrings will be longer than the code they describe)
  • Generate the documentation again
  • Read the resulting documentation
  • Make changes to code and documentation so that the “dialog” flows sensibly
  • Repeat the previous 3 steps until complete

Following the steps outlined will work to make your code not only easier to follow — it will make it better. The very process of using Marginalia will help4 to crystalize your understanding of problem and its solution(s). The quality of the prose in your documentation will often reflect the quality of the code itself thus highlighting problem areas. The elimination of problem areas will solidify your code and its accompanying prose. Marginalia provides a virtuous circle spiraling inward toward maximal code quality.

:F


  1. While a worthy goal initially, it has since been superseded by the excellent work by Tim Daly of which I hope to contribute ideas, code, and documentation. 

  2. These guidelines are a mixture of my own process and those outlined by Reginald Braithwaite regarding Docco. 

  3. A proliferation of regular comments (i.e. ; this kind) within the body of Clojure functions can be a “smell”. If you find yourself with a mess of comments, take a step back and see how the surrounding context can be broken out into separate functions, each with its own docstring. 

  4. Code documentation will help crystalize your understanding, but other factors are equally important in squaring the circle of software complexity. 

5 Comments, Comment or Ping

  1. I’ve been thinking that it’s high time that documentation became part of a formal language spec. We should move beyond ad hoc formatting in comments strings to documentation being first class members of the code itself. I haven’t given any thought as to how you might go about doing this, but Racket’s Scribble system seems like a step in the right direction.

  2. I have not looked deeply at Scribble, but I will do so at the first opportunity. Thanks for reading.

  3. I’ve just tried out Marginalia and I must say that is a stroke of pure genius. It produce truly gorgeous results. This is the way I’d like to see all projects documented! I’ve already made some patches and will surely contribute more to it.

    P.S.: One thing to note, the website for the project is really as ugly as the docs produced is beautiful! It made a very bad first impression on me, I nearly gave up trying it because I thought the docs would look like that!

  4. Ken Causey

    Consider updating the link in the ‘Current features’ section for the ‘output of Marginalia’. I never saw the original page but presumably it was something like http://fogus.me/fun/marginalia/ .

  5. Supersym

    Shrutarshi Basu, I so agree. Most often I find myself pulling my hair smacking my head against the wall. Software documentation really is poor in most cases. Also the other side of the angle, people writing 3, 4 pages before getting to the point. You’d be amazed how many people don’t even put up some simple installation notes to begin with … sigh

Reply to “The Marginalia Manifesto”