Send More Paramedics

λ λ λ

Clojure builds as an amalgamation of orthogonal parts

tools.build

The Clojure Core team recently released a new Clojure library, tools.build, that is a culmination of thought around batteries-included build support for Clojure projects. I won’t go into detail around the history and contents of the library in this post because much of that is found elsewhere, including the announcement post, the tools.build guide, and the tools.build API docs. Instead, I’ll walk through adding tools.build support to a simple project that currently uses Leiningen for building and talk a little about how tools.build goes about the same tasks in a different way.

The project that I’ll work with is a small personal project called reinen-vernunft and it’s build needs are appropriate for the gentle introduction herein.

Building is a matter of orthogonal parts

The batteries-included build story for Clojure is composed of an amalgam of complementary pieces, including: tools.deps with deps.edn, Clojure CLI, and tools.build. Therefore, enabling building in reinen-vernunft will require thinking about how these parts work in conjunction. However, to start let me show the existing project.clj file:

;; project.clj
(defproject fogus/reinen-vernunft "0.1.1-SNAPSHOT"
  :description "Code conversations in Clojure regarding the application 
                of pure search, reasoning, and query algorithms."
  :url "https://github.com/fogus/reinen-vernunft"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.11.0-alpha1"]
                 [org.clojure/core.unify "0.5.7"]
                 [evalive "1.1.0"]]
  :profiles {:dev {:dependencies [[datascript "1.2.2"]]}})

This is as bare-bones for a project file as you can create but there are some interesting things to understand if you want to explore how to perform the same tasks as Leiningen with tools.build.

Dependencies declaration

First, and likely most importantly is the :dependencies section of the project file. Clojure provides a way to similarly describe the same set of dependencies using the deps.edn format and indeed, the same set as follows:

;; deps.edn
:deps  {org.clojure/clojure    {:mvn/version "1.11.0-alpha1"}
        org.clojure/core.unify {:mvn/version "0.5.7"}
        evalive/evalive        {:mvn/version "1.1.0"}}

This is a subsection of the total deps.edn file posted out of context so to see how it fits into the structure you can look at the reinen-vernunft deps.edn file. However, you can see that the declaration of dependencies for Leiningen and Clojure is pretty close. The map-based version in deps.edn allows for different types of specifications be they artifact based, Git based, or local libraries but I won’t go into those details in this post but that information is available on the clojure.org site. One point of interest is that the dependency coordinate for the Evalive library was prefixed in the deps.edn case and not in the project.clj case. While both will allow unqualified library declarations for now, the tools.deps library will issue a warning should your own projects declare them as dependencies — rest assured, the author of Evalive has been sacked.

To find those listed dependencies, Leiningen looks in a few select locations by default: the local Maven repository, Maven Central, and Clojars to name the most popular options. The tools.deps library also looks in these places and will download the artifacts into the local Maven repository as expected. Finally, local source is a dependency also and Leiningen looks in the src directory by default and so does tools.deps, but my personal preference is to declare the source path explicitly — YMMV:

;; deps.edn
:paths ["src"]

Building

Now that I have dependencies in place I’d like to build an artifact of my own for reinen-vernunft, specifically a jar file containing the Clojure source files for the project. First, I’d like to specify a build alias in the deps.edn file that pulls in the tools.build library as a dependency in the :aliases map as such:

;; deps.edn
:build {:deps {io.github.clojure/tools.build 
               {:git/tag "v0.1.3" :git/sha "688245e"}}
        :ns-default build}

This is a Git based dependency scoped under the :build alias that points to a specific Git repository tag and short SHA. However, now that I’ve pulled in that dependency how do I do anything? The tools.build library provides a set of functions and utilities that allow builds to be described as code. Indeed, a file named build.clj serves as this program and starts by pulling in the tools.build api:

;; build.clj
(ns build
  (:require [clojure.tools.build.api :as b]))

Where Leiningen’s project.clj declares its configuration parameters as syntax in the defproject form, tools.build parameters are just vars in code:

;; build.clj
(def lib 'fogus/reinen-vernunft)
(def version (format "0.1.%s" (b/git-count-revs nil)))
(def target-dir "target")
(def class-dir (str target-dir "/" "classes"))
(def jar-file (format "%s/%s-%s.jar" target-dir (name lib) version))
(def src ["src/clj"])
(def basis (b/create-basis {:project "deps.edn"}))

These vars describe various things, including version numbers built from calculated Git revisions, class target paths, Jar file names, and useful build configuration. To create a build target function in the build.clj file is as simple as writing a function, in this case clean that takes a map argument (although ignored in this case), that calls out to the tools.build API task functions:

;; build.clj
(defn clean
  "Delete the build target directory"
  [_]
  (println (str "Cleaning " target-dir))
  (b/delete {:path target-dir}))

This clean target is ready to run using the Clojure CLI by issuing the following command at your command prompt:

$ clj -T:build clean
Cleaning target

While not earth-shattering, the clean target is useful when you’re working on a project and need to clean existing artifacts before building them anew. Indeed, one such artifact is a Jar file that for reinen-vernunft means an archive of the name specified by the jar-file var and containing the source specified with the src var. A jar target would need to do the following tasks:

  • Create a pom file
  • Copy source to an intermediate location
  • Archive the contents of the intermediate location

The implementation is as follows:

;; build.clj
(defn jar
  "Create the jar from a source pom and source files"
  [_]
  (b/write-pom {:class-dir class-dir
                :lib lib
                :version version
                :src-pom "pom.xml"
                :basis basis
                :src-dirs src})
  (b/copy-dir {:src-dirs src
               :target-dir class-dir})
  (b/jar {:class-dir class-dir
          :jar-file jar-file}))

This jar target function is fairly straight-forward in that it: 1) writes a pom to target dir, 2) copies src files target dir, and 3) archives these files into a JAR file. One interesting aspect of the jar target is that it uses a source pom as the base for the new pom. This is the prefered way to seed a pom with metadata about a project that in Leiningen often stands as defproject parameters. Specifically, the :description and :license fields in the project.clj file shown in the beginning of this post become XML elements in the source pom.xml fed into the b/write-pom task:

;; pom.xml
  <description>Code conversations in Clojure regarding the application 
  of pure search, reasoning, and query algorithms!</description>
  <licenses>
    <license>
      <name>Eclipse Public License</name>
      <url>http://www.eclipse.org/legal/epl-v10.html</url>
    </license>
  </licenses>

There are hundreds of items that could go into a pom and so rather than offer a subset (or worse, all) as parameters on b/write-pom the tools.build library uses the source pom seed to create a new pom in the :class-dir directory.

Running the follow will create the jar file in the target directory:

 $ clj -T:build jar

Testing

While testing is technically outside of the purview of tools.build, the fact is that it’s an important part of most programmers’ dev cycle. From the beginning, Leiningen supported automated testing via a close integration with clojure.test. On the other hand, the Clojure CLI is agnostic to testing tools but instead allows a generic way to execute Clojure functions via the -X flag. To enable testing one should wire a test runner into their deps.edn file and create an alias for execution via the CLI. The reinen-vernunft library’s deps.edn has the following :test alias defined:

;; deps.edn
:test {:extra-paths ["test"]
       :extra-deps {io.github.cognitect-labs/test-runner 
                     {:git/tag "v0.4.0" :git/sha "334f2e2"}}
       :main-opts ["-m" "cognitect.test-runner"]
       :exec-fn cognitect.test-runner.api/test}

This sets up the :test alias to call out to the test-runner which looks for tests in a certain place of a certain filename and executes them with clojure.test. This is done with the following command:

$ clj -X:test
Could not locate datascript/core__init.class, datascript/core.clj...

Whoops! As it turns out the library has a test dependency on the Datascript library that in a project.clj file is declared in a :dev alias in the :profiles map. That same dependency can reside under the aforementioned :test alias in deps.edn but for my purposes I decided to create another alias named :dev that declares that dependency:

;; deps.edn
:dev {:extra-deps {datascript/datascript {:mvn/version "1.2.2"}}}

And the test execution is initiated with the following command:

$ clj -X:dev:test
Running tests in #{"test"}
...
0 failures, 0 errors.

And that’s it. Once again, you can view the whole reinen-vernunft deps.edn file and the build.clj file to view everything in context.

Summary

Clojure’s latest features in tools.build, tools.deps, and the Clojure CLI) work to define a orthogonal set of tools that work together to form the basis for a batteries-included story for Clojure builds. Each part is a powerful tool in its own right but the amalgamation forms a powerful way to express the build needs of your own projects. In the future I hope to expand on this post with some other interesting features available to Clojure programmers but in the meantime consider exploring this toolset yourself.

Clojure Core

Earlier today Rich Hickey let the cat out of the bag that I’m joining the Clojure Core development team.

This is a change from my previous relationship with the Clojure Core team that in the past few years has been sporadic at best. This circumstance arose through necessity rather than desire and so I’m overjoyed to work closely with the team once again. Over the past few years Alex Miller has exerted a Herculean effort in shepherding the language through numerous versions and steady growth. I’m overjoyed to be given the opportunity to join him in helping push the language forward, and I’ll do my very best to avoid breaking everything.

So what this means is that I’ll be far more visible1 in and around the Clojure ecosystem while spending my time thinking about Clojure more so than just using Clojure.2


  1. The range of online sources of Clojure-related chatter is quite daunting and my immediate goal is to establish some sort of tempo for interacting therein. 

  2. I’m certainly willing to help with both ClojureScript and Clojure-CLR, but those teams are rock solid and so my best move is probably just to stay out of the way. 

The best things and stuff of 2020

Great things and people that I discovered, learned, read, met, etc. in 2020. No particular ordering is implied. Not everything is new.

also: see the lists from 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011 and 2010

Great blog-posts / articles read

  • The Octopus: An Alien Among Usfor biology-curious goofs1 such as myself, the octopus is a fascinating example of aliens among us. no chimera was half as interesting.
  • A Conversation with Arthur Whitneycomputing luminary and designer of the A+, k, and q languages as well as an early contributor to J speaks about his life and contributions to computing and also that he never writes buggy code.
  • The True Glamour of Clarice Lispectora portrait of a brilliant author who’s characters lived and experienced life as the author experienced it. her body of work is a record of her life.
  • Mark I FORTH Computera home-brewed forth computer using discrete components.
  • Art Bell and the Eerie Joy of Late Night Radioas a college student i would often spend my nights slinging code, working proofs, and writing essays long into the night. my background sounds of choice were the dulcet tones of Art Bell, the original host of Coast to Coast AM and lover of all things weird. while most of the content of his shows were far-fetched at best, i couldn’t help but marvel at the possibilities of a world that was much more interesting than the one in which i lived.
  • How is the Linux Kernel Tested?from linus’ gut to static analysis to automated testing to continuous integration to people running commands.
  • The Rise and Fall of Commercial SmalltalkAllen Wirfs-Brock responds to Gilad Bracha’s perspective on the subject, providing his own perspective in the process. interestingly what Smalltalk brought to computing was probably more important than Smalltalk itself.
  • A Constructive Look at TempleOSTempleOS is a legend in the weird world of hobby osdev and its developer Terry Davis (RIP) was as prolific as he was controversial (i.e. massively). the operating system is an altar (in more ways than one) to eccentricity and was developed with a singular focus on speed and direct hardware access. 2
  • Cemetery of Soviet Computersa retrocomputing bonanza of defunct Soviet computer systems found in a dilapidated building somewhere in Russia.
  • How SHRDLU Got Its NameTerry Winograd explains how his blocks world program SHRDLU got its name. 3
  • AI Ruined Chess, Now It’s Making the Game Beautiful Againhow AlphaZero is used to explore odd variants of Chess, finding new patterns and in some cases reducing draws dramatically.
  • Rewriting the Technical Interviewa work of beautiful madness.
  • thought leaders and chicken sexers by Zach Tellman – an essay that takes on the topic of Paul Graham head on, especially in regard to his programming language experiment Arc. computerists of a certain vintage have a complex relationship with pg in that many of his early, mostly pseudo-technical writings were inspirational. however, over time his focus has changed to that of concerns of finance and thought-leadership thus leaving many who admired his technicalish essays alienated. Zach captures the spirit of that group well in both the content of the article and its subtext.
  • I Could Do That in a Weekendif you’ve spent any time at all on the internet then you’ve likely heard the phrase “i could do that in a weekend” — the battle cry of the dickweed. Dan Luu talks about the difference between happy-paths and real work required to make modern software systems.

Most viewed blog posts by me

I’ve been trying something new over the past couple of years. That is, I’ve been posting threads and such on my Twitter with a small handful of posts here That said, there were a few high-traffic posts on my blog.

  • ToriLisp – an ersatz LISP for little birdsmy final post of 2020 turned out to be my most popular. this one described a little programming language that I created over the course of the year and the motivations behind it.
  • Six Works of Computer Science Fictionan older post that discusses computer science books describing systems that couldn’t possibly be real — except that they are!
  • Walking the Clojure Source History (a talk not given)some images of notebook pages with notes about the commit history of Clojure during its early days. the notes were in service to a conference talk that i never gave.
  • 8-bit Spiritualsa short discussion about old 8-bit programs and the gestalt of the small, yet feature-rich, in programming and some modern examples that fit this mold.4

Favorite technical books discovered (and read)

I’ve intentionally reduced the number of technical books that I consume, but there are a few that I “found” in 2020 that stood out.

  • UNIX: A History and a Memoir by Brian Kernigan – this was much better than i thought it would be. regardless of how you view UNIX the anecdotes and stories are worth reading.
  • Introduction to Very Large Scale Integration Systems by Carver Mead and Lynn Conway – this book might have motivated the spread of Alan Kay’s famous quote in across the general computer landscape — ‘People who are really serious about software should make their own hardware’ — but it didn’t. certainly Conway and Mead’s seminal work is deeply influential but even still most industry folks don’t understand VLSI except superficially, myself included.
  • Lucid, the Dataflow Programming Language by William Wadge – if enjoy twisting the langdev center of your brain into a pretzel then Lucid might be your thing. the book builds to the main language from subset languages, each teaching core principles. this is a new addition to my personal CS book pantheon.
  • Footsteps In An Empty Valley issue 3 by Chen-Hanson Ting – a book that tries to make a parallel between Zen and Forth. i can’t say if the two did meet but it was an entertaining journey nonetheless.
  • Beneath Apple DOS by Don Worth – while trying to squeeze out as much performance from his Apple ][ to run his game Beneath Apple Manor, Don Worth explore the darkest recesses of his computer to find every speed hack that he could use. his findings were published in this book.

Favorite non-technical books read

  • Understanding Media by Marshall McLuhan – this book has been… a challenge… to read so far, but so far I’ve found the effort worthwhile. McLuhan died in 1980, but seems to have been talking about us.
  • A Confederacy of Dunces by John Kennedy Toole – i had this book all wrong. you see, before 2020 i viewed it from a position of pure ignorance, tainted by prejudices. you see, in the past some of the worst people that i have known loved this book so naturally i made an association with it. couple these biases with the fact that a gloss of the story — three intermingled storylines populated by utterly broken people with the most broken of all taking center stage — seemed not worth my time. all of this said, i eventually broke down and decided to try the first chapter. well, the first motivated the second and it in turn motivated the third and so on until finally i was done. and wow was it a great book. it’s the kind of book that absolutely deserves its accolades and despite finding its place on the pedestals of the the Ignatius Reillys of the world, is a must read by lovers of literature.
  • Hitler: Ascent, 1889-1939 by Volker Ullrich – i’ve read my share of books on the subject but what i found especially compelling about this take was its focus on most of the points along the ascent where the powers of the time could have tamped out the eventual greatest evil but refused.
  • A Classical Education: The Stuff You Wish You’d Been Taught at School by Caroline Taggart – i was [this] close to attending St. Johns College where a classical education is compulsory but since i didn’t i’ve often wondered what i missed out on. while this book is no substitution for a classical education, it does provide a nice outline for diving deeper should one be compelled to do so.
  • Wise Blood by Flannery O’Connor – like ACoDunces mentioned above, my prejudices prevented me from reading Wise Blood for far longer than I should have. however, in the case of the O’Connor classic it was the negative view by people whom i admire that kept me away. again an opportunity to buck my prejudices presented itself and as a result a pantheonic book5 was found. the book masquerades as a morality tale but lurking beneath is a nihilistic romp that’s kept me thinking for weeks after reading it. it’s stuck with me and i can’t foresee it leaving my mind any time soon.

Number of books published

0

Number of books written

0

Number of books read

lots

Favorite musicians discovered

  • Vlora nice layering of haunting guitar and persistent drones. it’s always nice to discover new “headspace music.”
  • Shakulattea blend of traditional Japanese skakuhachi music with some blues and jazz for good measure. full disclosure: Adam is a very good friend of mine but i’ve always loved his music and would listen even if i didn’t know him.
  • Dirty Knobsspecifically the “Field Recordings From The Edge Of Hell” album which is atmospheric and brooding.

Favorite show about an old dude riding trains and talking and looking at stuff

British Railway Journeys with Robert Symes

Favorite films/shows discovered

  • The Lighthousea fantastic freaking film. it’s horror in that it’s somewhat tangential to a Lovecraftian experience all the while skirting the line between myth and time. on the surface the film tells a tale of loneliness, frustration, alcoholism, and the slow slide into madness but there’s a lot more going on than meets the eye.
  • Fortitudequite grim in terms of exposing the dark side of the human condition but I was in the mood for that. you have to be. It has some truly memorable characters.

Favorite games discovered

  • 18Chesapeakefor fans of 18XX games this is train-game comfort food. however, the reason that it makes the list is that if you’ve ever been (or become) curious about the 18XX genre of games then this is a very good entry point into the sub-genre.
  • Fox in the Forest Dueta chill game where you and a friend or loved one cooperate to traverse a forest path using trick-taking as a way to proceed.
  • Marvel Champions: The Card Gamenormally i wouldn’t be drawn to license games but 2020 was a weird year for gaming and so here i am. Marvel Champions is an interesting way to spend some alone time building decks, solving scenarios, and starting over and trying again. there are many expansions to the game so the possibilities moving forward will expand. i will probably keep playing even after quarantine-times end.
  • Nokosu Dicethe best trick-taking game that i’ve found in years. combines card play and dice drafting to make for a truly unique experience.
  • Top 10 Games You Can Play in Your Head, by Yourself by Theophrastus J. Bartholomew- more a set of thought exercises rather than a game per se but i found myself occupied by the ideas in the book from time to time during those essential unguarded moments.

Favorite programming languages (or related) I hacked on/with

  • Modern C++i wrote c++ for a time around 9/11 but have been out of that loop for many years. in the meantime the language has evolved to include interesting features. however, the real strength in modern c++ comes from the unbelievable boost library and exploring that angle proved to be an interesting journey. am i ready, willing, and able to jump back into c++ again — the answer to all three of these questions is a definitive NO but i enjoyed the refresher.
  • Forthi’ve always been fascinated by Forth but the fact is that there are so many variants that diving in felt daunting. however, thanks to Chen-Hanson Ting’s books i latched onto figForth as it’s designed for portability and simple enough to understand its implementation.

Programming languages used for work-related projects

  • Clojure2021 marks the 11th year6 as a full-time Clojure programmer.
  • ClojureScript
  • DatalogThe Datomic flavor of Datalog is the flavor of choice for database access, be it in-process, in the cloud, or even in the browser.

Programming languages (and related) that I hope to explore more deeply

  • Forthi’d love to dig deeper into SectorForth but figForth and/or eForth is a big draw for me.
  • Prologi may pull the post-functional programming thread to see where it leads me.
  • Common Lispthere’s so much to explore in Common Lisp that I could spend the rest of my days studying it deeply. i’ll start with one-year.

Favorite papers discovered (and read)

Here are a few that I enjoyed in 2020.

  • Systems Guide to figForth by Chen-Hanson Ting – the kernel of the figForth language is tight and a joy to read about. Ting’s paper is a nice place to start learning more.
  • Lucid – A Nonprocedural Language with Iteration by Ashcroft & Wadge – Lucid is one of those many interesting languages that emerged during the wild-west 1980s. the books explores the language and its interesting patterns of usage and these lessons are mind-bending for those of us raised on chaff.
  • JOSS – Introduction to the System Implementation by Bryan – JOSS was a pre-BASIC language that went on to influence many of the languages along its somewhat withered branch on the language tree.

Still haven’t read…

Snow Crash, A Fire upon the Deep, Don Quixote, The Contortionists Handbook and a boat-load of sci-fi

Favorite/Only technical conference attended

None

Favorite code read

  • Ludii AIthe Ludii general game system is a program that designs tabletop abstract strategy games. there’s an interesting mini-language around game descriptions and its weights and scales for measuring fitness are a lot of fun to dig into.
  • Caliperi just have a thing for units of measure conversion programs.
  • SectorForthtouches both of my osdev and langdev proclivities — a near-perfect project.
  • Ring Lispan interesting take on Lisp in that it uses a ring buffer for cons allocation instead of a garbage collector. this encourages tail-call style. the implementation is nice and clean.
  • Awficethis is the kind of project that i love exploring. the author took an idea — build a minimal office suite that works in the browser — and ran with it. every implementation works as a data url.

Life-changing technology “discovered”

  • Air fryer — I just can’t go back to frying stuff in puddles of grease.7

State of plans from 2020

  • Explore the depths of the train game genreCOVID put a kibosh on any group gaming plans in 2020.
  • Rethink and reorganize my website. — This is slow going but it is making progress.
  • At least one installment of Read-Eval-Print-λove in 2020 — Despite a lot of quarantine time I just couldn’t motivate myself to do this.
  • Resuscitate my old Clojure projects. — My pods project has been the main focus with examples and documentation filling a fair amount of time.
  • Write a paperAgain, writing has not been a primary motivation for me in 2020 so this didn’t get much further than some outline tweaks.
  • Explore more deeply the fields of Augmented Intelligence and Cybernetics – The reading lists for these fields are inexhaustible but I did manage to make a scratch on them in 2020. Much more exploration needed here.
  • Explore vintage computing systems. – I managed to get my TRS-80 Model 100 talking to a Pi that runs Clojurescript, so I’d call that a success.

Plans for 2021

  • Spend more time working on Clojure and ClojureScriptit’s been a while since i last contributed directly to Clojure and ClojureScript but 2021 offers a unique opportunity for me to jump feet first back into the guts of the language(s) and their supporting libraries.
  • Dive deeper into microcontroller programming, especially with the Teensymy son is a member of a rocketry club and is tasked with the microcontroller side of a project. he and i have had a blast learning about the target system and exploring the guts.
  • Add another entry to my personal programming languages zoo2020 saw the addition of ToriLisp so who knows what 2021 may bring.
  • Read more philosophy and mathematicsi started my academic life double majoring in math and philosophy but eventually transitioned into CS as I viewed it as a way to mix the two. unfortunately abandoning both means that what little knowledge i have had a sell-by date somewhere around WWII.
  • Get more involved with the local Old School MtG scenei played Magic: the Gathering waaaaaay back when it started and still have my cards from back in the day. it seems that old school is the only way to resurrect them.
  • Return to civilization.if the universe allows.

2021 Tech Radar

  • try: Dart
  • adopt: “Problem statements”
  • assess: Babashka
  • hold: “Modern” C++
  • stop: Rails

People who inspired me in 2020 (in no particular order)

Yuki, Keita, Shota, Craig Andera, Carin Meier, Justin Gehtland, Rich Hickey, Jenn Schiffer, Nick Bentley, Paula Gearon, Zeeshan Lakhani, Brian Goetz, David Nolen, Jeb Beich, Paul Greenhill, Kristin Looney, Andy Looney, Kurt Christensen, Chas Emerick, Stacey Abrams, Paul deGrandis, Nada Amin, Alvaro Videla, Slava Pestov, Yoko Harada, Mike Fikes, Dan De Aguiar, Christian Romney, Russ Olsen, Alex Miller, Alan Kay, Alan Watts, Elizabeth Warren, Warren Ellis, Naoko Higashide, Zach Tellman, Nate Prawdzik, Tim Good, Tim Ewald, Stu Halloway, and Michael Berstein.

Onward to 2021!

:F


  1. “biology-curious” meant to convey that I’m far below even an amateur in the field. 

  2. I came across Terry Davis many years ago when TempleOS was known as Losethos. He was an interesting person to correspond with as at the time he would often respond with incredibly long paragraphs of Markovian nonsense generated by a program he called “Word of God.” 

  3. Blocks World was a project that i did as one of my undergraduate independent studies. over the years i’ve gone back to this code writing and rewriting its guts. it’s a challenging task but not world-shattering and i find working on it better than jigsaw puzzles. 

  4. i’d like to continue pulling on this thread but it seems that the love of this ideal is not as widespread as i had hoped. 

  5. a number of things have entered my personal pantheon this year which I suppose is a shining positive in this otherwise retched year: Lucid, A Confederacy of Dunces, Wise Blood, The Lighthouse, Ludii AI, and figForth

  6. This is strictly my work-life time. My total use of Clojure has been longer. 

  7. that’s not entirely true as good tempura is one of my favorite foodstuffs in the universe. 

ToriLisp – an ersatz LISP for tiny birds

When deciding to work on a side-project three factors are needed to transition from fancy to application: goal, motivation, and time.

Time is usually the biggest sticking point for me personally but with COVID most of what I may have spent my time on this year was cancelled. However, motivation was still a huge sticking point until I came across a couple of projects that helped propel me forward. First, I spent some time earlier this year combing over Mary Rose Cook‘s lovely Little Lisp interpreter code. Given what I knew about Mary’s previous projects it was no surprise that the Little Lisp implementation was simple and elegant. However, what I wasn’t prepared for was that hacking on the interpreter turned out to be straight-forward and addictive. However, it wasn’t until I re-discovered William Taysom‘s old Scheme-like language Misp that I had a form for the interpreter in mind. At the time of William’s original blog posts about Misp I was drawn to his passion and enjoyed the implementations of the language that he posted.1 Around the same time I found Paul Graham’s original Arc tutorial tut.txt and used it extensively to guide me in what to implement next.2 All discussion about Arc aside, I definitely appreciate the clarity and layout of the Arc tutorial and found it a great outline for a little language.

Finally, as some of you might know I dabbled in functional programming in JavaScript and even went so far as to create a couple of libraries pushing the fringe of fp in js; namely Lemonad and underscore-contrib. Some of the ideas in these libraries found their way into my own interpreter which ultimately pushed my code away from Misp, Little Lisp, and Arc into …something else — that something else I’m calling Tori Lisp — an ersatz LISP for little birds.

ToriLisp = Litle Lisp + Misp + tut.txt + Lemonad + underscore-contrib + a pinch of CLJ

Here’s a very small sample of the language:

鳥>  (let (x 3 y 4)
       (+ (* x 2) (* y 2)))
;;=> 14

(def map
  (λ (fn list)
    (if (no list)
      list
      (cons (fn (first list))
            (map fn (rest list))))))

鳥>  (map (+ 10) '(1 2 3))
;;=> [ 11, 12, 13 ]

鳥>  ({x y | (/ (+ x y) 2)} 2 4)
;;=> 3

鳥>  (len "abc")
;;=> 3
鳥>  (len {a | a})
;;=> 1
鳥>  (len +)
;;=> 2

鳥>  (read "(push [1] 'Z)")
;;=> [ "'push", [ "'list", 1 ], [ "'quote", "'Z" ] ]
鳥>  (eval (read "(push [1] 'Z)"))
;;=> [ "'Z", 1 ]  

If I wanted to present a list of features then the following list would work:3

  • Functional
  • Immutable access to JavaScript arrays and hash-maps.
  • Function auto-currying
  • Function introspection
  • Bottoms-out at JavaScript structures and functions
  • ML-like Refs
  • Lightweight syntax for common language forms

If you’re interested in checking out the language then the ToriLisp Github repository has a quick start, test suite, and its own tut.txt.


  1. Though the implementations did not make the Internet Archive it seems. I reached out to Mr. Taysom years ago and he was kind enough to send me the code but I hesitate to share it publicly without his approval. 

  2. Consider this a form of README-driven language development

  3. Currently ToriLisp doesn’t have macros though not because it would have been terribly difficult to add. Instead, I wanted to start with functional literals and auto-currying and see how far it could take me. I may add them at a later date but only time will tell. 

8-bit Spirituals

An 8-bit Spiritual (8BS) is a modern program that hearkens back to a time when applications provided a robust experience but fit onto a single floppy disk. Modern apps are bloated by comparison, but there are applications that fit the spirit of the 8-bit days.

That said, a modern 8BS should provide a minimal set of modern features: unicode support and Internet and/or WWW support (where appropriate) seem like a minimal set.

I don’t want to limit the size of the eventual binary or linked libs because limiting to 8-bit era floppy sizes is not necessary at this point. I think modern-system-relative “small” is acceptable. Also, 8BSes are programming language agnostic and both VM-bound and direct hardware applications are fair game. 8BS is a state of mind so to speak.

For now, an example of the kinds of applications that I would be interested in WRT 8-bit Spirituals are as follows…

Programs of Yore

Before I point out a few examples of modern programs that fit the 8BS mold I’d like to take a moment to talk about a few interesting examples. Certainly, ANY program from the 8-bit era fits the literal definition of a constrained environment program by default but only a small subset could be representational of the type of program that I’m after for this post. The programs below were all constrained — but they were never constraining.

KAMAS

KAMAS was ground-breaking outliner software originally designed for the Kaypro II (eventually finding its way onto other systems) that provided a full-fledged editor, BBS system, and internal programming language interpreter — all on a 191K floppy disk.

I’m somewhat of a fan of the KAMAS software, even though I’m only able to run a truncated 1 version on the DOSBox emulator. Even minus the more advanced features found on the Kaypro versions, KAMAS for DOS is a solid application; however with those advanced features the program was a powerful digital workspace.

Electric Pencil

Electric Pencil was the first program to implement word processing-like features and ran on the MITS Altair on 8K of memory.

While certainly not at the level of many word processors that came after it, Electric Pencil set the pace for many of its successors. Indeed, many retrocomputerists consider the now defunct Word 5.1 for the Macintosh the pinnacle of word processing.

I was never fortunate enough to use Word 5.1 but it certainly seems to shine as an example of an application that an 8BS should shoot for.

Turbo Pascal

Turbo Pascal was originally released on the 8-bit CP/M OS and contained an IDE and compiler that could run programs in memory or compile them blazingly fast — all for $35.2

Pascal was a great language to the 8-bit era and amenable to fast compilation but the Turbo Pascal software is famous for being a shining example of Anders Hejlsberg’s programming prowess.3

Modern 8BSes

Having shown a few examples that act as goalposts for the modern 8BSes, I’ll offer a few examples of modern programs that hit the mark set in the beginning of this post.

Redis

Something like pre-clustering Redis is a decent candidate. Certainly it’s not a floppy-sized program but it did a lot given its small footprint. While it’s grown in scope over time, the original vision was a sleek data-structures database that performed well, was quite stable, and was even easy to understand.

Frink

Frink is a programming language unlike any other. It is generally known for its ability to handle and convert between a bevy of units of measure consistently throughout calculations. For anyone who has created a complex system (or even a simple one for that matter) that needed to juggle varying units of measure, the complexity of the problem that Frink handles seamlessly is staggering.

// Levenshtein distance is a built-in
println[editDistance["kitten","sitting"]]
//=> 3

// Convert feet to meters
38 feet -> meters
//=> 7239/625 (exactly 11.5824)

Frink requires a JVM, which is as far from an 8BS as you can get. However, VM aside “the Frink programming language including all its reference data, GUI, and graphics, can ship in less than 500 kilobytes (pack200 compression).”4

Dwarf Fortress

Dwarf Fortress is another interesting candidate.

It’s a world-builder / rougelike / simulation game that is obsessive in its approach to detail — despite its textual graphics. The game has maintained a strict vision of providing strong-simulative gameplay by its designers Tarn and Zach Adams.

PuTTY

The final program that I’ll nominate is the workhorse terminal emulator PuTTY. After 20+ years of development, PuTTY is still a 3 MB install and provides a useful set of functionality. When you need the functionality that PuTTY provides you REALLY need it to run well and right away. PuTTY has been meeting this need year after year and has been rock-solid all along the way.

8BS systems

Before I end I should point out the existence of hardware 8BSes. I suspect that the Pi, Arduino, ESP32, etc. communities have a ton of examples of viable 8-bit Spirituals to learn from. More exploration needed on my part and perhaps a future post will tackle them in more detail — though pointers and info about them is always welcomed in the comments below.

Finis

That’s all that I have for now. The 8-bit era has passed us by and is not likely to ever return in the realm of the general-purpose user-centric application. However, the spirit of the 8-bit system is still something that we can strive for and sing the praises of. The question is of course — should we?

:F


  1. If I could find a workable copy of the original Kaypro II disks then I would immediately grab an old dusty system to run it on. Alas, the KAMAS software has turned out more difficult to find than the computer to run it on. 

  2. It was eventually sold for $35 but was released for $49.99. 

  3. Pascal of the time could be scanned, parsed, optimazed, and code-gen’d in a single compiler pass. 

  4. As explained to me by Alan Eliasen – Frink’s creator.