Send More Paramedics

Programming language development: the past 5 years

Oct 18, 2011

I recently compiled a list of Perlis Languages that were meant to illustrate varying paradigms not typically found in mainstream programming languages. Those languages were meant to stretch your mind and are not necessarily representative of the state of the art in language design and implementation. In this post I will provide a list of fairly new languages (let’s say 5 years with a little flex) that display¹ interesting features and display higher-order thinking in the way that they tend toward an evolution of past learnings in programming language thinking. Not all of these languages enjoy active development, but the more important point is that they represent in some way “new language thinking”. Remember that this goal does not necessarily mean “innovative”.

Unlike my Perlis post, I have not fully explored all of the languages below, so caveat emptor. Please correct me if I misrepresent something. No meaning behind the ordering implied.

take the associated poll!

Shen

released: 2011, author: Dr. Mark Tarver

I included the fantastic language Qi in my Perlis Languages post, so it may seem gratuitous to include its successor Shen here. In fact Shen and Qi offer most of (all of?) the same high-level features:

• A Lisp
• Optional static typing
• Pattern matching with guards
• Macros
• Partial application
• Function backtracking
• Builtin Prolog
• Builtin compiler compiler

However, Shen is indeed the next evolutionary step after Qi. Shen builds on the ideas in Qi in various ways, but the primary motivating force is targetability. What exactly does that mean? The precursor Qi was built to target Common Lisp as its host language and did so to maximum effect. However, Qi was found to target a very small subset of Common Lisp. Therefore, Dr. Tarver devised the idea that the successor Shen should be defined in terms of a minimal kernel Lisp language called Kl which would, in theory, provide an easier port target across various runtime hosts, including, but not limited to: JavaScript, Python, Clojure, and Common Lisp. I have been thinking of “kernel Lisps” a lot and so Shen is ripe with ideas.

An example of a member function using the embedded Prolog:

(defprolog member
  X [X | _] <--;
  X [_ | Y] <-- (member X Y);)

(prolog? (member 1 [1 2 3]))
/*=> true */

Partial application is automatic (simplified below):

(* 2)
/*=> #<FUNCTION LAMBDA (x) (* 2 x)>

((* 2) 54)
/*=> 108 */

Here is a function that calculates the nth triangle number:

(define triangle
  0 -> 0
  N -> (+ N (triangle (- N 1))))

(triangle 100)
/*=> 5050 */

And a typed version of the same:

(tc +)  /* turn on type-checking */

(define triangle
  {number --> number}
  0 -> 0
  N -> (+ N (triangle (- N 1))))

(triangle 5)
/*=> 15 : number */

(triangle a)    
/* type error */

Shen is a natural evolutionary step along the long, winding path that is Lisp history. Many modern programming languages are absorbing features that Lisp innovated long ago, but still the most exciting languages are Lisps.

More resources

• Official standard
• Source
• Shen in 15 minutes

Agda 2

released: 2009, author: Ulf Norel

I’ve yet to truly wrap my head around Agda (any year now), but I can say a few things about it. First, Agda is a purely functional, pattern matching, dependently typed programming language that walks the thin border adjacent to proof assistants. In dependently typed languages, type expressions can contain (depend on) a program expression. Therefore, the resolution of type constraints is a function of code or predicated on values. It’s like a kick in the face. As an added bonus, the Agda type language is the same as the value language (i.e. the programming language itself). What this means is that type invariants in Agda can express a much wider spectrum of constraints than typically attributed to static type systems; for example, a list type that statically guarantees sorted order (PDF). There’s a drop-kick for you.

Here is an encoding of even and odd numbers in the Agda type system:

data Nat : Set where
  zero : Nat
  suc  : Nat -> Nat

fortyTwo : Nat
fortyTwo = 42

plus : Nat -> Nat -> Nat
plus  zero   m = m
plus (suc n) m = suc (plus n m)

mutual
  even : Nat -> Bool
  even zero    = true
  even (suc n) = odd n

  odd : Nat -> Bool
  odd zero    = false
  odd (suc n) = even n

The previous code defines two datatypes: 1) the natural numbers and 2) the even natural numbers. You can also define type functions as infix operators:

_+_ : Nat -> Nat -> Nat
zero  + m = m
suc n + m = suc (n + m)

This represents the extent of my understanding about Agda so far. While it would be nice to know more, what I’ve seen is amazing enough to warrant inclusion.

More resources

• The Agda Wiki
• Intro sources
• Papers using Agda
• Source

Ioke

released: 2008, author: Ola Bini

Ola Bini’s Ioke is based on a simple question: what kind of language can you build if you completely disregard performance and instead focus on expressivity?² As it turns out you gain incredible expressiveness as shown in Bini’s presentation series (links below). One of the more interesting aspects of Ioke is that it is a homoiconic language providing macros.

myfor = dsyntax(
"takes a name, an enumerable, and a transforming expr
and returns the result of transforming each entry in 
expression, with the current value of the enumerable
bound to the name given as the first argument",

  [argName, enumerable, argCode]

  ''(`enumerable map(`argName, `argCode))
)

myfor(x, 1..5, x*2) 
;=> [2,4,6,8,10]

Another advantage in studying Ioke is my own personal first axiom:

When brilliant people create things, study them.

As programmers the onus is on us to push our skills to the point of uncomfortableness. Studying the works of great minds is a highly effective way to do this.

More resources

• Ioke
• Ioke: a Folding Language (video)
• Ioke Wiki
• Announcement
• Source
• Macro types in Ioke – or: What is a dmacro?

Pure

released: 2008, author: Albert Gräf

Pure is a functional language built around term rewriting. Term rewriting is very similar to what we did in high-school algebra using the FOIL method:

(x1 + y1) * (x2 + y2) = 
  (x1 * x2) +            // First
  (x1 * y2) +            // Outer 
  (y1 * x2) +            // Inner 
  (y1 * y2);             // Last

The code above defines the steps needles to transform the multiplication of two binomials into the FOIL steps. Testing this rewrite rule yields:

(x + 3) * (x + 5);
//=> x*x+x*5+3*x+15

Now you might expect that the answer would be x^2 + 8*x + 15, but there are missing reductions in the definition:

x*x = x^2;

(x + 3) * (x + 5);
//=> x^2+x*5+3*x+15

That is close enough for me. :-)

A more complex example would be a function to reduce a number to its prime factors:

factor n = factor 2 n with
  factor k n = k : factor k (n div k) if n mod k == 0;
         = if n>1 then [n] else [] if k*k>n;
         = factor (k+1) n if k==2;
         = factor (k+2) n otherwise;
end;

factor 138;
//=> [2,3,23]

Of the languages listed in this post, I am most blown away lately by Pure.

More resources

• Pure wiki
• Interview With Albert Gräf
• Term Rewriting and All That
• Source

Go

released: 2009, authors: Robert Griesemer, Rob Pike, and Ken Thompson

I have gone back and forth on including Go into the mix because I have had a strong negative reaction against it so far. However, people seem to be excited about the possibilities of a new systems-level programming language, so I felt that it was a shame to exclude it because of my own prejudices.

Here is an example of run-length encoding in Go:

package main

import (
    "fmt"
    "strings"
)

var order = 4
var grain = "*"

func main() {
    t := []string{grain + strings.Repeat(" ", len([]int(grain)))}
    for ; order > 0; order-- {
        sp := strings.Repeat(" ", len([]int(t[0]))/2)
        top := make([]string, len(t))
        for i, s := range t {
            top[i] = sp + s + sp
            t[i] += s
        }
        t = append(top, t...)
    }
    for _, r := range t {
        fmt.Println(r)
    }
}

The world needs more options for system-level programming and it’s virtuous on its own that Go targets that space.

More resources

• Official site
• Source
• Tutorial

Arc

released: 2008, authors: Paul Graham and Robert Morris

It’s hard to believe that Arc is only 3 years old (by release date). I suppose its “oldness” is a function of the countless hours that I spent reading Graham’s posts about it. The world was excited about Arc before its release and unfortunately many opinions on it range from disappointed to outright angry. I too really wanted to like it, but feel far short of the feelings of vitriol that some have maintained. I imagine that if Clojure, Racket, and Qi had not been invented then I probably would have willingly taken the Arc plunge. I will say that its core tenet “succinctness is power” really resonates with me. More than the language itself (which succeeds in this maxim only partially), it’s the core philosophy that stands strong. The world was introduced to Arc through Paul Graham’s “Arc challenge”, partially quoted as:

Write a program that causes the url said (e.g. http://localhost:port/said) to produce a page with an input field and a submit button. When the submit button is pressed, that should produce a second page with a single link saying “click here.” When that is clicked it should lead to a third page that says “you said: …” where … is whatever the user typed in the original input field.

And Arc’s solution was/is thus:

(defop said req
  (aform [w/link (pr "you said: " (arg _ "foo"))
           (pr "click here")]
    (input "foo") 
    (submit)))

A side effect of studying Arc is that one can explore the fantastic Racket language. This is my way to cheat and get a shout out to Racket³ in this post.

More resources

• Succinctness is Power
• Arc tutorial
• Arc’s Core
• Arc Forum
• Arc Source

CoffeeScript

released: 2009, author: Jeremy Ashkenas

I find Jeremy Ashkenas to be an extremely bright programmer with some philosophies that align with my own. As an added bonus, CoffeeScript is one of the few languages on this list that I’ve used in my professional life. The syntax is fairly clean and unceremonious, but its greatest strength (and some might say weakness) is that it’s a very thin skin on top of JavaScript. CoffeeScript is indeed Crockford’s “The Good Parts” manifest, with much better syntax.

One of the interesting features of CoffeeScript is that it provides a very light weight JSON syntax:

jsn = 
  foo: 'a'
  bar: 'b'

jsn.bar
#=> 'b'

That in turn facilitates a lightweight object syntax:

sanders = 
  speed: 8
  juke: 10
  report: -> "#{@speed} and #{@juke}"

sanders.report();
#=> '8 and 10'

There are also list comprehensions:

transpose = (matrix) ->
    (t[i] for t in matrix) for i of matrix[0]

transpose [[1,2,3],[4,5,6]]

//=> [[1,4],[2,5],[3,6]]

For better or worse⁴ CoffeeScript provides class-based modeling:

class Animal
  constructor: (voice) ->
    @voice = voice

  speak: =>
    "#{@voice}!!"

class Dog extends Animal
  constructor: ->
    super("ruff")

d = new Dog
d.speak()
#=> 'ruff!!'

CoffeeScript is in no way the first language targeting JavaScript, but it has provided the perfect storm of adoption, support, and excitement proving its approach as viable to a much larger population. Equally important is that CoffeeScript is blazing the trail in how a symbiotic language can interoperate with existing JavaScript libraries and frameworks.

More resources

• Official site
• Source
• The CoffeeScript book

Potion

released: 2009, author: _why the Lucky Stiff

There was a guy named _why the Lucky Stiff who set the Ruby world on fire. While an enigmatic bloke, his thoughts on Ruby and programming in general were held in high regard by the Ruby community and in some cases the larger programming world. An epic open source contributor was _why and one of his many open source projects was an attempt at a tiny, yet powerful, programming language called Potion. Potion is very much in the Io, Lua, OCaml, and Ruby genre:

Person = class: /name, /age, /sex.
Policeman = Person class (rank): /rank = rank.
Policeman print = ():
  ("My name is ", /name, " and I'm a ", /rank, ".") 
  join print.

Policeman ("Constable") print
# My name is nil and I'm a Constable.

In addition, it has the notion of a separation of the data language and the expression language that I find compelling. While not terribly innovative I must say that what it lacks in innovation it makes up for in clarity of implementation. The whole language weighs in around 10K LOC so an enterprising young programmer (or an old one like me) can absorb the whole language in a few weeks. Code reading is a good exercise. Good code reading is a great exercise.

More resources

• Official site
• Source

Mirah

released: 2008, author: Charles Nutter

JRuby is a fantastic JVM language but it, like every other Java.next language, suffers from the runtime problem. That is, to provide a powerful and performant development and execution experience, symbiotic programming languages often include a library of core functionality that must accompany any runtime deployment. While this is typically not an issue in practice, this runtime environment is inhibiting for lightweight target environments. Enter Mirah. Mirah’s goal, and it is a worthy goal indeed, is to provide a Java.next language with no added runtime footprint while maintaining an experience that is statically typed with the feel of a dynamic language.

def reverse(s:string) 
    StringBuilder.new(s).reverse
end

puts reverse('reversed')

At first glance Mirah looks like JRuby if JRuby allowed type annotations. If I didn’t like JRuby so much I might secretly hope that Charles Nutter would spend all his days working on Mirah.⁵

More resources

• Offcial site
• Inital announcement (known then as Duby)
• Source

Scratch

released: 2007, author: Mitchel Resnick

While watching my 5-year old son effortlessly program some Lego Mindstorms it struck me that the world could use a general purpose language built around the graphical IDE available therein. Imagine my surprise when I found Scratch, a graphical language based off of the same principles, and by the same group at MIT as the Lego offerings.

Here is the ubiquitous “Hello World” in Scratch:

Scratch is a close cousin to Smalltalk, but its presentation and M.O. is geared toward building necessary math and programming skills in children through “playful experimentation” — an idea that I endorse fully. Scratch is in the same vein as Microsoft’s Kodu, but gets my vote because of my previous Lego experience.

More resources

• Official Scratch site
• Scratch Source

ClojureScript

released: 2011, author: Rich Hickey

ClojureScript is a JavaScript.next programming language taking most of the semantics of Clojure and compiling down to JavaScript. I work on ClojureScript, so I am certainly biased badly toward it. Maybe the video below by Brenton Ashworth can speak to how cool it is:

ClojureScript browser-connected REPL from Brenton Ashworth on Vimeo.

It made my jaw drop in any case.

More resources

• ClojureScript Announcement
• Introducing ClojureScript
• Source

Clojure

released: 2007, author: Rich Hickey

Again, I am incredibly biased in favor of Clojure, so I will leave this short. I will say this:

Immutability is a game changer.

Otherwise here is a beautiful implementation of Conway’s Game of Life by Christophe Grand:

(defn neighbours [[x y]]
  (for [dx [-1 0 1] dy (if (zero? dx) [-1 1] [-1 0 1])]
    [(+ dx x) (+ dy y)]))

(defn step [cells]
  (set (for [[loc n] (frequencies (mapcat neighbours cells))
             :when (or (= n 3) (and (= n 2) (cells loc)))]
         loc)))

(def board #{[2 1] [2 2] [2 3]})

(defn print-board [board w h]
  (doseq [x (range (inc w)) y (range (inc h))]
    (if (= y 0) (print "\n")) 
    (print (if (board [x y]) "[X]" " . "))))

(defn display-grids [grids w h]
  (doseq [board grids]
    (print-board board w h)
    (print "\n")))

Furthermore, I wrote a book about Clojure.

More resources

• Official site
• Source

OMeta

released: 2009 (?), author: Alessandro Warth

The buzz surrounding domain-specific languages has grown louder in the recent years and much mindshare is being devoted to the topic. However, as someone who has extensive work (and play) experience in DSL design and implementation, there is much left wanting in the current state of the art. The problem lies in that it’s often difficult to implement a powerful, robust, and robust DSL using general purpose programming languages. There are many reasons for this, but the main problem is that even in languages with incredible syntactic flexibility, it’s nearly impossible to implement DSLs without the ugliness of the host implementation bleeding through. Of course, it can be done. However, as you move toward the ideal DSL for any given domain you inch closer and closer to a full-fledged language parser and runtime. This is less than ideal in many cases. I could write a whole series of posts about this, but that remains for another day. Really, what the DSL-creating world needs is a language and set of tools specifically geared toward building both stand-alone DSLs and those of the embedded variant. Enter OMeta.

Example Ometa code on Github

While something like JetBrains’ Meta Programming System likely provides a more comprehensive experience and more powerful tools, OMeta is striking in its stark simplicity. That is, OMeta is built on the principle of Parser Expression Grammars (PEG) extended to support generalized data structures.

More resources

• Official Site
• JavaScript host source

Fortress

Fortress pushes the boundaries of the age limit in this post since it was internal to Sun years before its public release. The elephant in the room regarding Fortress is the fact that Guy Steele is involved in the project. Despite that fact, Fortress is interesting on its own thanks to its focus on parallelism, syntax inspired by mathematical notation, an object-functional paradigm, comprehensions, traits, contracts, static resolution of units of measure, and distributed data-structures.

for i←seq(1:m) do
  for j←seq(1:n) do
    print a[i,j]
  end
end

The simple nested for loop above exhibits an interesting property in Fortress. That is, the components of the loop execute in parallel. Fortress takes a hard-line stance on parallelism by not viewing it as a language feature, but instead as a language default. Besides implicit parallelism, Fortress also provides explicit parallelism by way of futures.

More resources

• Project page
• Source

What’s next?

There have been even more recent developments in programming language design and development, but in most cases I’ve only tweet-sized impressions to offer.

• Dart
  • Remember the first axiom
  • Seems poised for maximal adoption
• Rust
  • Seems in the vein of D
  • Remember the first axiom
• Ceylon
  • Not sure what this provides for a Java team with solid conventions
• Kotlin
  • Built from the IDE down
• Newspeak
  • smalltalk plus: FloatD epsilon.
• Perl6
  • Since I’m not a Perl programmer I don’t mind waiting
• Magpie
  • Best langdev “blog” ever
  • Sadly have not had a chance to explore the source
  • By one of the Dart devs
  • thanks to Tracy Harms for the reminder

Lesser knowns

These languages were recommended to me by others, so I know very little to nothing about them:

• Felix in the ML family
• Wheeler Seems truly unique. Looking forward to exploring.
• Opa next-gen web language
• Frege a Java.next language in the Haskell family
• Cobra had me at contracts
• Clay another systems language. looks clean to my eyes.

One surprising discovery on my part is that there seems to be no easily accessible information on the advances in array programming⁶ in the past 5 years. However, in that regard I suspect that there is much proprietary innovation. As time marches on the general feeling is that programming languages are becoming more derivative and evolutionary rather than made from whole-cloth and revolutionary. This is not necessarily a bad thing as there are many many ideas yet to be plumbed from the depths of the history of computing.

What have I missed?

:F