read


read
or learn more

Programming language development: the past 5 years

Oct 18, 2011

I recently compiled a list of Perlis Languages that were meant to illustrate varying paradigms not typically found in mainstream programming languages. Those languages were meant to stretch your mind and are not necessarily representative of the state of the art in language design and implementation. In this post I will provide a list of fairly new languages (let’s say 5 years with a little flex) that display1 interesting features and display higher-order thinking in the way that they tend toward an evolution of past learnings in programming language thinking. Not all of these languages enjoy active development, but the more important point is that they represent in some way “new language thinking”. Remember that this goal does not necessarily mean “innovative”.

Unlike my Perlis post, I have not fully explored all of the languages below, so caveat emptor. Please correct me if I misrepresent something. No meaning behind the ordering implied.

take the associated poll!

Shen

released: 2011, author: Dr. Mark Tarver

I included the fantastic language Qi in my Perlis Languages post, so it may seem gratuitous to include its successor Shen here. In fact Shen and Qi offer most of (all of?) the same high-level features:

  • A Lisp
  • Optional static typing
  • Pattern matching with guards
  • Macros
  • Partial application
  • Function backtracking
  • Builtin Prolog
  • Builtin compiler compiler

However, Shen is indeed the next evolutionary step after Qi. Shen builds on the ideas in Qi in various ways, but the primary motivating force is targetability. What exactly does that mean? The precursor Qi was built to target Common Lisp as its host language and did so to maximum effect. However, Qi was found to target a very small subset of Common Lisp. Therefore, Dr. Tarver devised the idea that the successor Shen should be defined in terms of a minimal kernel Lisp language called Kl which would, in theory, provide an easier port target across various runtime hosts, including, but not limited to: JavaScript, Python, Clojure, and Common Lisp. I have been thinking of “kernel Lisps” a lot and so Shen is ripe with ideas.

An example of a member function using the embedded Prolog:

(defprolog member
  X [X | _] <--;
  X [_ | Y] <-- (member X Y);)

(prolog? (member 1 [1 2 3]))
/*=> true */

Partial application is automatic (simplified below):

(* 2)
/*=> #<FUNCTION LAMBDA (x) (* 2 x)>

((* 2) 54)
/*=> 108 */

Here is a function that calculates the nth triangle number:

(define triangle
  0 -> 0
  N -> (+ N (triangle (- N 1))))

(triangle 100)
/*=> 5050 */

And a typed version of the same:

(tc +)  /* turn on type-checking */

(define triangle
  {number --> number}
  0 -> 0
  N -> (+ N (triangle (- N 1))))

(triangle 5)
/*=> 15 : number */

(triangle a)    
/* type error */

Shen is a natural evolutionary step along the long, winding path that is Lisp history. Many modern programming languages are absorbing features that Lisp innovated long ago, but still the most exciting languages are Lisps.

More resources

Agda 2

released: 2009, author: Ulf Norel

I’ve yet to truly wrap my head around Agda (any year now), but I can say a few things about it. First, Agda is a purely functional, pattern matching, dependently typed programming language that walks the thin border adjacent to proof assistants. In dependently typed languages, type expressions can contain (depend on) a program expression. Therefore, the resolution of type constraints is a function of code or predicated on values. It’s like a kick in the face. As an added bonus, the Agda type language is the same as the value language (i.e. the programming language itself). What this means is that type invariants in Agda can express a much wider spectrum of constraints than typically attributed to static type systems; for example, a list type that statically guarantees sorted order (PDF). There’s a drop-kick for you.

Here is an encoding of even and odd numbers in the Agda type system:

data Nat : Set where
  zero : Nat
  suc  : Nat -> Nat

fortyTwo : Nat
fortyTwo = 42

plus : Nat -> Nat -> Nat
plus  zero   m = m
plus (suc n) m = suc (plus n m)

mutual
  even : Nat -> Bool
  even zero    = true
  even (suc n) = odd n

  odd : Nat -> Bool
  odd zero    = false
  odd (suc n) = even n

The previous code defines two datatypes: 1) the natural numbers and 2) the even natural numbers. You can also define type functions as infix operators:

_+_ : Nat -> Nat -> Nat
zero  + m = m
suc n + m = suc (n + m)

This represents the extent of my understanding about Agda so far. While it would be nice to know more, what I’ve seen is amazing enough to warrant inclusion.

More resources

Ioke

released: 2008, author: Ola Bini

Ola Bini’s Ioke is based on a simple question: what kind of language can you build if you completely disregard performance and instead focus on expressivity?2 As it turns out you gain incredible expressiveness as shown in Bini’s presentation series (links below). One of the more interesting aspects of Ioke is that it is a homoiconic language providing macros.

myfor = dsyntax(
"takes a name, an enumerable, and a transforming expr
and returns the result of transforming each entry in 
expression, with the current value of the enumerable
bound to the name given as the first argument",

  [argName, enumerable, argCode]

  ''(`enumerable map(`argName, `argCode))
)

myfor(x, 1..5, x*2) 
;=> [2,4,6,8,10]

Another advantage in studying Ioke is my own personal first axiom:

When brilliant people create things, study them.

As programmers the onus is on us to push our skills to the point of uncomfortableness. Studying the works of great minds is a highly effective way to do this.

More resources

Pure

released: 2008, author: Albert Gräf

Pure is a functional language built around term rewriting. Term rewriting is very similar to what we did in high-school algebra using the FOIL method:

(x1 + y1) * (x2 + y2) = 
  (x1 * x2) +            // First
  (x1 * y2) +            // Outer 
  (y1 * x2) +            // Inner 
  (y1 * y2);             // Last

The code above defines the steps needles to transform the multiplication of two binomials into the FOIL steps. Testing this rewrite rule yields:

(x + 3) * (x + 5);
//=> x*x+x*5+3*x+15

Now you might expect that the answer would be x^2 + 8*x + 15, but there are missing reductions in the definition:

x*x = x^2;

(x + 3) * (x + 5);
//=> x^2+x*5+3*x+15

That is close enough for me. :-)

A more complex example would be a function to reduce a number to its prime factors:

factor n = factor 2 n with
  factor k n = k : factor k (n div k) if n mod k == 0;
         = if n>1 then [n] else [] if k*k>n;
         = factor (k+1) n if k==2;
         = factor (k+2) n otherwise;
end;

factor 138;
//=> [2,3,23]

Of the languages listed in this post, I am most blown away lately by Pure.

More resources

Go

released: 2009, authors: Robert Griesemer, Rob Pike, and Ken Thompson

I have gone back and forth on including Go into the mix because I have had a strong negative reaction against it so far. However, people seem to be excited about the possibilities of a new systems-level programming language, so I felt that it was a shame to exclude it because of my own prejudices.

Here is an example of run-length encoding in Go:

package main

import (
    "fmt"
    "strings"
)

var order = 4
var grain = "*"

func main() {
    t := []string{grain + strings.Repeat(" ", len([]int(grain)))}
    for ; order > 0; order-- {
        sp := strings.Repeat(" ", len([]int(t[0]))/2)
        top := make([]string, len(t))
        for i, s := range t {
            top[i] = sp + s + sp
            t[i] += s
        }
        t = append(top, t...)
    }
    for _, r := range t {
        fmt.Println(r)
    }
}

The world needs more options for system-level programming and it’s virtuous on its own that Go targets that space.

More resources

Arc

released: 2008, authors: Paul Graham and Robert Morris

It’s hard to believe that Arc is only 3 years old (by release date). I suppose its “oldness” is a function of the countless hours that I spent reading Graham’s posts about it. The world was excited about Arc before its release and unfortunately many opinions on it range from disappointed to outright angry. I too really wanted to like it, but feel far short of the feelings of vitriol that some have maintained. I imagine that if Clojure, Racket, and Qi had not been invented then I probably would have willingly taken the Arc plunge. I will say that its core tenet “succinctness is power” really resonates with me. More than the language itself (which succeeds in this maxim only partially), it’s the core philosophy that stands strong. The world was introduced to Arc through Paul Graham’s “Arc challenge”, partially quoted as:

Write a program that causes the url said (e.g. http://localhost:port/said) to produce a page with an input field and a submit button. When the submit button is pressed, that should produce a second page with a single link saying “click here.” When that is clicked it should lead to a third page that says “you said: …” where … is whatever the user typed in the original input field.

And Arc’s solution was/is thus:

(defop said req
  (aform [w/link (pr "you said: " (arg _ "foo"))
           (pr "click here")]
    (input "foo") 
    (submit)))

A side effect of studying Arc is that one can explore the fantastic Racket language. This is my way to cheat and get a shout out to Racket3 in this post.

More resources

CoffeeScript

released: 2009, author: Jeremy Ashkenas

I find Jeremy Ashkenas to be an extremely bright programmer with some philosophies that align with my own. As an added bonus, CoffeeScript is one of the few languages on this list that I’ve used in my professional life. The syntax is fairly clean and unceremonious, but its greatest strength (and some might say weakness) is that it’s a very thin skin on top of JavaScript. CoffeeScript is indeed Crockford’s “The Good Parts” manifest, with much better syntax.

One of the interesting features of CoffeeScript is that it provides a very light weight JSON syntax:

jsn = 
  foo: 'a'
  bar: 'b'

jsn.bar
#=> 'b'

That in turn facilitates a lightweight object syntax:

sanders = 
  speed: 8
  juke: 10
  report: -> "#{@speed} and #{@juke}"

sanders.report();
#=> '8 and 10'

There are also list comprehensions:

transpose = (matrix) ->
    (t[i] for t in matrix) for i of matrix[0]

transpose [[1,2,3],[4,5,6]]

//=> [[1,4],[2,5],[3,6]]

For better or worse4 CoffeeScript provides class-based modeling:

class Animal
  constructor: (voice) ->
    @voice = voice

  speak: =>
    "#{@voice}!!"

class Dog extends Animal
  constructor: ->
    super("ruff")

d = new Dog
d.speak()
#=> 'ruff!!'

CoffeeScript is in no way the first language targeting JavaScript, but it has provided the perfect storm of adoption, support, and excitement proving its approach as viable to a much larger population. Equally important is that CoffeeScript is blazing the trail in how a symbiotic language can interoperate with existing JavaScript libraries and frameworks.

More resources

Potion

released: 2009, author: _why the Lucky Stiff

There was a guy named _why the Lucky Stiff who set the Ruby world on fire. While an enigmatic bloke, his thoughts on Ruby and programming in general were held in high regard by the Ruby community and in some cases the larger programming world. An epic open source contributor was _why and one of his many open source projects was an attempt at a tiny, yet powerful, programming language called Potion. Potion is very much in the Io, Lua, OCaml, and Ruby genre:

Person = class: /name, /age, /sex.
Policeman = Person class (rank): /rank = rank.
Policeman print = ():
  ("My name is ", /name, " and I'm a ", /rank, ".") 
  join print.

Policeman ("Constable") print
# My name is nil and I'm a Constable.

In addition, it has the notion of a separation of the data language and the expression language that I find compelling. While not terribly innovative I must say that what it lacks in innovation it makes up for in clarity of implementation. The whole language weighs in around 10K LOC so an enterprising young programmer (or an old one like me) can absorb the whole language in a few weeks. Code reading is a good exercise. Good code reading is a great exercise.

More resources

Mirah

released: 2008, author: Charles Nutter

JRuby is a fantastic JVM language but it, like every other Java.next language, suffers from the runtime problem. That is, to provide a powerful and performant development and execution experience, symbiotic programming languages often include a library of core functionality that must accompany any runtime deployment. While this is typically not an issue in practice, this runtime environment is inhibiting for lightweight target environments. Enter Mirah. Mirah’s goal, and it is a worthy goal indeed, is to provide a Java.next language with no added runtime footprint while maintaining an experience that is statically typed with the feel of a dynamic language.

def reverse(s:string) 
    StringBuilder.new(s).reverse
end

puts reverse('reversed')

At first glance Mirah looks like JRuby if JRuby allowed type annotations. If I didn’t like JRuby so much I might secretly hope that Charles Nutter would spend all his days working on Mirah.5

More resources

Scratch

released: 2007, author: Mitchel Resnick

While watching my 5-year old son effortlessly program some Lego Mindstorms it struck me that the world could use a general purpose language built around the graphical IDE available therein. Imagine my surprise when I found Scratch, a graphical language based off of the same principles, and by the same group at MIT as the Lego offerings.

Here is the ubiquitous “Hello World” in Scratch:

Scratch is a close cousin to Smalltalk, but its presentation and M.O. is geared toward building necessary math and programming skills in children through “playful experimentation” — an idea that I endorse fully. Scratch is in the same vein as Microsoft’s Kodu, but gets my vote because of my previous Lego experience.

More resources

ClojureScript

released: 2011, author: Rich Hickey

ClojureScript is a JavaScript.next programming language taking most of the semantics of Clojure and compiling down to JavaScript. I work on ClojureScript, so I am certainly biased badly toward it. Maybe the video below by Brenton Ashworth can speak to how cool it is:

ClojureScript browser-connected REPL from Brenton Ashworth on Vimeo.

It made my jaw drop in any case.

More resources

Clojure

released: 2007, author: Rich Hickey

Again, I am incredibly biased in favor of Clojure, so I will leave this short. I will say this:

Immutability is a game changer.

Otherwise here is a beautiful implementation of Conway’s Game of Life by Christophe Grand:

(defn neighbours [[x y]]
  (for [dx [-1 0 1] dy (if (zero? dx) [-1 1] [-1 0 1])]
    [(+ dx x) (+ dy y)]))

(defn step [cells]
  (set (for [[loc n] (frequencies (mapcat neighbours cells))
             :when (or (= n 3) (and (= n 2) (cells loc)))]
         loc)))

(def board #{[2 1] [2 2] [2 3]})

(defn print-board [board w h]
  (doseq [x (range (inc w)) y (range (inc h))]
    (if (= y 0) (print "\n")) 
    (print (if (board [x y]) "[X]" " . "))))

(defn display-grids [grids w h]
  (doseq [board grids]
    (print-board board w h)
    (print "\n")))

Furthermore, I wrote a book about Clojure.

More resources

OMeta

released: 2009 (?), author: Alessandro Warth

The buzz surrounding domain-specific languages has grown louder in the recent years and much mindshare is being devoted to the topic. However, as someone who has extensive work (and play) experience in DSL design and implementation, there is much left wanting in the current state of the art. The problem lies in that it’s often difficult to implement a powerful, robust, and robust DSL using general purpose programming languages. There are many reasons for this, but the main problem is that even in languages with incredible syntactic flexibility, it’s nearly impossible to implement DSLs without the ugliness of the host implementation bleeding through. Of course, it can be done. However, as you move toward the ideal DSL for any given domain you inch closer and closer to a full-fledged language parser and runtime. This is less than ideal in many cases. I could write a whole series of posts about this, but that remains for another day. Really, what the DSL-creating world needs is a language and set of tools specifically geared toward building both stand-alone DSLs and those of the embedded variant. Enter OMeta.

Example Ometa code on Github

While something like JetBrains’ Meta Programming System likely provides a more comprehensive experience and more powerful tools, OMeta is striking in its stark simplicity. That is, OMeta is built on the principle of Parser Expression Grammars (PEG) extended to support generalized data structures.

More resources

Fortress

Fortress pushes the boundaries of the age limit in this post since it was internal to Sun years before its public release. The elephant in the room regarding Fortress is the fact that Guy Steele is involved in the project. Despite that fact, Fortress is interesting on its own thanks to its focus on parallelism, syntax inspired by mathematical notation, an object-functional paradigm, comprehensions, traits, contracts, static resolution of units of measure, and distributed data-structures.

for i←seq(1:m) do
  for j←seq(1:n) do
    print a[i,j]
  end
end

The simple nested for loop above exhibits an interesting property in Fortress. That is, the components of the loop execute in parallel. Fortress takes a hard-line stance on parallelism by not viewing it as a language feature, but instead as a language default. Besides implicit parallelism, Fortress also provides explicit parallelism by way of futures.

More resources

What’s next?

There have been even more recent developments in programming language design and development, but in most cases I’ve only tweet-sized impressions to offer.

  • Dart
    • Remember the first axiom
    • Seems poised for maximal adoption
  • Rust
    • Seems in the vein of D
    • Remember the first axiom
  • Ceylon
    • Not sure what this provides for a Java team with solid conventions
  • Kotlin
    • Built from the IDE down
  • Newspeak
    • smalltalk plus: FloatD epsilon.
  • Perl6
    • Since I’m not a Perl programmer I don’t mind waiting
  • Magpie
    • Best langdev “blog” ever
    • Sadly have not had a chance to explore the source
    • By one of the Dart devs
    • thanks to Tracy Harms for the reminder

Lesser knowns

These languages were recommended to me by others, so I know very little to nothing about them:

  • Felix in the ML family
  • Wheeler Seems truly unique. Looking forward to exploring.
  • Opa next-gen web language
  • Frege a Java.next language in the Haskell family
  • Cobra had me at contracts
  • Clay another systems language. looks clean to my eyes.

One surprising discovery on my part is that there seems to be no easily accessible information on the advances in array programming6 in the past 5 years. However, in that regard I suspect that there is much proprietary innovation. As time marches on the general feeling is that programming languages are becoming more derivative and evolutionary rather than made from whole-cloth and revolutionary. This is not necessarily a bad thing as there are many many ideas yet to be plumbed from the depths of the history of computing.

What have I missed?

:F


  1. As with any post, there should always be an implicit “to me” and “in my opinion” attached to any statement. Welcome to blogging! 

  2. Really the goal is probably extreme expressiveness, but I like the way the speed angle sounds — it’s more “in your face”. 

  3. I am constantly amazed at the endless well of amazing libraries available for Racket. 

  4. If you know me, then you know which I might say. ;-) 

  5. Mirah is an example of an idea that I like to call “programming language restoration”. While not exactly emblematic of the ideal, it’s right in spirit. I will write a blog post about this one day. 

  6. Technically J has been around for a long time, but has only recently been open sourced. It is a very interesting language that I hope to explore sooner rather than later. Additionally, the Q/KDB pair is considered to be the state of the art in the array programming space, but they fall outside of the limits of this post. 

61 Comments, Comment or Ping

  1. Don Park

    Elixir is a functional language on top of Erlang that was inspired by ruby’s syntax.

    https://github.com/josevalim/elixir#readme

  2. samwyse

    Speaking as an old APL programmer who never really adopted any of its successors, Numeric Python is pretty much Python with APL merged in. Let’s face it, if you’re getting rid of the APL character set, why not go all the way and adopt a “normal” syntax?

  3. Mark Wong-VanHaren

    Yeti: http://mth.github.com/yeti/ Minimalist ML on the JVM.

  4. Andreas

    Hi,

    you should also have a look at Pi:

    Pi (http://www.pi-programming.org/What.html) is a language that is based on pattern matching and allows to extend the language with ease.

    Greetings, Andreas

  5. I have big problems with the bizarre way syntax is designed in many of these newfangled functional languages. It’s as if we’re measuring expressiveness by number of keystrokes only. Many of the important language-level operators are being expressed with punctuation rather than keywords, and sometimes that punctuation is pretty ugly; like Ioke, really, using ” and ` as two different operators?

    It rather reminds me of perl, providing language-level global variables like $_ and $’… Why not require a few extra keystrokes in favor of increased readability?

    To some extent this might be related to how the sample code was written, and not the language itself.

  6. I would mention Nemerle, which does an interesting work on hygienic macro systems. F# is interesting too, but less innovative, I think.

  7. You really make it appear so easy with your presentation however I in finding this matter to be actually one thing which I feel I would never understand. It seems too complex and extremely wide for me. I am having a look ahead for your next post, I’ll attempt to get the cling of it!

  8. Our Website

    I seldom drop remarks, however I glanced through a few comments here fogus: Programming language development: the past 5 years. I do have 2 questions for you if you do not mind. Could it be just me or does it seem like a few of the remarks appear like they are left by brain dead individuals? :-P And, if you are writing on additional social sites, I would like to keep up with you.

    Could you make a list of every one of your social sites like your twitter feed, Facebook page or linkedin profile?

  9. Buying A Synthesizer

    I’m really enjoying the design and layout of your site. It’s a very easy on the eyes which makes it much more pleasant for me to come here and visit more often. Did you hire out a developer to create your theme? Excellent work!

  10. A Curious Observer

    A functional counter part to Dart as a web programming language is Elm. It has immutability and type inference and complete interoperability with JavaScript and HTML and CSS.

    import Text (..)

    main : Element main = asText “Hello World”

    ^ elm “hello world”

    elm-lang.org

  11. You need to try Felix a bit more. Despite being generally an OO-advocate/anti-functional/OCaml-dislike person, I’ve grown very fond of Felix.

Reply to “Programming language development: the past 5 years”