clojure.spec: Introduction
If you’ve looked into the relatively new Clojure library clojure.spec you might have come across something curious. Observe the use of core.spec/or
:
(require '[clojure.spec :as s]) (s/def ::num (s/or :float float? :int int? :ratio ratio?)) (s/conform (s/coll-of ::num) [0.25 1/2 1]) ;;=> [[:float 0.25] [:ratio 1/2] [:int 1]]
The result of the call to s/conform
is quite descriptive in the way that it mirrors the vector [0.25 1/2 1]
provided. Indeed, the form of the return looks interesting like a naive version of a parse-tree, but why should such a thing happen? What’s the purpose of the s/conform
function in light of the fact that spec already provides functions for validation and rich descriptions of their failures:1
(s/valid? (s/coll-of ::num) [0.25 1/2 1]) ;;=> true (s/valid? (s/coll-of ::num) [0.25 1/2 :blarg]) ;;=> false (s/explain-data (s/coll-of ::num) [0.25 1/2 :blarg]) ;;=> #:clojure.spec{ ; :problems ({:path [:float], ; :pred float?, ; :val :blarg, ; :via [:user/num], ; :in [2]} ; {:path [:int], ; :pred int?, ; :val :blarg, ; :via [:user/num], ; :in [2]} ; {:path [:ratio], ; :pred ratio?, ; :val :blarg, ; :via [:user/num], ; :in [2]})}
It certainly seems that if spec were limited to validation and explanation then it would be fairly useful in its own right. However, spec provides capability beyond validation and explanation precisely because it’s not designed to merely solve those problems but instead recognizes them as components for solving a much more pernicious “language problem”.
So what’s the problem?
Functional languages in general and Clojure specifically fosters the use of aggregations of simple data to represent complex domain information, the ugly truth is that each and every data aggregation necessarily constitutes its own mini-language. Of course, the nature of languages is such that their meaning and interpretation is encoded in custom parsing code. That is, prior to the introduction to spec, Clojure programmers had to create ad hoc parsers for walking their domain structures and identifying and reporting any errors or inconsistencies. Tools like the useful Schema library helped to alleviate the complexities around the problem of the data mini-language, but it’s focused mainly on the problem of validation. The spec library on the other hand recognizes that there is a fundamental synergy between specification, parsing, combination, validation, explanation, and generation and provides an extremely powerful tool for managing the complexities inherent in data design.
I’m going to take some time over the coming weeks to write about spec and explore some of its uses and advantages, specifically as they pertain to the problem of mini-languages brought on by the use of domain data encoding and in its use as a general tool for thinking about data and program design.
:F
Thanks to David Nolen, Alex Miller, Rich Hickey, and Carin Meier for reading a draft of this post.
-
The
#:clojure.spec{ ... }
form shows the new namespaced map feature slated for the Clojure 1.9 release. ↩
4 Comments, Comment or Ping
Alistair R
You’re off to a good start, Michael. Can’t wait for more!
Jan 10th, 2017
Tzach
Thanks for the post! Something bothers me here: why can’t the syntax be more like
(def num? (or float? int? ratio?)) (s/conform (s/of seq? num?) [0.25 1/2 1])
Seems to me like “or” and “s/or” are similar, and the lib should take care of extracting one from the other. Is it a deliberate design choice or a language limitation?
Jan 10th, 2017
drc
Fogus,
Your comments regarding the creation of ad hoc parsers reminded me of langsec.org’s remark that “the only path to trustworthy software that takes untrusted inputs is treating all valid or expected inputs as a formal language, and the respective input-handling routines as a recognizer for that language”. BTW, there are some really interesting papers on that site.
Looking forward to the rest of the series.
Jan 11th, 2017
madstap
@Tzach You can use clojure.core/or (or some-fn) kind of like you’re doing in your post, and it’ll work fine. What clojure.spec/or does differently is that it labels the branch taken, so that you know which of the predicates were true. If you just want to check if the data is valid, it works exactly the same, but this allows you to also parse your data using the same spec as you use for validation.
Observe the difference between the two.
(def num? #(or (float? %) (int? %) (ratio? %))) ;; Could use (some-fn float? int? ratio?)
(s/conform (s/coll-of num?) [0.25 1/2 1]) => [0.25 1/2 1] ; The returned data is the same.
(s/def ::num (s/or :int int? :float float? :ratio ratio?))
(s/conform (s/coll-of num?) [0.25 1/2 1]) => [[:float 0.25] [:ratio 1/2] [:int 1]] ;; now the data is labeled, so you know which branch it took ;; and you don’t have to write any more parsing code ;; than the spec you already wrote.
Jan 25th, 2017
Reply to “clojure.spec: Introduction”