A sweet Clojure-ish language

It's likely no secret that I really dig Clojure. It's the first unityped language I've used since Objective-C in the early 90s that I really like. Yes, I still love Scala, but I think Clojure is pretty much the best designed computer language I've ever used... and maybe that's because Clojure is mostly "less is more" but in a few places, it's not.

The things I really like about Clojure are:

• You have to work to escape immutability, but you don't have to work as hard as you do in Haskell and swap! is a lot less daunting than unsafePerformIO
• The right set of collections: lazy sequences, strict vectors, hashmaps, and sets. All immutable. This contrasts with the 850,000 or so collections class variations in Scala. You need a PhD to navigate Scala's collections.
• The collections have syntactic representation in Clojure's S-expressions. This is a deviation from most other Lisps. It's one of the places that Clojure opted for a little "more" and the balance is really nice.
• Protocols are the nicest mechanism for polymorphic dispatch I've seen in any language.
• Keywords as functions makes so much sense.
• Transducers because that's the way functions as applied to collections are supposed to work.

Java Virtual Machine

For much of the work that I do, the Java Virtual Machine is the best thing since sliced bread. It's performant, predictable, and has a very nice ecosystem of libraries.

But the JVM really needs at least 128MB of RAM just to print "Hello World" and any non-trivial code needs a process size of 512MB and most of the code I work with, the JVM needs 2GB of RAM to work reasonably well.

And the JVM has the slowest startup time of any runtime I've ever encountered.

So, if you've got the RAM, the JVM ultimately gives you the performance. But that's not always what I want.

Enter Pixie

Pixie is a side project from the mind of Timothy Baldridge. Tim is also a key part of Clojure's core.async.

Pixie has all the good attributes of Clojure, but has a 10MB runtime. It's small and fast. How? It's written in RPython and compiles down to native code. It's got a tracing JIT and full garbage collection. It's got very fast start-up time and Tim has some benchmarks that demonstrate that it runs reasonably fast.

So, Pixie is "Clojure without the weight of the JVM". Pixie interoperates with the outside world pretty much the same way Python does... via libffi. This means your Pixie code can call to C code... to native .so libraries.

I did not dig too deeply into Pixie's threading model, but it seems to be green threads that are limited to a single OS thread and tight numeric loops need occasional yield statements to cooperate with other green threads.

So, far, mostly sweet.

There are some things that I would like to see from/in Pixie... but this is as much a list of stuff I need to work on as a list for others... and it is by no means a complaint list:

• Full nrepl support
• Library documentation... and a system for generating HTML documentation.
• Some form of package distribution support... could this be piggy-backed on Clojars?
• Support for embedding Pixie in other native programs

Embedding Pixie

Okay... so the truth comes out... I need to do some yak shaving.

I did a presentation on Securing PaaS using Docker and Weave at QCon London. Most of the work I've done has been a collection of hand-written, specific to my app, scripts to manage my network.

After watching some of the other Docker-related presentations at QCon, I realized the state of Docker management tools is weak at best. So, I figured I'd write some tools for defining and deploying and managing secure PaaS (Platform as a Service) tenants on a Docker cluster.

Docker-land uses mostly Go to write Docker-related tools. So, Go seemed like the right choice. However, I immediately ran into Greenspun's Tenth. There was some "dynamic" stuff I needed to express in my configuration files. Some of it was simple (e.g., how to name collections of instances, for example "Spark1", "Spark2", etc.) Some of it was more complex... expressing the logic for expanding or contracting instances for a given tenant.

So, my choice was to write some Turing Complete, one-off language that could be expressed in JSON or YAML or I could use a LISP.

I was also looking for a reason to using Pixie in anger, so I started to look into embedding Pixie in my Go code.

As a side note, expressing configurations in Clojure S-expressions makes a ton of sense. Clojure S-expressions provide a superset of JSON and Clojure's eval allows you to define a function to compute certain values. It's an escape hatch so that you're configuration files don't need to become accidentally Turing complete. The Turing complete nature of your configuration files is well defined as Clojure.

Where I am in this Yak-Shave-Fest... I can build Pixie as a .so rather than executable by changing the --no-shared build flag to --shared. I need to expose some of the RPython functions that initialize the Pixie/PyPy runtime. Next, I have to create a function in RPython that takes a char * and does all the read/eval parts of it. Finally, I have to figure out the boundaries between how Go hands threading and how the Pixie/PyPy runtime handles threading.

Lots of fun... but potentially lots of excellent upside. I'll blog about and open source stuff as it becomes more than just arm-waving.

But, please do check out Pixie

With all the above... I urge you to check out Pixie. It's a really nicely done Clojure-esque language that has what I consider to be a bright future.