Compile the Lumo ClojureScript interpreter for AWS Lambda.

May 25, 2019
Andrea Richiardi

Every now and then, in order to be more productive from then onwards, you have to bang your head against a wall in the now. This is a bit of what happened when my friend and mentor Matt Bishop set me thinking about writing ClojureScript lambda functions on AWS.

The goal here would be to avoid using the standard JVM-based compiler and instead use Lumo to interpret directly ClojureScript code in the cloud. This would allow us to capitalize on AWS tools like the integrated and collaborative IDE Cloud9.

I would like to share the journey that brought me to successfully compile the lumo binary statically against musl on the Amazon linux AMI so that I can go back to write Lisp.

AWS Lambda Runtime

One of the newest and coolest features of AWS Lambda is the possibility of providing your own runtime. The runtime layer is akin to a Docker layer and is shared across invocations. Each file you put in the layer will be visible to your running function.

This is already very useful for cross-cutting concerns: one could provide a layer for common dependencies that you upload once, or on update, instead of every time along with your function.

The killer feature though is that if one of the layers contains a shell script called bootstrap, AWS will run that by convention on cold start.

Why is that a killer feature?

Because bootstrap can be used as the trampoline for running anything you like - including of course ClojureScript interpreters!

The Dark Side

Everything looks shiny and bright on paper. Unless you find out that the paper is that bright because is burning.

The most popular interpreter or, to be more precise, JIT JavaScript transpiler for ClojureScript is called lumo. It targets Node.js. Lumo starts up very quickly, usually under the noticeable barrier of 100 ms on most machines. This is great news because you actually pay for that time on AWS during the lambda cold start.

In order to achieve that though, lumo uses the Node.js snapshot feature, which effectively means that the lumo binary is in reality a node binary plus the JavaScript necessary for transpiling ClojureScript. The problem is that AWS Lambda functions are running inside the Amazon linux AMI but the lumo binary is built as dynamically linked binary for Ubuntu.

My guess is that by now you realize that this is not going to be fun. Do not worry, I have done the grunt work already and there is an happy ending.

Self-inflicted wounds

Linking is actually a non-problem as long as we can build lumo statically. No surprises there, a quick PR and voila, you can now compile lumo statically.

The whole idea also already existed in the world, as Mikkel Gravgaard had already worked something out in his grav/aws-lumo-cljs-runtime. Thank you Mikkel for showing me the path and answering to questions on GitHub.

So one day you demo your breakthrough to the aforementioned: "Hey Matt look! there is better way to run ClojureScript on AWS, without touching the JVM 1 !".

You are happy, you show your superior workflow at the REPL.

(require 'aws-sdk)
(def s3-client (aws-sdk/S3.))
(.getObject s3-client ...)

...Segmentation fault (core dumped) !?!$%*...

Node.js segmentation fault, interesting. You try again, maybe it is a demo curse issue but inside you already know that you are no, the problem lies somewhere else.

Why, glibc why?

The real problem here is glibc.

This is a weird one though. The lumo binary that the grav/aws-lumo-cljs-runtime produces looks good, it even runs ClojureScript forms within a deployed AWS lambda layer. It crashed at the first use of the AWS SDK for JavaScript.

This is the time when you need Stack Overflow again 2.

It turns out that the standard glibc included in most of the linux distributions is not very "flexible". Not that I really know what flexible technically means here but because node needs to compile against it, we need something else.

Enter musl, pronounced like the word "mussel": an open source standalone implementation that can be used as drop in replacement to glibc.

Achievement unlocked, the plan is now to build lumo inside the amazonlinux docker image with static linking against musl.

Exercising your Musls

It is a smooth sail from now on, just build an image that allows the compilation steps to use musl-gcc instead of the standard gcc (again thanks Mikkel for the initial DockerFile).

I wish it was, sigh. The last iceberg to dodge is that you cannot really rely on musl-gcc for compiling node. You need the fully fledged cross-compiled toolchain for it. In particular g++ is used as linker.

Fortunately, the rabbit hole ends here.

Firstly because the suite can be cross compiled easily with richfelker/musl-cross-make3 and finally because I am open sourcing the docker image that does it all:

Use it in conjuction with grav/aws-lumo-cljs-runtime for building the actual AWS Runtime layer.

Happy Lisping!

  1. We were trying to avoid the JVM cause of its higher startup time compared to tooling for other languages.

  2. It is difficult to believe but I rarely open Stack Overflow when I develop Clojure. The language rarely gets in the way.

  3. It goes without saying that I am immensely grateful to the maintainer and the contributors.