Unison Computing is the company behind the open source Unison programming language and the Unison Cloud Platform. This post has our candid experience report on using Unison over the years.
What do we use Unison for at our company? Several mission-critical applications:
- Unison Cloud's basic compute fabric, an elastic pool of nodes that runs arbitrary distributed Unison code. This is the basis for our cloud platform. It handles moving computations between nodes, on-the-fly loading of dependencies, and code syncing.
- Unison Cloud's scalable transactional storage layer. This is backed by DynamoDB, but significant engineering went into building the capabilities we wanted. (For instance, we support arbitrary transactions, storage of arbitrary typed Unison values, and defining new durable data structures besides just key-value tables.)
We're also currently working on a few other important projects in Unison: code and documentation search for Unison Share, powered by a scalable distributed search index, and a programmer blog engine (with posts rendered from the excellent Unison documentation format).
In addition to these more serious applications, most of us have little side projects or libraries written in Unison just for fun. 😀 Suffice to say, we've been using Unison heavily.
So how has it gone? Well, we're quite happy with Unison today, but it's been a journey to get here.
What we struggled with
On the one hand, Unison makes new things possible and it's been a thrill working with the language. But when we first started on Unison Cloud, the language, ecosystem, and tooling weren't nearly as mature as today, and we encountered all sorts of difficulties. All in all, if you'd asked me even a year ago if Unison were a language I'd recommend for real stuff, I'd have said probably not!
Here's what we encountered, in increasing order of annoyance:
- Runtime bugs. When Unison Cloud's basic compute fabric was first written, it was by far the most significant system written in Unison by anyone (likely it still is). It's also fairly intricate. Our implementation exposed several bugs in the Unison runtime which were not fun to minimize and track down. This at least stabilized pretty quickly and the runtime today is quite stable, though not super performant. We'll be shipping the JIT native compiler with much faster performance in the coming few weeks.
- Gaps in the ecosystem. The ecosystem and core language weren't nearly as developed when we started using Unison for our cloud platform. Even very basic libraries did not exist (for TLS, HTTP, JSON encoding / decoding, various AWS libraries, etc) and we regularly needed to stop what we were doing and build them or add core language builtins. While this has improved greatly over the years, in the early days it was a major bottleneck.
- Issues with Unison's tooling. Unison is a language built around an amazing core idea with huge potential for vastly better tooling than other languages, but actually realizing all that potential has been a colossal amount of work, with a lot of little details and engineering to get right. More on this below.
Issues with Unison's tooling in the early days
The experience of developing Unison code today is quite lovely (more on that later), but in the early days it had all kinds of problems. Some were engineering issues, others were more conceptual.
On the engineering side, before Unison's switch to SQLite for storing codebases (a huge effort that took us many months), Unison used the file system as a terrible database, which was both complicated and led to all sorts of performance problems. Even when we fixed that, Unison Share didn't exist when we first started on Unison Cloud, so we didn't even have a great way of browsing and collaborating on Unison code. Instead we stashed large SQLite database files on GitHub and shared code that way, which was exactly as janky as it sounds.
In addition, the Unison Codebase Manager formerly had annoying workflows and bugs we hit regularly when developing Unison Cloud, especially around basic things like updating code and library dependencies. The situation is now vastly improved (update generally just works, as does upgrade for upgrading library dependencies).
There were more conceptual issues, too. Unison was for a long time missing any real notion of "projects", "releases", and "merge requests", instead providing just a freeform “versioned namespace tree". This seemingly elegant abstraction was, in theory, capable of representing many things, but not well or with good ergonomics. We at one point published a ridiculously complicated document laying out certain conventions that could be followed to track a project with a bunch of releases, branches, and in-flight merge requests. I think maybe 2 people in the world actually read it and understood it. 😀
Be opinionated if it helps to free people from needing to think about stuff that probably doesn't matter much for their work. (Also see: Unison's automatic formatting of code.) This both reduces burden of choice for users and lets us craft a more tailored UI with better ergonomics.
We've since introduced first-class support for projects, releases, merge requests, and tickets, and we think the experience is much better overall. On the one hand, Unison's core model doesn't actually care about these things. Code is referenced by hash, and dependencies are tracked at the level of individual definitions. Any level of grouping or organization above the level of individual definitions is purely for the convenience of humans.
But this "for humans" level of grouping is incredibly important! For authors, it's just easier to maintain a self-contained collection of definitions with sensible names. These definitions can all be updated together, with a common set of release notes, a README for the library as a whole, and so on. As a consumer of a library, relying on a curated bundle of definitions and documentation is also often easier. It's easier to learn ("an expert has documented and bundled together all these bits of functionality that I'm likely to need when working with this domain") and easier to maintain as a dependency due to less burden of choice. ("An expert in the domain of this library has made hundreds of related changes or additions that are all consistent with each other, and I'm just going to go with their recommendations.")
While Unison doesn't prevent you from making decisions in a more fine-grained way—you can easily reference multiple versions of "the same" definition or library within your project—the most common use case should be easy and ergonomic and that's what we've settled on.
What's been great
Overall, the Unison language is very capable. The combination of basic functional programming support (data types, pattern matching, proper tail calls) plus abilities, along with Unison's fancy runtime which supports async I/O and green threads feels like a real sweet spot. While Unison is simpler than languages like Haskell or Scala, which many of us have used heavily, we haven't missed the additional features of these languages nearly as much we thought. There's always a way of accomplishing what you want in Unison.
Besides the core language, there's much more about Unison that we appreciate every day:
- Unison's fundamental superpower of being able to serialize arbitrary values, including functions, and ship them to a different location without dependency conflicts. We obviously use this heavily in Unison Cloud.
- Almost never waiting around for code to compile, since the Unison codebase is a "perfect" shared compilation cache that invalidates as little as possible on updates. We get the low build times of a dynamically typed language, but without having to give up static types!
- The code browsing experience on Share is amazing, with all code hyperlinked, docs right there, and the ability to pop open individual definitions rather than "files". Browsing code on GitHub feels archaic in comparison.
- It's a joy to create and document libraries. The documentation experience (both viewing and authoring documentation) is nicer than any language ecosystem we've ever used, and there's no friction to it. With a single
pushcommand, everything shows up on Unison Share, the code fully hyperlinked, with live examples in the docs, and it looks awesome. - Cutting releases with a click of a button on Share is great.
- The workflow of using scratch files with watch expressions is very freeing and pleasant. It's easy to get into a flow state. Just start coding without worrying too much about where things should live. Once definitions are added to the codebase, you can move them around later with a single command and without breaking anything.
- It's nice to be able to reference multiple versions of the same library, without it being a "stop everything until this is fixed" dependency conflict. Our compute fabric is happily using multiple versions of libraries for backwards compatibility with older clients.
- Packaging of Unison code for deployment is easy. A single
compilecommand produces a bytecode bundle file with the minimal set of needed dependencies. This bytecode file can then be passed toucmviaucm run.compiled mything.ucto start your application. We use this everywhere for packaging code used by Unison Cloud's implementation. - The Unison community is great, full of friendly, kind, and helpful people.
Conclusions
Despite all the problems we've encountered during the early days of using Unison for real work, we've stuck with it and continued making things better. We're actively using the language in production now and feel good about recommending that others do the same, with the following caveats:
- Even today, Unison's ecosystem is not as well-developed as other languages. If you're thinking of using it for something, we recommend taking an inventory of the libraries you'll need and decide if you're comfortable writing whatever libraries are missing.
- The current runtime performance isn't great, so if you're doing something performance sensitive, check that Unison is fast enough for your use case. But important note: we're shipping the first version of the JIT native compiler in the next few weeks, which will make a huge difference here.
Besides just the niceties of Unison today, it's also highly motivating to use a technology that feels like it has huge potential, and which is getting better and better with each release.
As much as I've enjoyed using other languages, the basic experience of building systems with these languages hasn't changed appreciably in a long time. Programs are still trapped in the tiny box of a single OS process despite most systems being distributed, we're still versioning and viewing code using lowest-common denominator tools that only understand text, and so on. Sure, languages introduce new features and tooling, but it's always within the existing narrow paradigms.
Getting to use Unison is like stepping into an alternate programming universe where hard things are easy and new things are possible. It's fun, inspiring, and helps the work feel more joyful.