Sandstorm Blog

Sandstorm vs. Kernel Exploits

By Kenton Varda - 13 Aug 2014

A few weeks ago, I discovered a vulnerability in the Linux kernel that may allow sandboxed programs to break out of a container. The same problem can also be exploited in a different way on many typical Linux systems to gain local root privileges. I reported the problem to the kernel’s security team. Yesterday, upon release of a fix for the issue, the vulnerability was disclosed publicly (CVE-2014-5206 and CVE-2014-5207).

Docker, in many configurations, is likely vulnerable to this exploit. Sandstorm, however, is not.

To be fair, Docker is not primarily designed to be a sandbox. It is instead designed to run arbitrary off-the-shelf Linux applications and distributions in a clean, hermetic way – and at this, it excels. There are some provisions for security, but the general recommendation is to avoid deploying anything that could be actively malicious.

Sandstorm, on the other hand, cares deeply about sandboxing, and is willing trade off compatibility for security. We are happy to hide or disable whole swaths of the Linux platform that most apps don’t need, at the expense of having to develop work-arounds for the few apps that do need them. By doing this, we reduce the “attack surface” of the Linux kernel – any vulnerabilities found in the features we disabled do not affect Sandstorm. Of course, it is exactly these rarely-used obscure features which have received the least scrutiny and thus are the most likely to contain vulnerabilities.

Some examples of features we remove:

• We use seccomp-bpf to disable system calls that are not needed by apps. For example, by disabling the mount(2) system call as well as blocking the creation of new UID namespaces, Sandstorm sidesteps the vulnerability I discovered. (Ironically, Sandstorm itself relies on these very features to set up the sandbox, but apps do not need them.)
• We do not mount /proc or /sys, which are filesystem-based interfaces exposing myriad kernel features.
• The only devices we present to apps are /dev/null, /dev/zero, and /dev/urandom.
• We prohibit sandboxes from using setuid binaries to gain privileges by setting the no_new_privs process flag and other measures.

Many of these things can be enabled in Docker as well, but only by manual configuration.

All of these things potentially break some applications, but when problems do come up, it’s usually easy to work around the issue by tweaking the application code. Sandstorm’s willingness to require such changes is a key design difference from Docker. To be clear, I am not criticizing Docker’s choice – it makes tons of sense for what Docker is trying to accomplish. But Sandstorm is trying to accomplish something different.

In the future, we hope to further improve the Sandstorm sandbox by moving more kernel functionality to userspace. Using seccomp-bpf, we can actually cause some system calls to raise SIGSYS, thus allowing us to “virtualize” the call by implementing it in a signal handler. Through this, we can do anything from mocking out an irrelevant system call to implementing a whole virtual filesystem (by intercepting open(2)).

One thing in particular that we’d like to do is implement exec(2) in userspace. Parsing ELF binaries is non-trivial, and there have been vulnerabilities in the kernel’s ELF parser in the past. But, this work does not actually need to happen in the kernel. The main function of exec which cannot be performed in userspace is granting privileges to setuid binaries – but we actively don’t want that feature anyway.

Two of Sandstorm’s advisors – Andrew Lutomirski (a security-focused kernel developer) and Mark Seaborn (a member of Chrome’s Native Client sandbox team) – are experts on these issues and have been helping us work out the details. Andy actually implemented the no_new_privs kernel feature and contributed Sandstorm’s seccomp filter. He has been keeping us informed as new kernel vulnerabilities arise. Mark has implemented several Linux native code sandboxes before and is helping us design what we hope will be the best yet. (Note: Our advisors contribute in their spare time, not on behalf of their employers.)

Of course, to see any of this happen, we need your help. :) Please fund our Indiegogo campaign.

Porting Meteor apps to Sandstorm in three minutes

By Kenton Varda - 12 Aug 2014

As you know, Sandstorm apps can be written using any tech stack that runs on Linux. We have apps written in PHP, Python, Node.js, C++, Go, and even Rust.

But if you’re starting from scratch and need to choose, the easiest way to write a Sandstorm app today is using Meteor. Meteor is a modern open source web framework that makes it ridiculously easy to build high-quality, low-latency, real-time web apps quickly. Sandstorm’s own front-end is built on Meteor.

I recently gave a lightning talk at Meteor Devshop where I demonstrated porting a Meteor app to Sandstorm. The app I chose was written by independent developers with no intent to target our platform. As you’ll see, it took me under three minutes to port live on stage:

Note: The video shows the process for older versions of Meteor, in which packages had to be installed with the separate tool Meteorite (mrt). This blog post has since been updated for Meteor 0.9.x, which has its own built-in package repository.

When it comes to writing a Sandstorm app, Meteor works particularly well as a platform because it tends to produce self-contained apps. Meteor’s tools already know how to construct a “bundle” containing all of the app’s dependencies. From there, we just need to add Node and Mongo, and we have ourselves a Sandstorm package. We don’t even need to use our trick to automatically detect dependencies.

To make this even easier, we’ve written a tool called meteor-spk which automates this. We’ve also published a Meteor package kenton:accounts-sandstorm which integrates Meteor’s accounts system with Sandstorm’s unified login. Together, these mean that porting a Meteor app to Sandstorm usually involves three commands:

meteor add kenton:accounts-sandstorm
meteor-spk init
meteor-spk pack myapp.spk

Our intrepid fan Jake Weisz – who previously contributed a port of EtherCalc – has used this tool to port Meteor Blocks, a simple voxel-based model editor written using Meteor. You can find it on the Sandstorm app list now – try the demo if you don’t already have your own server.

We hope to see tools like this developed for other frameworks in the future. meteor-spk is just a simple bash script wrapping Sandstorm’s spk tool and providing a dependency bundle to merge into the package. If you’d like to contribute a similar script for your own favorite framework, let us know!

Disclosure: Jade Wang, a member of the Sandstorm project, also works for Meteor, but Sandstorm and Meteor are not affiliated.

Roundcube; and avoiding walled gardens

By Kenton Varda - 11 Aug 2014

Today we’re releasing our port of Roundcube, another open source e-mail web app. You can go install it from the app list now, either on your own Sandstorm server or on our demo.

To this you might ask: What happened to Mailpile?

Nothing happened. We support both! We like Mailpile, but we also want to give you choices. With Sandstorm, you can mix and match apps from competing developers as you see fit, while still retaining the unified experience of Sandstorm.

Let’s end walled gardens

Usually, when you sign up for a company’s suite of “cloud apps”, you have two choices:

1. Restrict yourself only to the apps that company provides and no others. This way you get a nicely integrated experience, but if one or more apps in the suite aren’t to your liking, you are stuck with them.
2. Also use apps from other developers. But you probably have to log into those apps separately, and they won’t integrate well with the first company’s suite. You end up with a fragmented experience in which you must remember multiple passwords and re-enter the same data into each service.

We call this the “walled garden” problem. Big SaaS (Software-as-a-Service) providers tend to build walled gardens of apps that only integrate with other apps from the same developer.

It would be easy and convenient for us to say that these companies are Evil and just want to lock you in, but that’s not true. The reality is that integration across security realms is hard. When each company has their own notion of users that can’t easily be mapped to each other, and understandably no company is willing to trust other companies with their security, how can you integrate apps containing private data? The answer in practice is OAuth, but OAuth permissions requests are clunky and disruptive to the UX. Some users are scared off by it – while others aren’t scared enough and just click “OK”.

But developers within a company are perfectly willing to trust other developers in the same company to get security right, and so the entire OAuth dance can be skipped in these cases. This makes first-party integrations easy. Now imagine what would happen if a single company’s own apps had to OAuth to each other before integration. Imagine, for example, if Google Docs had to ask you for permission to fetch your contact list from GMail before enabling auto-complete of contact names in the share dialog. Probably, Google Docs wouldn’t bother, and there would be no autocomplete. Or imagine if the Facebook profile page had to ask for permision to connect to your Facebook photos before you could set your profile photo. This would be a huge pain! And even then, it wouldn’t necessarily end the walled garden: with OAuth, an app says “I need permission to talk to GMail,” not “I need permission to talk to an e-mail app.” There’s still no opportunity for you to substitute a third-party app in its place.

With Sandstorm, we’re solving a lot of these problems:

• All your apps run under the same roof, whether they are from the same developer or competing developers. You can use a diverse set of apps without having to remember a separate password for each one.
• In place of OAuth, we’re building the Powerbox. This UI integrates better into the app’s UX by simultaneously acting as a picker and permissions granter. Apps usually need to present a picker anyway, so this is less disruptive. For example, if an app wants you to choose some contacts to which to send invitations, instead of first asking for permission to all your contacts just to display a picker UI back to you, the app will ask Sandstorm to display a Powerbox picker for contacts. Only the contacts you choose will be passed to the app. So, the UX is exactly the same, except without the separate permissions check, and yet it is more secure. The same idea would apply to choosing photos, or choosing anything else.
• When using the Powerbox UI, the requesting app does not name an exact receiving app it wants to access, but instead specifies a protocol. So instead of saying “I need access to Mailpile,” the app would say “I need access to an e-mail app,” and you could potentially choose Mailpile, Roundcube, or any other e-mail app. Apps do have to implement compatible protocols, which can be difficult, but this approach at least leaves open the possibility of a newcomer app emulating the protocols already used by other apps in the space, thereby breaking down the network effect that would otherwise prevent a new app from gaining a foothold.

We hope these design choices mean you get a unified experience no matter which apps you choose to use, and new players can break into the market more quickly, leading to faster innovation.

EtherCalc: Online spreadsheets

By Kenton Varda - 05 Aug 2014

Today’s release is a pleasant surprise even to us. Last week, a Sandstorm fan, Jake Weisz, e-mailed me out of the blue to tell me he had ported EtherCalc, an online collaborative spreadsheet editor, to Sandstorm.

I hadn’t heard of EtherCalc before, but Jake included a link to a spreadsheet running on our own alpha server, so I was able to try it out immediately, and I was impressed. Sure enough, it’s a spreadsheet editor, it does the things you expect spreadsheets to do, it supports real-time collaboration, and it’s Free Software.

Jake professes to have had no idea what he was doing. “I have never even seen Node.js before, I don’t even know JavaScript, and I have only a passing familiarity with using Linux.” Yet, by just following the porting guide, he managed to port an app. Jake attributes this to EtherCalc’s simple and elegant design: “The way it was already designed, it was incredibly easy to port to Sandstorm. I think I changed three or four lines of code tops to get it working.”

Speaking of EtherCalc’s design, developer Audrey Tang has written a long and fascinating history of EtherCalc (and its previous iterations, WikiCalc and SocialCalc) for the book series The Architecture of Open Source Applications. These chapters have also been reproduced on EtherCalc’s web site. Check it out!

Meanwhile, Jake’s port is on the Sandstorm app list now. As always, you can install it on your Sansdtorm server, or try the demo.

MediaGoblin: Photo sharing and more

By Kenton Varda - 01 Aug 2014

Today we’re releasing a port of yet another crowd-funded open source web app, MediaGoblin. You can think of MediaGoblin as a Free Software alternative to Flickr, SoundCloud, YouTube, and others.

The app is on the app list now – try it on the demo server.

MediaGoblin is our first app to take full advantage of Sandstorm’s new user identification features. Whenever a request comes in from a user, Sandstorm’s front-end proxy adds a few headers related to authentication and authorization:

• X-Sandstorm-Username: Specifies the user’s display name, e.g. “Kenton Varda”. This name is not unique nor stable; it is meant only for displaying to other users.
• X-Sandstorm-User-Id: A hex string identifying the user. This ID is both unique and stable, so it can be used to identify when the same user returns multiple times.
• X-Sandstorm-Permissions: A list of permissions that have been shared with this user via the Sandstorm sharing interface. An app can define what permissions are available to share in its package config. However, note that we have not yet implemented the full sharing UI, so for now the creator of the app instance is granted all permissions and everyone else is granted none. It’s up to the app to actually implement this permissions, so for now you can simply use it to distinguish between the owner and non-owners.

Our MediaGoblin port uses these headers to automatically populate MediaGoblin’s internal user list. This way you get MediaGoblin’s commenting and other multi-user features without the need to separately log into it.

Note that there are a few things missing from this port, which we plan to fix in the future:

• Currently, under the hood, apps are exposed only at ephemeral hostnames; the hostname may change every time you use the app. This is great for security but means that apps cannot expose APIs, since API clients want a persistent URL to talk to. Atom feeds are a kind of API. So, for now, we have disabled Atom feeds in our port. This is unfortunate, as it changes MediaGoblin from “federated” to “isolated”. That said, enabling APIs is high on our wish list of features, so expect this to return in the future.
• Related to the above, it’s currently not possible to deep-link into an app, since it runs in an iframe. So, sending someone an individual photo with MediaGoblin is hard. We can fix this by automatically passing through any additional path added to the outer app URL, and creating a postMessage()-based API for apps to tell the Sandstorm shell what the current path should be.
• Due to a limitation in our proxy implementation, we currently do not support very large file uploads. As a result, we had to disable video uploads for now. We plan to fix this soon.
• Currently, if you wish to share your MediaGoblin content, you must share the app instance by copy/pasting the URL. When another user visits that URL, your MediaGoblin server starts up to handle the request. This differs from Ghost and Hacker CMS which are able to publish web content in such a way that the server does not need to start up for every visitor, only for the editor. Unfortunately, this means that a widely-shared MediaGoblin instance may be expensive as it may have to run frequently; Sandstorm normally tries to cut costs by only running servers a small fraction of the time. We’d like to change the app to take advantage of Sandstorm’s web publishing features like Ghost and Hacker CMS do, so that the server only needs to run when a visitor posts a comment, uploads media, or otherwise makes a change. However, this may require some invasive changes to MediaGoblin, so we’ll need to discuss them with MediaGoblin’s developers.
• MediaGoblin implements a number of permissions which can be granted to users, such as permissions to comment, upload media, etc. We eventually want permissions to be set using Sandstorm’s unified sharing model, but as described above it is not yet complete. So, for the time being, you have to set users’ permissions inside the MediaGoblin user management interface.

All that said, despite some missing features, Sandstorm is by far the quickest way to get MediaGoblin up and running. We hope our port allows this great piece of Free Software to reach a wider audience than it could before.

And if you’d like to see the missing features fixed in the future, be sure to fund our Indiegogo campaign. :)