How self-hosted analytics preserve user privacy
By Asheesh Laroia - 12 May 2016
Filippo Valsorda just published an overview of why to use Piwik for web analytics instead of Google’s product. He reflects on the fact that he saw 18% more pageviews with Piwik compared to Google Analytics, seemingly because self-hosted analytics get past ad-blocking browser extensions. I want to reflect on the privacy implications of running your own analytics and explain the nuts and bolts of how Piwik runs on Sandstorm.
Here’s what Piwik on Sandstorm looks like.
How a web analytics service works
For a website (let’s say example.com) that that embeds Google Analytics’ Javascript tracking code, a web browser like Chrome takes the following steps:
-
Download the HTML content of example.com.
-
Notice a SCRIPT or IMG tag going to Google Analytics.
-
Download the Google Analytics tracking code from Google, whose purpose is to tell Google Analytics the URL of the page that included it. One main purpose of web analytics is to see a chart of which pages are popular, so it stands to reason that if Google runs the analytics tool, Google needs to know what page is being viewed.
Google has a strong privacy policy, and this information typically falls under the heading of “Information we get from your use of our services,” and I am personally OK with sharing this information with Google Analytics.
Some people like staying private
According to W3Techs, more than 50% of all websites use Google Analytics. This has a somewhat surprising result: Google’s analytics service knows about a huge fraction of all web page visits. After all, any central analytics service can see the webpage visits for all the sites using it. They can even see traffic patterns that each individual site operator cannot.
Some web surfers want to keep their surfing information private. They may be OK with the site owner knowing that someone visited, but Google is not the site owner. They may see data as a toxic asset, per Bruce Schneier. Many of them use web browser add-ons like Privacy Badger.
As a website operator, this puts you in a bind. If you want to use Google Analytics to get a graph of which pages are popular, you need to share your users’ surfing patterns with Google. Google doesn’t distribute Google Analytics as software you can run; they only run it as a central service.
A different way to do web analytics is to use software running on a server that you maintain. This is called self-hosting web analytics.
The easiest way to do that is to do what Filippo Valsorda said: run a Sandstorm server and install Piwik from the app market, and copy-paste something into your website.
Today, that’s possible with a few presses on a touchpad. Getting to that point involved some fascinating challenges.
How to share an API token you cannot see
One design goal of Sandstorm is that apps are confined. An app in Sandstorm never learns a URL that outsiders can use to reach it. That way, the app must rely on Sandstorm for access control, which means the “Share access” button can always show the users who has access. Sandstorm can provide one interface for granting and revoking access across all apps.
However, Piwik needs a way to show a URL that a website operator can embed into their website, so that website visitors can be tracked by Piwik!
To get past this Catch-22, Drew Fisher built a new Sandstorm feature called offer templates. Piwik asks Sandstorm to show a templated message, and an IFRAME from Sandstorm appears with that token placed into the templated message. Drew’s Piwik package was the first to use it, and it’s become the most common way for apps on Sandstorm to create copy-pastable instructions for how to connect to their APIs.
When to make IP addresses available
Another obstacle was IP addresses. Piwik needs to know the IP address of visitors so it can create a visual map to show website operators where in the world their visitors come from. However, we didn’t want every Sandstorm app to be able to collect this data without user intervention. Drew added a way for client-side Javascript to optionally share the user’s IP address, while keeping the defaults safe. I’ll quote him here:
Normally, we strip the remote address from requests, since most applications shouldn't need it. However, for those that benefit from it (like analytics), clients can opt into passing their IP on to the backend by adding an "X-Sandstorm-Passthrough: address" header to their request. This would be a privacy leak for WebSession, since the grain can give the client scripts which would send the header, but ApiSession requires a user action, so it's safe here.
In the context of the above, an ApiSession covers requests that come in via an app’s HTTP APIs such as Piwik’s tracking system, and a WebSession covers requests that come in from the grain owner clicking around, such as a site owner viewing their statistics.
Enabling the community
When we finished, we had built a foundation that other apps could build upon. You can try Hummingbird, Michael Nutt’s real-time analytics tool, and Radicale, Aleksandr Bogdanov’s package that enables calendar and contact synchronization by combining a few tools.
If you want to make an open source web app that anyone can self-host safely, start at the Sandstorm Developer Hub. Sandstorm supports any programming language or stack, like PHP or Meteor or Rust, that runs on Linux. You’ll find a packaging tutorial and detailed information about offer templates. And if you just want to start using Piwik, check it out on the app market.
Sandstorm lets university IT be the hero
By Asheesh Laroia - 06 May 2016
Picture yourself running a university IT helpdesk. When professors come to you, you point to the tech tools your colleagues have already deployed. It takes a long process to get something new approved. You know that’s been limiting professors’ creativity, but you haven’t known what to do about it.
Then one day, Sandstorm comes along.
Now educators are empowered to help themselves. They can make course websites with Ghost, teach students how to typeset documents using ShareLaTeX, or create shared folders for classes using Davros. They can even use GrooveBasin to set up a collaborative radio station for a music history course.
All that happens on your Sandstorm server, running safely within the institution.
Can you help spread the word?
Néna Nguyễn just finished designing a page with the information educators need about Sandstorm. I’m writing because I need your help reaching educators and technologists to share it with.
Can you think of an IT staffer or professor who is wondering how to get access to a wider variety of software available at their institution? If so, please send them a link to https://sandstorm.io/go/education. I just sent it to the head of IT at my alma mater’s CS department.
If they ask you about security or privacy, you can tell them that Sandstorm is self-hostable open source software, integrates with LDAP/SAML single sign-on, and automatically mitigates 95% of security issues, before they are even discovered. For more on Sandstorm’s security design, read our security practices documentation.
Thanks!
Sandstorm for Work Beta: LDAP, SAML, organization management
By Kenton Varda - 06 Apr 2016
Sandstorm is a great way to run open source collaborative productivity apps like document editors, task managers, chat rooms, file sharing, and more (54 apps and growing), all in one place. Lots of people – including us – use Sandstorm every day as part of doing their jobs. For example, right now I am composing this blog post in Dillinger running on Sandstorm Oasis (our hosting service), but many people choose to self-host Sandstorm on their own machines.
Sandstorm for Work makes self-hosted Sandstorm easier to integrate into a corporate environment. That means you can:
- Integrate with LDAP (including Active Directory) or SAML for single-sign-on.
- Manage user groups and domains as whole units rather than inviting each user individually.
- Specify organization-wide access control policies, like “no sharing outside of the organization”.
- And more…
We’re in beta now, so not all features are ready yet, but once you’ve installed Sandstorm you’ll automatically receive updates as they become available.
To get started now, install Sandstorm and choose “Sandstorm for Work” during setup (or if you already have a server, look for the “for Work” tab in the admin settings). Sandstorm for Work is priced at $15/user/month, and currently we’re offering a 90-day free trial. (It’s still open source! More on that in a bit.)
Productivity Apps Large and Small
With Sandstorm for Work, you can run apps similar to popular SaaS products, but keep your data in-house. For example, you can run Etherpad and EtherCalc instead of Google Docs, Wekan instead of Trello, Rocket.Chat instead of Slack, and Davros instead of Dropbox. Once you have Sandstorm running, you can install each new app with a click – no need to create a new account on a new service, and no worry about whether that service is sufficiently private and secure. With Sandstorm, running apps in-house is actually easier than using SaaS.
But even more interestingly, you can run apps on Sandstorm that aren’t available anywhere else, like Simon Vansintjan’s Annotate. The idea behind Annotate is simple: upload an image, then annotate it with comments. We at Sandstorm use it to discuss UI and illustration mockups. Here’s me using it to comment on a slide deck we’re working on:
What I love about Annotate is its simplicity: Simon wrote the whole thing in a couple weekends. And yet, it’s enough to be useful to design teams anywhere. No matter what Simon decides to do in the future, this app will never disappear, and you don’t have to trust Simon because your data stays on your server.
Annotate is just one of 54 apps and growing on our app market. By installing Sandstorm at your workplace, you get access to all of these at once, under a single login, and without giving up control of your data.
Security Behind the Firewall
Of course, if you want to run apps on your own infrastructure, behind your firewall, you need to think about security. A malicious app – or a buggy app that gets hacked – can compromise your whole network.
Fortunately, Sandstorm protects you. Sandstorm is the only container engine that implements fine-grained isolation, locking every document in its own container. By doing so, it prevents any app from compromising the server or network, and indeed it renders 95% of app security vulnerabilities moot before they are even discovered.
Developing for Enterprise
Are you a developer of a web app aimed at enterprise? Do you ever get requests for an on-prem version, but find it hard to fulfill this request given the myriad environments and infrastructure you’d need to support? Perhaps we can help. If you target Sandstorm, then these logistics become our job. If your app works on one Sandstorm server, it will work everywhere. If you are interested in learning more, check out our developer features and e-mail us.
Is it Open Source?
Yes! Sandstorm for Work features are part of the same codebase as the rest of Sandstorm and under the same Apache 2.0 license. However, in order to unlock Sandstorm for Work features, we ask that you buy a “feature key” from us.
So how does that work? Can’t anyone just remove the feature key check? In fact, yes, you can. However, if you did that, you would not be able to take advantage of our automatic updater, which ensures that your server is updated to the latest version within 24 hours of any release with no effort on your part. Automatic updates are important to keep your server secure and to make sure you can always run the latest apps.
Purchasing a feature key also entitles you to priority support. However, we don’t want to be a company who primarily sells support, because we think that creates a perverse incentive for us to make our product hard to use. Indeed, it is our goal that no one should ever need to contact support at all, but under a support model, we’d be putting ourselves out of business! Under the feature key model, we are selling features, and promising support if there are problems. This way, we are incentivized to make sure there are no problems, because then we don’t have to answer support tickets.
The Future
We like to release features early and often – we push a new release almost every week. What we are announcing today is only the beginning of what we have in store for Sandstorm for Work. Over the coming months, we’ll be adding features like group management (to make it easier to share documents with your team), audit logging (keep track of who has been accessing what, for security and compliance purposes), customizable access control policies, and much more.
If you install Sandstorm today, you’ll automatically get these features as they become available – your server will automatically update after every weekly release with no action needed on your part. So why not install now and see what you think? Feel free to file a bug to tell us what you want to see next.
Sandstorm's security track record, and what it means for self-hosting
By Asheesh Laroia - 29 Feb 2016
Today I want to share the results of our own analysis of security issues of web apps available on the Sandstorm app market.
95% of security issues automatically mitigated, before they were discovered
Sandstorm automatically protects users from a huge fraction of the publicly disclosed security vulnerabilities discovered in apps on the Sandstorm app market, before the vulnerabilities were even disclosed. Of the issues we examined, 95% were wholly or partly mitigated. You can read the full report here in our documentation. The analysis covers publicly-disclosed vulnerabilities in Etherpad, WordPress, Roundcube, ShareLaTeX, and Tiny Tiny RSS. In WordPress, we limited our analysis to security issues of severity score 6 or higher, due to the large number of issues. We also mitigated 21 CVEs in the Linux kernel to prevent sandbox breakout.
We built Sandstorm to create a viable ecosystem for indie and open source web apps. When server apps are as safe to run as apps on a phone, people will feel free to choose whatever software they like. Consider that some Sandstorm apps, like Giftr, are small and don’t have as many people checking the code for bugs. Sandstorm protects you when you use those apps, too.
We know that security is risk-management, not binary. No software, Sandstorm included, will ever protect all user data from all bugs in all programs. However, raising barriers to a successful attack means fewer successful attacks will occur.
Self-hosted apps can be as secure as a centralized web app
With Sandstorm, you get an experience as easy to use as software-as-a-service, and you retain the privacy benefits of self-hosting. One of our key security strategies is to isolate each grain (typically, one document) separately, so that a buggy or malicious app has a hard time ruining your day. That degree of isolation is enabled by our various security practices.
I hope you’ll read the full analysis, prepared by myself and Kenton Varda. Let us know what you think!
When self-hosting is secure, users are free to choose
Security enables freedom of choice. If you use a Sandstorm server, you can choose productivity tools that fit your needs, even if the server is maintained by someone else.
Want to chat with colleagues? Install Rocket.Chat or Let’s Chat. Want to track tasks and stay organized? Install WeKan or Simple Todos. Want to organize a gift exchange? Install Giftr. Want to share files quickly with friends? Install FileDrop or Davros.
If you prefer managed hosting, you can make an account on Oasis and enjoy any of these apps or upload your own.
Or you can run your own Sandstorm install. Over the past six months, we’ve integrated free SSL certificates and cryptographically-verified automatic updates into self-hosted Sandstorm. As soon as someone installs Sandstorm for an organization, they can safely allow colleagues to choose their own tools. If you install Sandstorm for yourself, you can use the best indie web apps and let the platform handle security for you. Get started on the Sandstorm install page.
Andrew Wansley shares his story of Giftr
By Nena Nguyen - 25 Feb 2016
Giftr is one of the newest additions to our App Market; it helps make coordinating gift exchanges less painful for you and other gifters involved. At our Sandstorm San Francisco meetup in January, app author Andrew Wansley shared his story of how his decision to create & package his Giftr app all started from a simple family tradition.
Watch Andrew’s talk here:
Try Giftr & remember to review it on the App Market!