MirrorBrain, Our New Download Mirror System

Improving Users Download Experience

As mentioned in one of my previous posts, as the site has grown in popularity so has our bandwidth requirements. Previously builds were served from a single server and it was seeing more traffic than I had ever really intended. As the user base has expanded, I now see visitors from all over the world – so again the single server didn’t provide the best experience and I was getting numerous requests asking if I could provide faster download links.

To improve on this situation I put in place a number of download mirrors, we now have downloads mirrors in the UK, France and US locations (Atlanta, New York, Dallas and Seattle). Links to each of these mirrors were provided with each download. Some users clicked their closest mirror, however the majority of users still clicked the default download link. So whilst there was a slight improvement, the download mirrors weren’t fully being utilised. Likewise there is a number of XBMC addons that use my builds and they still pointed at the default link – so again these weren’t taking advantage of the download mirrors. Clearly there needed to be a better solution….

Introducing MirrorBrain

I looked a several options for how to best handle mirrors and ensure that users grabbed their download from their nearest mirror. We use MirrorBrain on the main XBMC Download Mirror Site, so rather than reinventing the wheel it seemed more logical to replicate the setup we use there.

MirrorBrain is a feature rich open-source geographic based load-balancer. One of its key features, is that MirrorBrain monitors each mirror server, both at a network and file level – providing a self healing delivery system. If a mirror is having issues, it is disabled automatically, and requests will be routed to its closest available mirror. As soon as it recovers it will be marked and available again and put online.

Whilst not perfect, MirrorBrain is a reasonably robust and easily scalable solution for providing mirrors. It will automatically redirect a users download to the server which is closest to them and contains the file. In the event of no mirrors having the file, it will revert back to serving from itself. Better yet, only one link is needed to be used – so everyone (including addons) will be forced to use the mirrors. You can visit the new MirrorBrain based download site here: mirrors.xbmcnightlybuilds.com you can also find the current list of mirrors. You will soon see new builds using this link (you can manually select an alternative mirror should you want to).

MirrorBrain is used by plenty of other open-source projects – XBMC, VideoLan, openSUSE, GNOME, KDE, The Document Foundation (LibreOffice) and many more. Some of these sites are serving 10′s of  terabytes of data every day. If it works for them, then its more than capable of handling the usage it will get here.

Another reason for choosing MirroBrain is that it allows me to easily add additional third-party mirrors – which was more difficult than it needed to be previously.

Can you provide a mirror?

We are now looking to add mirrors to our our MirrorBrain setup. If you would like to help and become an official mirror, please get in touch a leave a comment below. One of the great features of MirrorBrain is mirror sites do not need to provide a full mirror – so do not need lots of space. If you only want to provide RaspberryPi builds or Virtual builds for example then thats completely possible with this system.