Boing Boing 3.0, and Ask the Sysadmin!

Boing Boing 3.0

The new Boing Boing infrastructure is going to implement several new advancements that have come up since 2008, which is when I built the existing setup. Most of the gear was leftover from a migration contract I handled for Federated Media, where they paid me partially in equipment, which leads to some strange archetecture decisions (like 15K drives instead of SSDs!

This is still a work in progress!

Servers:

  • 6 HP DL360p G7 servers, dual-hexa-core Xeons, 64GB RAM, 6 15k drives in RAID10, 1 hot spare, configured as follows:
  • 2 web front-ends with 64GB RAM, 6 15k drives in RAID10, 1 hot spare
  • 1 admin server (for SSH Agent Forwarding, as well as Wordpress admin interface for the Boingers)
  • 1 “tools” server for monitoring, backup, misc admin functions
  • 2 DB servers. Unlke the rest, they’re configured with 4 SSDs, and 128GB Ram.

Software:

  • We’ll be running Red Hat Enterprise Linux 7, including it’s pretty awesome support for docker-latest
  • Database will be Percona Server 5.7
  • Monitoring via Icinga 2, munin, and PMM for the database
  • Backups using Duplicity (offsite with Vaultpress)
  • Config management for the whole thing using the super-awesome Ansible
  • Shared storage using either GlusterFS or Ceph, I haven’t decided yet
  • Encryption via LUKS and encrypted OwnCloud (Can’t wait for client-side encryption for the latter!)
  • LVS/HAProxy load-balacing with Let’s Encrypt automated HTTPS certs for origins

CDN setup - We’ll continue to use Fastly as our reverse proxy CDN talking to our origins, however we’re adding two more layers to the mix:

  • Wordpress Photon for image caching, and
  • Google has graciously offered us to have access to Project Shield for DDoS protection, in front of everything

Whew! So yeah, that’s my current project. :slight_smile:

15 Likes