Bring the Noise: Continuously Deploying under a Hailstorm of Metrics
The best and worst thing about Etsy's StatsD is that it lets you measure everything and anything. But what happens when you end up with 250k metrics, up and down the stack, and you actually want to know when one of them misbehaves? Dashboards don't quite cut it at that scale, and when you deploy thirty times a day, you want to be sure you aren't breaking anything unexpected when you push.
To safely change the tires on Etsy while the site is barreling along at full speed, we have a cocktail of deployment tools and processes that I'd like to tell you about in this talk. We'll go over how we do realtime anomaly detection, realtime error logging, massive-scale operational metrics correlations, and all the other ways we manage to deploy so often with no branches and 200+ committers, each with the power to push to production with the click of a button.
Abe is a data engineer at Etsy, where he builds foundational systems and tools for the Etsy Data stack. In 2011, he founded Hacker League - the easiest way to power innovation through internal hackathons. You might know him as the irreverent creator of commitlogsfromlastnight.com. He's pretty sure he didn't sign up for any of this while he studied for his BA in philosophy.