Lessons Learned Building Distributed Systems at bitly

Distributed systems can often be challenging to build and operate, but they can offer significant benefits in terms of availability, cost effectiveness, and capacity. Over the years bitly has built a number of large scale systems to handle and analyze billions of clicks each month.

In this talk we'll be covering:

  • What are distributed systems?
  • What makes building distributed systems hard?
  • What makes distributed systems awesome?
  • How does one deal with the challenges of distributed systems and reap the benefits?

Some of the specific topics to be covered include the benefits of Service Oriented Architecture, using asynchronous streams, scaling, dealing with failure, and monitoring.

Overall this talk will focus on fundamental concepts and techniques, but will pull in specific tools and examples for illustration purposes.

  • Sean O'Connor is the Lead Application Developer at bitly. Day to day he builds systems, reviews code, and works with the awesome crew at bitly to create powerful new tools to help people understand the social web. Outside of bitly Sean spends most of his time biking around NYC, brewing cider, and keeping his dog out of trouble.