Speaker

Ben Reed, Yahoo!

Abstract

Yahoo! has a very diverse set of offerings from niche applications that require a few servers to search which involves thousands of machines. Application developers need tools to coordinate the processes running across machines and recover from failures. In this context we developed a service called ZooKeeper to allow developers to easily coordinate their processes. The service interface we came up with incorporates ideas from a diverse set of previous work and brings them together in a very simple API. So simple, in fact, that we were surprised at its power. ZooKeeper has become a key tool for coordination at Yahoo! In this talk I will discuss the observations and motivations of ZooKeeper's design, some of the interesting results of our design, how ZooKeeper is used, interesting performance aspects, and future work left to do for ZooKeeper. In the process attendees will also get some insights to life at a big Internet company and life at an industrial research lab.