Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network

Joon Ong
Amit Agarwal
Glen Anderson
Ashby Armistead
Roy Bannon
Seb Boving
Gaurav Desai
Paulie Germano
Jeff Provost
Jason Simmons
Eiichi Tanda
Jim Wanderer
Amin Vahdat
Sigcomm '15, Google Inc (2015)
Google Scholar

Abstract

We present our approach for overcoming the cost, operational complexity, and limited scale endemic to datacenter networks a decade ago. Three themes unify the five generations of datacenter networks detailed in this paper. First, multi-stage Clos topologies built from commodity switch silicon can support cost-effective deployment of building-scale networks. Second, much of the general, but complex, decentralized network routing and management protocols supporting arbitrary deployment scenarios were overkill for single-operator, pre-planned datacenter networks. We built a centralized control mechanism based on a global configuration pushed to all datacenter switches. Third, modular hardware design coupled with simple, robust software allowed our design to also support inter-cluster and wide-area networks. Our datacenter networks run at dozens of sites across the planet, scaling in capacity by 100x over ten years to more than 1Pbps of bisection bandwidth.

Research Areas