The revolution in Deep Learning, initially fueled by GPUs, has inspired a new generation of computer architectures, including Google's TPU, Microsoft's Project BrainWave, and a huge number of startups for both inference and training. But the field lacks consistent benchmarks, making useful, let alone fair, comparisons difficult. MLPerf began as a collaboration among researchers at Baidu, Google, Harvard, and Stanford based on their earlier experiences with benchmarking. The group has grown to include dozens of companies and universities and hundreds of participants. This talk will describe the philosophy and methodology behind MLPerf, and the kinds of things we expect to measure in the first set of results this Fall.

Cliff Young is a software engineer in the Google Brain team, where he works on codesign for deep learning accelerators. He is one of the designers of Google's Tensor Processing Unit (TPU), which is used in production applications including Search, Maps, Photos, and Translate. TPUs also powered AlphaGo's historic 4-1 victory over Go champion Lee Sedol. Previously, Cliff built special-purpose supercomputers for molecular dynamics at D. E. Shaw Research and worked at Bell Labs. Cliff holds AB, MS, and PhD degrees in computer science from Harvard University.