Ever wonder how Google services like Image Search and Photos are able to achieve such high performance? Google’s secret sauce is its custom microchip technology, which allows it to perform AI tasks like object recognition at high speeds. In fact, Google’s Tensor Processing Unit (TPU) chips can operate about 30 times faster than alternative approaches using general-purpose processors, the company claims.
Although Google has been utilizing TPUs in its data center since 2015, the company only recently began revealing details about its chip technology, as reported by TechCrunch. Google in April released performance specifications that showed its TPUs offer dramatically superior performance in deep neural network processing compared to running software algorithms on conventional systems that employ central processing units (CPUs) or graphics processing units (GPUs).
“The need for TPUs really emerged about six years ago, when we started using computationally expensive deep learning models in more and more places throughout our products,” Google explained in a blog. “The computational expense of using these models had us worried. If we considered a scenario where people use Google voice search for just three minutes a day and we ran deep neural nets for our speech recognition system on the processing units we were using, we would have had to double the number of Google data centers!”
Instead of initiating this massive construction project, Google developed the TPUs, which are based on application specific integrated circuits (ASICs) that are optimized to run the company’s TensorFlow machine-learning framework. The ASICs are built on a 65,536 8-bit multiplier–accumulator (MAC) unit that boasts a peak throughput of 92 TeraOps/second (TOPS), Google said.
In addition to running 15 to 30 times faster than a GPU or CPU, the TPUs deliver TOPS-per-watt performance as much as 80 times faster as the general-purpose processor. The TOPS-per-watt performance is particularly important as data centers continue to gobble more energy.
Tyler Schulze is vice president, strategy & development at Veritone. He serves as general manager for developer partnerships, cognitive engine ecosystem, and media ingestion for the Veritone Platform. Learn more about our platform and join the Veritone developer ecosystem today.