The SoC is the new motherboard.
Data centres are no longer betting on the one-size-fits-all compute. Decades of homogenous compute strategies are disrupted by the need to optimise. Modern-day data centres are embracing purpose-built System on Chip (SoC) designs to have more control over peak performance, optimise power consumption and scalability. Thus, customisation of chips has become the go-to solution for many cloud providers. Companies like Google Cloud especially are doubling down on this front.
Google introduced the Tensor Processing Unit (TPU) back in 2015. Today TPUs power services such as real-time voice search, photo object recognition, and interactive language translation. TPUs drive DeepMinds powerful AlphaGo algorithms, which outclassed the worlds best Go player. They were later used for Chess and Shogi. Today, TPUs have the power to process over 100 million photos a day. Most importantly, TPUs are also used for Googles search results. The search giant even unveiled OpenTitan, the first open-source silicon root-of-trust project. The companys custom hardware solutions range from SSDs, to hard drives, network switches, and network interface cardsoften in deep collaboration with external partners.
Workloads demand even deeper integration into the underlying hardware.
Just like on a motherboard, CPUs and TPUs come from different sources. A Google data centre consists of thousands of server machines connected to a local network. Google designs custom chips, including a hardware security chip currently being deployed on both servers and peripherals. According to Google Cloud, these chips allow them to securely identify and authenticate legitimate Google devices at the hardware level.
According to the team at GCP, computing at Google is at a critical inflection point. Instead of integrating components on a motherboard, Google focuses more on SoC designs where multiple functions sit on the same chip or on multiple chips inside one package. The company even claimed that the System on Chips is the modern-day motherboard.
To date, writes Amin Vahdat of GCP, the motherboard has been the integration point, where CPUs, networking, storage devices, custom accelerators, memory, all from different vendors blended into an optimised system. However, the cloud providers, especially companies like Google Cloud, AWS which own large data centres, gravitate towards deeper integration in the underlying hardware to gain higher performance at lesser power consumption.
According to ARM acquired by NVIDIA recently renewed interest towards design freedom and system optimisation has led to higher compute utilisation, improved performance-power ratios, and the ability to get more out of a physical datacenter.
For example, AWS Graviton2 instances, using the Arm Neoverse N1 platform, deliver up to 40 percent better price-performance over the previous x86-based instances at a 20 percent lower price. Silicon solutions such as Amperes Altra are designed to deliver performance-per-watt, flexibility, and scalability their customers demand.
The capabilities of cloud instances rely on the underlying architectures and microarchitectures that power the hardware.
Amazon has made its silicon ambitions obvious as early as 2015. Amazon acquired Israel-based Annapurna Labs, known for networking-focused Arm SoCs. Amazon leveraged Annapurna Labs tech to build a custom Arm server-grade chipGraviton2. After its release, Graviton2 locked horns with Intel and AMD, the data centre chip industrys major players. While the Graviton2 instance offered 64 physical cores, AMD or Intel could manage only 32 physical cores.
Last year, AWS even launched custom-built AWS Inferentia chips for the hardware specialisation department. Inferentias performance convinced AWS to deploy them for their popular Alexa services that require state of the art ML for speech processing and other tasks.
Amazons popular EC2 instances are now powered by AWS Inferentia chips that can deliver up to 30% higher throughput and up to 45% lower cost per inference. Whereas, Amazon EC2 F1 instances use FPGAs to enable delivery of custom hardware accelerations. F1 instances are easy to program and come with an FPGA Developer AMI and support hardware level development on the cloud. Examples of target applications that can benefit from F1 instance acceleration include genomics, search/analytics, image and video processing, network security, electronic design automation (EDA), image and file compression and big data analytics.
Followed by AWS Inferentias success in providing customers with high-performance ML inference at the lowest cost in the cloud, AWS is launching Trainium to address the shortcomings of Inferentia. The Trainium chip is specifically optimised for deep learning training workloads for applications including image classification, semantic search, translation, voice recognition, natural language processing and recommendation engines.
The above table is a performance comparison by Anandtech, which shows how the cloud providers can ditch the legacy chip makers, thanks to ARMs license provisions. Even Microsoft is reportedly building an ARM-based processor for Azure data centres. Apart from custom chips thats under wraps, Microsoft too had a shot at silicon success. They have collaborated with AMD, Intel, and Qualcomm Technologies and announced the Microsoft Pluton security processor. The Pluton design builds security directly into the CPU.
To overcome the challenges and realise the opportunities presented by semiconductor densities and capabilities, electronic product cloud companies will look into System-on-a-Chip (SoC) design methodologies of incorporating pre-designed components, also called SoC Intellectual Property (SoC-IP), which can then be integrated into their own algorithms. As SoCs incorporate processors that allow customisation in the layers of software as well as in the hardware around the processors is the reason why even Google Cloud is bullish on this. They even roped in Intel veteran Uri Frank to lead their server chip design efforts. According to Amin Vahdata, VP, GCP, SoCs offer many orders of magnitude better performance with greatly reduced power and cost compared to assembling individual ASICs on a motherboard. The future of cloud infrastructure is bright, and its changing fast, said Vahdat.
- AI use cases are expanding and evolving in healthcare - Tech Wire Asia - August 1st, 2021
- DeepMind releases massive database of 3D protein structures - STAT - STAT - July 25th, 2021
- DeepMinds AI uncovers structure of 98.5 per cent of human proteins - News Nation USA - July 25th, 2021
- Good Morning, News: $600000 Settlement for 2017 PPB Killing, Deep Pockets Try Influencing MultCo DA, and Everything is GREAT at the Olympics! - The... - July 25th, 2021
- DeepMind open-sources AlphaFold 2 for protein structure predictions - VentureBeat - July 21st, 2021
- DeepMind's AlphaFold 2 reveal: Convolutions are out, attention is in - ZDNet - July 21st, 2021
- AI in Healthcare Market Growing Trade Among Emerging Economies Opening New Opportunities (2021-2031) | Nuance Communications, Inc., DeepMind... - July 21st, 2021
- Self-driving data centres: Managing the transition from human-to-AI workload management - IT Brief Australia - July 21st, 2021
- The scene is set | Royal St George's | The 149th Open - The Open - July 14th, 2021
- Tech Companies Leading the Cognitive Computing Race in 2021 - Analytics Insight - July 14th, 2021
- Science fiction: From hero's tales to deep thoughts - The Daily Advance - July 14th, 2021
- ADOT urges drivers to be prepared for monsoon weather dangers | Arizona Emergency information Network - az.gov - July 14th, 2021
- Taking the long view, taking control - Cape Cod Times - July 14th, 2021
- The ultimate reading list if you love to delve into the mind - Livemint - July 14th, 2021
- Midland Theatre executive director Anderson to retire Sept. 1 - The Newark Advocate - July 14th, 2021
- The Future of Deep Learning | - Analytics Insight - July 10th, 2021
- The movie you should watch this weekend: Eternal Sunshine of the Spotless Mind, the heart wants what it wants - The Indian Express - July 10th, 2021
- From the Extension: Keep health and safety in mind when firing up the grill - Daily Commercial - July 10th, 2021
- Follow your fever dreams with Nashville Hot Hot Dogs - The Takeout - July 10th, 2021
- Former Bureaucrats, Civil Society Express Deep Anguish On Violations and Breakdown of Governance in UP - The Citizen - July 10th, 2021
- The 36 questions that lead to love (but with your co-founder) - TechCrunch - July 10th, 2021
- Googles Supermodel: DeepMind Perceiver is a step on the road to an AI machine that could process anything... - ZDNet - July 8th, 2021
- At the ICA Watershed, artist Firelei Bez draws history from the deep - The Boston Globe - July 8th, 2021
- The Streaming Age Has Turned Poland Into a Deep-Pocketed Production Paradise - Hollywood Reporter - July 8th, 2021
- What Toxic Positivity Actually Is (& Why You Should Avoid It) - mindbodygreen.com - July 8th, 2021
- UCF Lake Nona Scientific Heart Is The First Sanatorium In Central Florida To Be offering Incisionless Mind Surgical treatment As An Complex Remedy For... - July 8th, 2021
- Young Cho's goals are even bigger than deep space; His team is working on astronaut nutrition that translates to earth - Westside Seattle - July 8th, 2021
- Two books to make you think differently about the ocean and the beach - Christian Science Monitor - July 8th, 2021
- Jimmy McGovern: Even the psychopaths are a deep, horrible part of me - Sydney Morning Herald - July 8th, 2021
- 'I took a deep breath and got on with it' - Jorginho on his ice-cold winning penalty and Italy's belief | Official Site | Chelsea Football Club -... - July 8th, 2021
- Tour de France: Geraint Thomas to ride on through pain with Olympics in mind - The Guardian - July 8th, 2021
- Poetry Amid Parkinsons - The Wall Street Journal - July 8th, 2021
- Quinlan-Hammond Hall of Honor Taking Shape on USM Campus - Southern Miss Now - July 8th, 2021
- Lynn Hershman Leeson: The Artist Is Prescient - The New York Times - July 8th, 2021
- The Great Artist: A film that explores the brilliance of a suffering mind - The Indian Express - July 8th, 2021
- 10 UK retreats to re-energise mind, body and soul - The Guardian - June 27th, 2021
- Keep safety in mind when cooling off in the Willamette River - KGW.com - June 27th, 2021
- Matters of the Mind: Recruiting the body mind and soul for relief from emotional tumult - The Indian Express - June 27th, 2021
- 'You weep and then you try to determine how to help': Families, worshippers cling to hope after Florida condo building collapse - USA TODAY - June 27th, 2021
- Danielle Lao Gets Back to Wimbledon, and Makes the Most of It - The New York Times - June 27th, 2021
- Bernie Rabik: Is Age Only a Number? - The Times - June 27th, 2021
- New sales platform brand In Mind Cloud to rapidly digitalize the manufacturing sector - PRNewswire - June 25th, 2021
- Ophthalmology: A pioneer in the field of artificial intelligence - Ophthalmology Times - June 25th, 2021
- A Deep Dive into the Scoring Metrics of the NTIA's Broadband Infrastructure Program - Benton Foundation - June 25th, 2021
- Don't mind the gap: 'Innovation cohesion' is new route to bridge east-west divide - Science Business - June 25th, 2021
- A menu for the mind: Angelus reading recommendations 2021 - Angelus News - June 25th, 2021
- Finding Relaxation: Your 5-Minute Read - Healthline - June 25th, 2021
- DeepMind wants to use its AI to cure neglected diseases - Wired.co.uk - June 23rd, 2021
- Shame in Medical Training, and Owning Our Stories - Medscape - June 23rd, 2021
- Are toxins flushed out of the brain during sleep? - Harvard Health - June 23rd, 2021
- Thinking Differently: The Most Important Thing You Need to do to Advance Your Company - insideARM.com - June 23rd, 2021
- Vegan pop-up Don't Be Chick'n launches a Bay Area food truck - The Oaklandside - June 23rd, 2021
- Boosting inspection performance with AI and deep learning - Packaging Europe - June 22nd, 2021
- Time for a Casual Deep Dive on Karl From The Bachelorette - Yahoo Lifestyle - June 22nd, 2021
- AI in Healthcare Market 2021 Best Workable Strategy That Will Help to Boost your industry Growth Till 2031 | DeepMind Technologies Limited, IBM... - June 22nd, 2021
- International Yoga Day 2021: The Benefits Of Deep Breathing And Meditation For Diabetics - NDTV - June 22nd, 2021
- International Yoga Day: Does yoga really help heal your body and mind? - Gulf News - June 22nd, 2021
- Public Lands Preserve More Than What's on the Surface - Sierra Magazine - June 22nd, 2021
- Does alcohol have a place in a healthy lifestyle? - Walla Walla Union-Bulletin - June 22nd, 2021
- Dannals, Fisher, and Kyung Receive 2021 Tuck Teaching Excellence Awards - Tuck School - June 22nd, 2021
- Emergency and survival items you need to put in your pack - Part 2 - goHUNT.com - June 19th, 2021
- Studying content moderation, disinformation, and deep fakes. Astroturfing, bias, and rumor control. - The CyberWire - June 19th, 2021
- From The Deep Sea, A Chance To Learn, Preserve on the Gulf Coast - Mississippi Free Press - June 19th, 2021
- India Inc can delve deep within and construct its own reality: Deepak Chopra to ETILC members - Economic Times - June 19th, 2021
- Investors say Eindhoven poised to become Netherlands No. 2 tech hub - TechCrunch - June 19th, 2021
- What should you read in the summer of 2021? Here are the best books. - Berkeleyside - June 19th, 2021
- Inside the elite UK tech event attended by the rich and famous - CNBC - June 19th, 2021
- Inside the fight to reclaim AI from Big Techs control - MIT Technology Review - June 19th, 2021
- International Yoga Day: 6 tips for de-stressing the overthinker in you - wknd. - June 19th, 2021
- Validation and clinical applicability of a deep learning system for retinal disease (Tuesday, 10th August 2021) City, University of London - City,... - June 16th, 2021
- Physical works out the dark side of the mind in an honest way - Metro US - June 16th, 2021
- The Number One Voice Certain To Drive Your Leadership Off The Deep End - Forbes - June 16th, 2021
- Consumer Companies Bail on Non-Core Assets: "Deep is the New Wide" - Mergers & Acquisitions - June 16th, 2021
- Bengio Team Proposes Flow Network-Based Generative Models That Learn a Stochastic Policy From a Sequence of Actions - Synced - June 16th, 2021
- It's more than just skin-deep: Feel and look amazing with Bubble Skincare | Sponsored - Harvard Crimson - June 16th, 2021
- AI in Healthcare Market Drivers, Challenges, Opportunities and Competitive Strategy Over 2021-2031 | Nuance Communications, Inc., DeepMind... - June 16th, 2021
- NVIDIA and the battle for the future of AI chips - Wired.co.uk - June 16th, 2021
- Accelerating Deep Learning on the JVM with Apache Spark and NVIDIA GPUs - InfoQ.com - June 16th, 2021
- AI in Europe: Who's leading the way and where is it heading? - Siliconrepublic.com - June 16th, 2021
- Review: Bo Burnhams Inside is a successful depiction of a lonely mind - Los Angeles Times - June 16th, 2021