The history of computer chips is a thrilling tale of extreme miniaturization.
The smaller, the better is a trend thats given birth to the digital world as we know it. So, why on earth would you want to reverse course and make chips a lot bigger? Well, while theres no particularly good reason to have a chip the size of an iPad in an iPad, such a chip may prove to be genius for more specific uses, like artificial intelligence or simulations of the physical world.
At least, thats what Cerebras, the maker of the biggest computer chip in the world, is hoping.
The Cerebras Wafer-Scale Engine is massive any way you slice it. The chip is 8.5 inches to a side and houses 1.2 trillion transistors. The next biggest chip, NVIDIAs A100 GPU, measures an inch to a side and has a mere 54 billion transistors. The former is new, largely untested and, so far, one-of-a-kind. The latter is well-loved, mass-produced, and has taken over the world of AI and supercomputing in the last decade.
So can Goliath flip the script on David? Cerebras is on a mission to find out.
When Cerebras first came out of stealth last year, the company said it could significantly speed up the training of deep learning models.
Since then, the WSE has made its way into a handful of supercomputing labs, where the companys customers are putting it through its paces. One of those labs, the National Energy Technology Laboratory, is looking to see what it can do beyond AI.
So, in a recent trial, researchers pitted the chipwhich is housed in an all-in-one system about the size of a dorm room mini-fridge called the CS-1against a supercomputer in a fluid dynamics simulation. Simulating the movement of fluids is a common supercomputer application useful for solving complex problems like weather forecasting and airplane wing design.
The trial was described in a preprint paper written by a team led by Cerebrass Michael James and NETLs Dirk Van Essendelft and presented at the supercomputing conference SC20 this week. The team said the CS-1 completed a simulation of combustion in a power plant roughly 200 times faster than it took the Joule 2.0 supercomputer to do a similar task.
The CS-1 was actually faster-than-real-time. As Cerebrus wrote in a blog post, It can tell you what is going to happen in the future faster than the laws of physics produce the same result.
The researchers said the CS-1s performance couldnt be matched by any number of CPUs and GPUs. And CEO and cofounder Andrew Feldman told VentureBeat that would be true no matter how large the supercomputer is. At a point, scaling a supercomputer like Joule no longer produces better results in this kind of problem. Thats why Joules simulation speed peaked at 16,384 cores, a fraction of its total 86,400 cores.
A comparison of the two machines drives the point home. Joule is the 81st fastest supercomputer in the world, takes up dozens of server racks, consumes up to 450 kilowatts of power, and required tens of millions of dollars to build. The CS-1, by comparison, fits in a third of a server rack, consumes 20 kilowatts of power, and sells for a few million dollars.
While the task is niche (but useful) and the problem well-suited to the CS-1, its still a pretty stunning result. So howd they pull it off? Its all in the design.
Computer chips begin life on a big piece of silicon called a wafer. Multiple chips are etched onto the same wafer and then the wafer is cut into individual chips. While the WSE is also etched onto a silicon wafer, the wafer is left intact as a single, operating unit. This wafer-scale chip contains almost 400,000 processing cores. Each core is connected to its own dedicated memory and its four neighboring cores.
Putting that many cores on a single chip and giving them their own memory is why the WSE is bigger; its also why, in this case, its better.
Most large-scale computing tasks depend on massively parallel processing. Researchers distribute the task among hundreds or thousands of chips. The chips need to work in concert, so theyre in constant communication, shuttling information back and forth. A similar process takes place within each chip, as information moves between processor cores, which are doing the calculations, and shared memory to store the results.
Its a little like an old-timey company that does all its business on paper.
The company uses couriers to send and collect documents from other branches and archives across town. The couriers know the best routes through the city, but the trips take some minimum amount of time determined by the distance between the branches and archives, the couriers top speed, and how many other couriers are on the road. In short, distance and traffic slow things down.
Now, imagine the company builds a brand new gleaming skyscraper. Every branch is moved into the new building and every worker gets a small filing cabinet in their office to store documents. Now any document they need can be stored and retrieved in the time it takes to step across the office or down the hall to their neighbors office. The information commute has all but disappeared. Everythings in the same house.
Cerebrass megachip is a bit like that skyscraper. The way it shuttles informationaided further by its specially tailored compiling softwareis far more efficient compared to a traditional supercomputer that needs to network a ton of traditional chips.
Its worth noting the chip can only handle problems small enough to fit on the wafer. But such problems may have quite practical applications because of the machines ability to do high-fidelity simulation in real-time. The authors note, for example, the machine should in theory be able to accurately simulate the air flow around a helicopter trying to land on a flight deck and semi-automate the processsomething not possible with traditional chips.
Another opportunity, they note, would be to use a simulation as input to train a neural network also residing on the chip. In an intriguing and related example, a Caltech machine learning technique recently proved to be 1,000 times faster at solving the same kind of partial differential equations at play here to simulate fluid dynamics.
They also note that improvements in the chip (and others like it, should they arrive) will push back the limits of what can be accomplished. Already, Cerebras has teased the release of its next-generation chip, which will have 2.6 trillion transistors, 850,00 cores, and more than double the memory.
Of course, it still remains to be seen whether wafer-scale computing really takes off. The idea has been around for decades, but Cerebras is the first to pursue it seriously. Clearly, they believe theyve solved the problem in a way thats useful and economical.
Other new architectures are also being pursued in the lab. Memristor-based neuromorphic chips, for example, mimic the brain by putting processing and memory into individual transistor-like components. And of course, quantum computers are in a separate lane, but tackle similar problems.
It could be that one of these technologies eventually rises to rule them all. Or, and this seems just as likely, computing may splinter into a bizarre quilt of radical chips, all stitched together to make the most of each depending on the situation.
Image credit: Cerebras
- Quantum computing research helps IBM win top spot in patent race - CNET - January 14th, 2021
- Error Protected Quantum Bits Entangled: A Milestone in the Development of Fault-Tolerant Quantum Computers - SciTechDaily - January 14th, 2021
- You can find a $180K solar-powered car, qubit controls, and breathing tips at the NL Tech Pavilion at CES 2021 - TechRepublic - January 14th, 2021
- NSA Cites DoD Crypto Work in First-Ever Cybersecurity Year-End Report - MeriTalk - January 14th, 2021
- 'Magic' angle graphene and the creation of unexpected topological quantum states - Princeton University - December 15th, 2020
- Bringing Your Mainframe Into the Cloud Age - CMSWire - December 15th, 2020
- ASC20-21 Student Supercomputer Challenge Kickoff: Quantum Computing Simulations, AI Language Exam and Pulsar Searching with FAST - Business Wire - November 26th, 2020
- Virtual ICM Seminar with Hiroaki Kitano, 'Nobel Turing Challenge-Creating the Engine of Scientific Discovery' to Be Held Nov 26 - HPCwire - November 26th, 2020
- Imperfections Lower the Simulation Cost of Quantum Computers - Physics - November 24th, 2020
- Is the blockchain vulnerable to hacking by quantum computers? - Moneyweb.co.za - November 24th, 2020
- Can a Computer Devise a Theory of Everything? - The New York Times - November 24th, 2020
- Cracking the Secrets of an Emerging Branch of Physics: Exotic Properties to Power Real-World Applications - SciTechDaily - November 24th, 2020
- Inside the Competition That Will Save Bitcoin From Quantum Computers - Decrypt - November 22nd, 2020
- Foreign policy expert: China is 'outstripping us' in technologies of the future - Brainerd Dispatch - November 22nd, 2020
- Cracking the secrets of an emerging branch of physics - MIT News - November 22nd, 2020
- #SpaceWatchGL Opinion: Quantum Technology and Impact of the Global Space Security - SpaceWatch.Global - November 22nd, 2020
- A Scoville Heat Scale For Measuring The Progress Of Emerging Technologies In 2021 - Forbes - November 22nd, 2020
- Democracies must team up to take on China in the technosphere - The Economist - November 22nd, 2020
- Honeywell fires up the H1, its second-generation quantum computer - CNET - November 2nd, 2020
- Quantum computers could soon reveal all of our secrets. The race is on to stop that happening - ZDNet - November 2nd, 2020
- Quantum Computing Expert Warns Governments May Be First to Crack Algorithms Keeping Bitcoin and the Internet Secure - The Daily Hodl - November 2nd, 2020
- Australia's Archer and its plan for quantum world domination - ZDNet - November 2nd, 2020
- Quantum Computing Is Bigger Than Donald Trump - WIRED - November 2nd, 2020
- Will Quantum Mechanics Produce the True Thinking Computer? - Walter Bradley Center for Natural and Artificial Intelligence - November 2nd, 2020
- Strategic Partnership will aid smooth work in the event of regional crisis: Australia High Commissioner - The Hindu - November 2nd, 2020
- Valuation of quantum computer maker D-Wave slashed by more than half after company struggles to raise financing - The Globe and Mail - October 27th, 2020
- 60-year-old limit to lasers overturned by quantum researchers - Griffith News - October 27th, 2020
- A Measured Approach to Regulating Fast-Changing Tech - Harvard Business Review - October 27th, 2020
- The Importance of Funding Quantum Physics, Even in a Pandemic - Inside Philanthropy - October 23rd, 2020
- Material found in paint may hold the key to a technological revolution - Advanced Science News - October 23rd, 2020
- What is Quantum Computing, and How does it Help Us? - Analytics Insight - October 13th, 2020
- QCE20: Here's what you can expect from Intel's new quantum computing research this week - Neowin - October 13th, 2020
- Canadian quantum computing firms partner to spread the technology - IT World Canada - October 13th, 2020
- Ten-year Forecasts for Quantum Networking Opportunities and Deployments Over the Coming Decade - WFMZ Allentown - October 13th, 2020
- Berkeley Lab Technologies Honored With 7 R&D 100 Awards - Lawrence Berkeley National Laboratory - October 5th, 2020
- IBM Partners With HBCUs to Diversify Quantum Computing Workforce - Diverse: Issues in Higher Education - September 25th, 2020
- IBM, Alphabet and well-funded startups in the race for quantum supremacy - IT Brief Australia - September 25th, 2020
- How This Bangalore Based Startup Is Driving Innovation With Quantum Technology-Based Products - Analytics India Magazine - September 25th, 2020
- New faculty add to Yale's strength in applied mathematics - Yale News - September 25th, 2020
- NU receives $115 million federal grant to research and develop beyond state-of-the-art quantum computer - Daily Northwestern - September 24th, 2020
- IBM Just Committed to Having a Functioning 1,000 Qubit Quantum Computer by 2023 - ScienceAlert - September 24th, 2020
- IBM plans to build a 1121 qubit system. What does this technology mean? - The Hindu - September 24th, 2020
- Extending the life of the qubit | Temple Now - Temple University News - September 24th, 2020
- OSTP, NSF, DoE, and IBM make major push to strengthen research in AI and quantum - BlackEngineer.com - September 24th, 2020
- Heres why quantum computing is a cat among the pigeons - BusinessLine - September 12th, 2020
- The Hyperion-insideHPC Interviews: ORNL Distinguished Scientist Travis Humble on Coupling Classical and Quantum Computing - insideHPC - September 12th, 2020
- Oxford Instruments Partners With The 10 Million Consortium, To Launch The First Commercial Quantum Computer In UK - AZoNano - September 10th, 2020
- Combinations of new technologies will upend finance - The Australian Financial Review - September 10th, 2020
- Quantum Computing Market Analysis by Growth, segmentation, performance, Competitive Strategies and Forecast to 2026 - Galus Australis - September 10th, 2020
- The Quantum Dream: Are We There Yet? - Toolbox - September 7th, 2020
- 17 extremely useful productivity tips from this years 40 Under 40 - Yahoo Finance UK - September 7th, 2020
- How Amazon Quietly Powers The Internet - Forbes - September 7th, 2020
- Study Expands Types of Physics, Engineering Problems That Can Be Solved by Quantum Computers - HPCwire - September 4th, 2020
- New evidence that the quantum world is even stranger than we thought - Purdue News Service - September 4th, 2020
- How Andersen Cheng plans to defend against the quantum computer - The Independent - September 4th, 2020
- Quantum computer to be hosted in Abingdon - ClickLancashire - September 4th, 2020
- Assistant director of NSFs Computer and Information Science and Engineering to give virtual talk Sept. 11 - Vanderbilt University News - September 4th, 2020
- Fermilab to lead $115 million National Quantum Information Science Research Center to build revolutionary quantum computer with Rigetti Computing,... - August 29th, 2020
- I confess, I'm scared of the next generation of supercomputers - TechRadar - August 29th, 2020
- Q-NEXT collaboration awarded National Quantum Initiative funding - University of Wisconsin-Madison - August 29th, 2020
- UArizona Scientists to Build What Einstein Wrote off as Science Fiction - UANews - August 29th, 2020
- Quantum leap? US plans for unhackable internet may not fructify within a decade, but India is far behind - The Financial Express - August 4th, 2020
- Google distinguished scientist Hartmut Neven is one of Fast Company's - Fast Company - August 4th, 2020
- Quantum physicists say time travelers don't have to worry about the butterfly effect - The Next Web - August 2nd, 2020
- Week in review: BootHole, RCEs in industrial VPNs, the cybersecurity profession crisis - Help Net Security - August 2nd, 2020
- New UC-led institute awarded $25M to explore potential of quantum computing and train a future workforce - University of California - July 31st, 2020
- The future of encryption: Getting ready for the quantum computer attack - TechRepublic - July 31st, 2020
- IBM and University of Tokyo team up for Quantum Innovation Initiative Consortium - SmartPlanet.com - July 31st, 2020
- 'Butterfly effect' is wrong and reality can 'heal itself', quantum scientists find in time travel experiment - The Independent - July 31st, 2020
- Research: the butterfly effect does not exist in the quantum model - FREE NEWS - July 31st, 2020
- Solving problems by working together: Could quantum computing hold the key to Covid-19? - ITProPortal - July 2nd, 2020
- Spain Introduces the World's First Quantum Phase Battery - News - All About Circuits - July 2nd, 2020
- Professor tackles one more mystery about quantum mechanics and times flow - GeekWire - July 2nd, 2020
- This Week's Awesome Tech Stories From Around the Web (Through June 27) - Singularity Hub - June 29th, 2020
- Kudos: Read about faculty, staff and student awards, appointments and achievements - Vanderbilt University News - June 29th, 2020
- This Is the First Universal Language for Quantum Computers - Popular Mechanics - June 21st, 2020
- Universal Quantum raises $4.5 million to build a large-scale quantum computer - VentureBeat - June 17th, 2020
- Ethereum (ETH) Might Not have Quantum Resistance on its Roadmap, the QRL Team Reveals - Crowdfund Insider - June 17th, 2020
- Craig Knoblock Named Michael Keston Executive Director of the USC Information Sciences Institute - USC Viterbi School of Engineering - June 17th, 2020
- European quantum computing startup takes its funding to 32M with fresh raise - TechCrunch - June 11th, 2020