Cloud Hosting

Dec. 17, 2021 Scientists come to the U.S. Department of Energy (DOE) national laboratories to solve big problems. Increasingly, these scientists are turning to artificial intelligence (AI) and machine learning (ML) to help them answer scientific questions. AsAIandMLcontinue to scale and advance, so does the complexity of running them on supercomputers and distributed computing networks.

Scientists at theDOEs Argonne National Laboratory are tackling this challenge by modeling, simulating, predicting and optimizing the performance of workflows. These workflows orchestrate and manage large computational and data science applications running on supercomputers connected by large data-transfer networks, such as those connecting theDOEs national laboratories, user facilities, and data storage centers across the country. A new project funded by theDOE, PosEiDon: Platform for Explainable Distributed Infrastructure, is turning toAIandMLto improve the performance of these workflows.

By optimizing science workflows that run on distributed computing and data infrastructure, we will be able to accelerate scientific discovery, said Prasanna Balaprakash, a computer science leader at Argonne whose research focuses on data-efficient machine learning methods for scientific applications. The results could speed up the discovery of new battery materials, aid in the exploration of the universe, advance the science of nuclear physics and improve climate simulations.

Balaprakash and his team at Argonne will collaborate on PosEiDon with partners at the University of Southern California, theDOEs Lawrence Berkeley National Laboratory, and the Renaissance Computing Institute (RENCI) at the University of North Carolina at Chapel Hill. The interdisciplinary team brings together a unique combination of expertise in high performance computing, simulation and modeling, workflows, networking, and anomaly detection, like glitches in the system. Together, they will model science workflows, predict their performances, automatically identify performance anomalies and optimize entire workflows to ensure they run as fast and efficiently as possible.

TheDOEruns some of the fastest supercomputers in the world. As the scientific experiments and the simulations that run on them become more complex, the process for understanding outcomes has become more distributed. Often, experiments conducted in one place transfer data to another to be processed by a supercomputer, which pulls in more data from other sources, then sends the results back to scientists. Workflows provide a way to manage the complexity of these large scientific endeavors. Essentially, they are a series of highly interdependent tasks that need to be executed in a certain order, in a certain place, with minimal human intervention.

First, the project will use traditional modeling and simulation approaches to simulate workflows running on different computing and data infrastructures. However, this approach is computationally expensive as it can take several weeks to simulate even a few hundred workflow configurations. But by usingML, the team will reduce this time drastically. Once PosEiDon is completed, it will be able to predict millions of workflow configurations in a few minutes to determine which will work best. To that end, PosEiDon will leverage DeepHyper, a scalable automatedMLpackage.

With DeepHyper, we will automate the design and development ofMLmodels required for predicting the workflow performance, for detecting anomalies and for tuning the performance, said Balaprakash. Moreover, scientists will be able to use the predictive models to identify anomalies, or differences between how long the workflow should take and how long it actually takes. It will also be able to tell scientists where, when and why the anomaly is happening, so they can identify and fix any issues in the workflow or in the computing system.

Once PosEiDon is tuned, researchers will test it on several real-world scientific problems at Argonne and otherDOEcomputing facilities. These include nuclear physics and weather and climate simulations, which will run on various distributed computing resources and supercomputers, including the next-generation Polaris and Aurora systems at the Argonne Leadership Computing Facility (ALCF), aDOEOffice of Science user facility.

Balaprakash hopes this project will accelerate and broaden the use ofAIfor science applications.A breakthrough in workflow simulation, modeling and optimization will not only improveDOEs artificial intelligence and machine learning applications, but it also radically changes how computational and data science can be used to pursue new scientific discoveries in a variety of fields, he said.

PosEiDon is funded by the Department of Energy under the Integrated Computational and Data Infrastructure (ICDI) for Scientific Discovery program. To learn more about the project, visit thePosEiDon website.

DeepHyperis funded by BalaprakashsDOEEarly Career Award from the Advanced Scientific Computing Research program within theDOEOffice of Science.

The Argonne Leadership Computing Facilityprovides supercomputing capabilities to the scientific and engineering community to advance fundamental discovery and understanding in a broad range of disciplines. Supported by the U.S. Department of Energys (DOEs) Office of Science, Advanced Scientific Computing Research (ASCR) program, theALCFis one of twoDOELeadership Computing Facilities in the nation dedicated to open science.

Argonne National Laboratoryseeks solutions to pressing national problems in science and technology. The nations first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance Americas scientific leadership and prepare the nation for a better future. With employees from more than60nations, Argonne is managed byUChicago Argonne,LLCfor theU.S. Department of Energys Office of Science.

The U.S. Department of Energys Office of Scienceis the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visithttps://energy.gov/science.

Source: Liz Thompson, Argonne National Laboratory

Read the original:

Accelerating Discovery: Optimizing Workflows to Advance the Use of AI for Science - HPCwire