The cloud infrastructure market today is dominated by an oligopoly of giants that got into the business early and grew their operations quickly: Amazon (AMZN 0.21%), Alphabet (GOOG -0.67%), and Microsoft (MSFT 0.54%). Now, with the benefit of economies of scale, they are positioned to maintain and expand their commanding leads in a market with a long growth runway.

But their stocks are down significantly this year, which some investors may see as an opportunity.

Between them, Amazon, Google, and Microsoft have captured a majority of the $180 billion global cloud infrastructure market. Amazon Web Services (AWS) is the 800-pound gorilla with about a third of the market. Microsoft's Azure and Alphabet's Google Cloud check in at 21% and 10%, respectively.

Image source: Getty Images.

A business that can benefit from economies of scale gains cost advantages as it grows, and in the world of cloud infrastructure, there are a host of size advantages to be had.

For instance, by grouping server farms into massive data centers, companies pay stable costs for the cooling systems those servers require. In the same vein, it takes an enormous amount of electricity to operate a data center. As cloud companies add customers, their revenues grow while their energy costs remain relatively fixed. Their decreasing marginal costs can be passed on to customers, giving the largest cloud players a pricing edge over their smaller peers.

In addition, Amazon, Alphabet, and Microsoft continue to add software and new functionalities to their cloud services, bundling them with their cloud hosting packages and giving their customers the ability to save time and money by only dealing with a single vendor. It's a strategy that harks back to former Amazon CEO Jeff Bezos' famous quote: "Your margin is our opportunity."

Launching a cloud infrastructure hosting service requires massive initial investments in constructing data centers. Today's leading players were able to approach that hurdle from a position of strength because they had strong cash-generating business segments to fund those outlays. For example, Microsoft may have used income generated by its Windows and Office businesses to start its Azure segment. On the other hand, cloud-centric start-ups need to raise those vast sums via equity or debt sales just to get off the ground and compete with the cost-advantaged incumbents.

Interestingly, the legacy business segments that gave these three companies the financial muscle to build out their cloud operations are now in part the causes of their falling stock prices. Alphabet's primary source of revenue and profits is advertising. In 2021, 81% of its top line came from advertising while just 7.5% came from Google Cloud. Advertising is a cyclical business, and fears that a recession is imminent have helped drive Alphabet's shares down by 20% this year.

Similarly, Amazon generated 69% of its revenues from its e-commerce platform, while just 13% came from AWS. Investors mulling the possibility of a slowdown in consumer spending have pushed Amazon's stock down by 34% in 2022.

Researchers at Precedence Research estimate that the cloud computing market -- worth $380 billion in 2021 -- will grow at a compound annual rate of 17.4% through 2030 to reach $1.614 trillion. Due to their dominance and economies of scale, Amazon, Alphabet, and Microsoft should continue to gain market share as that growth progresses. And the roles those companies' cloud segments play in their overall businesses will likely become more prominent too. As such, opportunistic investors may find long-term value in these three cloud stocks while they're down.

Suzanne Frey, an executive at Alphabet, is a member of The Motley Fool's board of directors. John Mackey, CEO of Whole Foods Market, an Amazon subsidiary, is a member of The Motley Fool's board of directors. BJ Cook has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Alphabet (A shares), Alphabet (C shares), Amazon, and Microsoft. The Motley Fool has a disclosure policy.

Original post:
Why These 3 Cloud Companies Will Continue to Take Market Share - The Motley Fool

DUBLIN--(BUSINESS WIRE)--The "Global Cloud Computing Market, By Deployment Type, By Service Model, Platform as a Service, Software as a Service, By Industry Vertical & By Region - Forecast and Analysis 2022 - 2028" report has been added to ResearchAndMarkets.com's offering.

The Global Cloud Computing Market was valued at USD 442.89 Billion in 2021, and it is expected to reach a value of USD 1369.50 Billion by 2028, at a CAGR of more than 17.50% over the forecast period (2022 - 2028).

Cloud computing is the delivery of hosted services over the internet, including software, servers, storage, analytics, intelligence, and networking. Software-as-a-Service (SaaS), Infrastructure-as-a-Service (IaaS), and Platform-as-a-Service (PaaS) are three types of cloud computing services (PaaS).

The expanding usage of cloud-based services and the growing number of small and medium businesses around the world are the important drivers driving the market growth. Enterprises all over the world are embracing cloud-based platforms as a cost-effective way to store and manage data. Commercial data demands a lot of storage space. With the growing volume of data generated, many businesses have moved their data to cloud storage, using services like Amazon Web Services, Microsoft Azure, and Google Cloud Platform.

The growing need to regulate and reduce Capital Expenditure (CAPEX) and Operational Expenditure (OPEX), as well as the increasing volume of data generated in websites and mobile apps, are a few drivers driving the growth of emerging technologies. Emerging technologies like big data, artificial intelligence (AI), and machine learning (ML) are gaining traction, resulting in the global cloud computing industry growth. The cloud computing market is also driven by major factors such as data security, Faster Disaster Recovery (DR), and meeting compliance standards.

Aspects covered in this report

The global cloud computing market is segmented on the basis of deployment type, service model, and industry vertical. Based on the deployment type, the market is segmented as: private cloud, public cloud, and hybrid cloud. Based on the service model, the market is segmented as: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). Based on industry vertical, the market is segmented as: Government, Military & Defense, Telecom & IT, Healthcare, Retail, and Others. Based on region it is categorized into: North America, Europe, Asia-Pacific, Latin America, and MEA.

Driver

Restraint

Key Market Trends

Companies Mentioned

For more information about this report visit https://www.researchandmarkets.com/r/m9wewu

Go here to read the rest:
Global Cloud Computing Market Report to 2028 - Featuring Accenture, Adobe, IBM and Intel Among Others - ResearchAndMarkets.com - Business Wire

Digital transformation, modernization, zero trust and data-driven are buzzwords every IT executive has likely heard one too many times. Nevertheless, these concepts are keys to survival in the digital economy. This is a truism for companies across all vertical industries, including those that build the technology fueling modernization projects.

HPE started its transformation in 2018, with the introduction of GreenLake. At Discover 2019, Antonio Neri announced that by the end of 2022, every HPE product in its portfolio would be offered as-a-Service. And then Covid hit.

HPE Discover felt much more energized this year. Maybe it was the two-year hiatus and the transformation of HPEs portfolio in the intervening years, or maybe it was Janet Jackson. Probably a bit of all three. Regardless, here are my big takeaways from HPE Discover 2022.

HPE is embracing native cloud with Arm

In case you didn't hear, HPE announced the ProLiant RL300 Gen11 server, a 1u single-socket server packing up to 128 single-threaded Ampere Altra (or Altra Max) CPUs. While the ProLiant team may not have been consciously trying to make a statement, the introduction of an Arm-based server as the first member of the ProLiant Gen11 family did it for them.

HPE introduces the first major OEM server based on Arm

According to the announcement materials, the new server specifically targets digital enterprises and service providers. Both of these target markets make sense. Arm-based instances in the cloud have seen a lot of success since they achieved performance parity with x86 for cloud-native applications, at a lower cost and power envelope to boot. It makes sense that an enterprise with a significant digital presence would want to replicate this environment in-house. Think about ithundreds to thousands of servers running your cloud-native workloads, with each server delivering considerable cost savings. It adds up.

The service provider space also makes perfect sense. As-a-Service can help customers of all sizes realize the same economic benefits as the cloud giants such as AWS, Azure and Oracle Cloud (OCI).

One of the things I found very interesting about this announcement was the inclusion of the OpenBMC firmware stack. HPE, like other server vendors, likes to provide a premium management experience through its iLO baseboard management controller (BMC). Utilizing a vendor-specific BMC stack hooks IT organizations into using that vendor's management console (in this case, OneView).

By populating the RL300 with the OpenBMC firmware stack, HPE fully is fully embracing the realities of the cloud-native datacenter, where open-source tools are used to deploy, provision and manage infrastructure. It is the seemingly smaller things like this (support for OpenBMC) that demonstrates HPEs understanding of what the hybrid future looks like and how it should align its portfolio to meet the needs of the market.

HPE ProLiant RL300 Gen11 highlights

Some may read this and think interesting, but this isnt going to be successful. Given the fits and starts that Arm has had in the datacenter, I get it. But think about this HPE would not be investing the millions of non-recurring engineering (NRE) dollars into a mainstream platform unless there was a market and customers were asking for it. In an era when server portfolios are shrinking across many vendors, HPE invested in this platform. This should tell you something.

For complete coverage on the RL300, read this analysis I wrote with Moor Insights & Strategy (MI&S) Founder and CEO Patrick Moorhead. Also, watch this video where MI&S colleague Steve McDowell and I quickly analyze the announcement.

HPE is a services company

HPE announced GreenLake back at Discover 2018. A year later, HPE CEO Antonio Neri stood on stage and proclaimed that the entire HPE portfolio would be made available as-a-Service through this still newish consumption-based model by the end of 2022.

Fast forward to 2022, and the transformation has happened. HPE is, without a doubt, a services company. Throughout the keynotes and the individual sessions, the company seems to be singularly focused on GreenLake as the delivery vehicle for tailored solutions such as cloud-native, data analytics, machine learning and HPC.

How focused is the company? Lets put it this way apart from the RL300 announcement, I don't think I heard ProLiant mentioned. Nor did I see the brand on any signage on the show floor or anywhere else. Id say the same for the HPE storage portfolio.

Does HPE no longer sell servers, storage and networking? Of course not! In addition to being the building blocks of GreenLake, the company will continue to sell the entire portfolio of servers, storage boxes and networking solutions. The world is full of companies that still buy, rack and deploy infrastructure in the traditional manner. Still, the company's direction is clear.

A final note on HPE as a service company. It's one thing to deliver the portfolio in a consumption-based manner. Other OEMs have certainly followed HPE's lead. What makes HPE unique is its organizational pivot and drive to as-a-Service, not just through branding and consistency of messaging, but across the company and functions. I have spoken with HPE customers, channel partners and ecosystem partners. This company has a GreenLake-first mindset.

GreenLake Managed Services (GMS) is a function within HPE that maybe doesnt get as much coverage as it should, but I find it extremely valuable as an ex-IT person. For companies looking to deploy one of the over 70 cloud services on GreenLake, there are still many challenges around planning, deploying, provisioning, securing and governing my environment. But as any IT person can attest, there aren't hours in the day to stand up and support new environments. With GMS, I can hand this function to the GreenLake team to manage for me. And this is lifecycle management, including regulatory and licensing compliance and license optimization. In shortIT can continue to drive transformation, the IT budget can realize savings through licensing optimization and the CISO can rest a little better at night.

HPE GreenLake Managed Services can deliver lifecycle management

Security continues to be a key pillar of the HPE strategy

I've been writing about HPE's security capabilities since it introduced silicon root of trust into its ProLiant servers in 2017. What started as a method to protect infrastructure at the lowest levels has evolved into a full-stack, zero-trust architecture. Actually, full-stack may not be a fair statement as security starts in the supply chain.

HPE's posture on security as a selling point has also evolved since 2017. When the company's ProLiant Gen9 servers first launched, the selling point was "the industry's most secure servers." Fast forward to 2022, and the view is that security and zero trust is a fundamental design principle.

With this said, I think security is more than a design principle it appears to be baked into the companys fabric. This may seem hyperbolic, but the company has invested heavily in securing HPE environments. Below are a few quick thoughts on what I gleaned from Discover a more detailed viewpoint will be published next week.

1. Remember Project Aurora? Its real.

HPE announced Project Aurora at Discover 2020. Project Aurora is a zero-trust architecture that locks down HPE infrastructure from sourcing materials to end of life. Not only that, Project Aurora is designed to build a secure chain of custody across the lifecycle and up the stack from silicon to the data being created and used in workloads and applications (you can read my coverage of it here).

Fast forward a year, and you may wonder whether Project Aurora ever moved beyond the project phase. The answer is yes. HPE is enabling the functionality of Project Aurora in its newly announced GreenLake Enterprise Private Cloud solution. It makes sense that we would see Project Aurora instantiated in the Private Cloud solution first, as this is a very controlled environment. I expect this to be rolled out into all GreenLake offerings in the near future.

2. Shared Responsibility HPE offers a framework for securing the hybrid environment.

So here's the question for you IT pros. Do you feel completely comfortable understanding the shared responsibility between you and your public cloud provider regarding security? From the conversations Ive had, the answers have been mixed.

However, when GreenLake or a GreenLake-like model is introduced into the environment, those lines get blurred.

The blurred lines of responsibility

Enter HPE GreenLake Security Shared Responsibility Model (SSRM). The SSRM gives HPE and its customers a clear line of responsibility and ownership spanning the different potential deployment scenarios.

For those who are still a little confused as to what SSRM is it is not a piece of software or hardware. Its an HPE-developed framework that enables an enterprise IT organization to manage its security profile when deploying workloads and data on GreenLake. I like to equate SSRM to kind of a RACi (responsible, accountable, consulted, informed) Matrix in project management in that clear boundaries are drawn so that security ownership is never called into question.

SSRM defining ownership of security

The SSRM session that HPE Chief Security Officer Bobby Ford and A&PS Operations Lead Simon Leech held was well worth watching (see it here),partly because this program is so well thought out, and partly because it reminds us how security in the enterprise is much bigger than just technology. It's people, processes, programs and constant poking and testing of these elements regularly.

1. Security strategies must be living.

There were a lot of other great security sessions throughout Discover and they can all be watched on-demand. However, one of the best discussions Steve McDowell and I had was with HPE Global Server Security Product Manager Cole Humphreys and InfusionPoints COO Jason Shropshire. The video can be found here, but the theme of the conversation was simple security must be a design and supply chain principle. In short, the security strategy that fails to evolve is the security strategy that will fail.

Other thoughts

Just a few more random observations to throw your way:

Finally, let me end where I started. HPE Discover 2022 was pretty special. The company has transformed before our eyes, and there is plenty of evidence of it being on a solid trajectory. It's time to stop thinking of HPE as a server company or a storage company. HPE is an IT solutions and services company, with a strong emphasis on services.

Note: Moor Insights & Strategy writers and editors may have contributed to this article.

Moor Insights & Strategy, like all research and tech industry analyst firms, provides or has provided paid services to technology companies. These services include research, analysis, advising, consulting, benchmarking, acquisition matchmaking, and speaking sponsorships. The company has had or currently has paid business relationships with 88, Accenture, A10 Networks, Advanced Micro Devices, Amazon, Amazon Web Services, Ambient Scientific, Anuta Networks, Applied Brain Research, Applied Micro, Apstra, Arm, Aruba Networks (now HPE), Atom Computing, AT&T, Aura, Automation Anywhere, AWS, A-10 Strategies, Bitfusion, Blaize, Box, Broadcom, C3.AI, Calix, Campfire, Cisco Systems, Clear Software, Cloudera, Clumio, Cognitive Systems, CompuCom, Cradlepoint, CyberArk, Dell, Dell EMC, Dell Technologies, Diablo Technologies, Dialogue Group, Digital Optics, Dreamium Labs, D-Wave, Echelon, Ericsson, Extreme Networks, Five9, Flex, Foundries.io, Foxconn, Frame (now VMware), Fujitsu, Gen Z Consortium, Glue Networks, GlobalFoundries, Revolve (now Google), Google Cloud, Graphcore, Groq, Hiregenics, Hotwire Global, HP Inc., Hewlett Packard Enterprise, Honeywell, Huawei Technologies, IBM, Infinidat, Infosys, Inseego, IonQ, IonVR, Inseego, Infosys, Infiot, Intel, Interdigital, Jabil Circuit, Keysight, Konica Minolta, Lattice Semiconductor, Lenovo, Linux Foundation, Lightbits Labs, LogicMonitor, Luminar, MapBox, Marvell Technology, Mavenir, Marseille Inc, Mayfair Equity, Meraki (Cisco), Merck KGaA, Mesophere, Micron Technology, Microsoft, MiTEL, Mojo Networks, MongoDB, MulteFire Alliance, National Instruments, Neat, NetApp, Nightwatch, NOKIA (Alcatel-Lucent), Nortek, Novumind, NVIDIA, Nutanix, Nuvia (now Qualcomm), onsemi, ONUG, OpenStack Foundation, Oracle, Palo Alto Networks, Panasas, Peraso, Pexip, Pixelworks, Plume Design, PlusAI, Poly (formerly Plantronics), Portworx, Pure Storage, Qualcomm, Quantinuum, Rackspace, Rambus, Rayvolt E-Bikes, Red Hat, Renesas, Residio, Samsung Electronics, Samsung Semi, SAP, SAS, Scale Computing, Schneider Electric, SiFive, Silver Peak (now Aruba-HPE), SkyWorks, SONY Optical Storage, Splunk, Springpath (now Cisco), Spirent, Splunk, Sprint (now T-Mobile), Stratus Technologies, Symantec, Synaptics, Syniverse, Synopsys, Tanium, Telesign,TE Connectivity, TensTorrent, Tobii Technology, Teradata,T-Mobile, Treasure Data, Twitter, Unity Technologies, UiPath, Verizon Communications, VAST Data, Ventana Micro Systems, Vidyo, VMware, Wave Computing, Wellsmith, Xilinx, Zayo, Zebra, Zededa, Zendesk, Zoho, Zoom, and Zscaler. Moor Insights & Strategy founder, CEO, and Chief Analyst Patrick Moorhead is an investor in dMY Technology Group Inc. VI, Dreamium Labs, Groq, Luminar Technologies, MemryX, and Movandi.

See the original post here:
HPE's Transformation Was On Full Display At Discover 2022 - Forbes

Dynamic allocation of resources inside of a system, within a cluster, and across clusters is a bin-packing nightmare for hyperscalers and cloud builders. No two workloads need the same ratios of compute, memory, storage, and network, and yet these service providers need to present the illusion of configuration flexibility and vast capacity. But capacity inevitably ends up being stranded.

It is absolutely unavoidable.

But because main memory in systems is very expensive, and will continue to grow more expensive over time relative to the costs of other components in the system, the stranding of memory capacity has to be minimized, and it is not as simple as just letting VMs grab the extra memory and hoping that the extra megabytes and gigabytes yield better performance when they are thrown at virtual machines running atop a hypervisor on a server. The number of moving parts here is high, but dynamically allocating resources like memory and trying to keep it from being stranded meaning all of the cores in a machine have memory allocations and there is memory capacity left over that cant be used because there are no cores assigned to it is far better than having a static configuration of memory per core. Such as the most blunt approach, which would be to take the memory capacity, divide it by the number of cores, and give each core the same sized piece.

If you like simplicity, that works. But we shudder to think of the performance implications that such a static linking of cores and memory might have. Memory pooling over CXL is taking off among the hyperscalers and cloud builders as they try to deploy that new protocol it atop CPUs configured with PCI-Express 5.0 peripheral links. We covered Facebooks research and development recently as well as some other work being done at Pacific Northwest National Laboratory, and have discussed the prognostications about CXL memory from Intel and Marvell as well.

Microsofts Azure cloud has also been working on CXL memory pooling as it tries to tackle stranded and frigid memory, the latter being a kind of stranded memory where there are no cores left on the hypervisor to tap into that memory and the former being a broader example of memory that is allocated by the hypervisor for VMs but is nonetheless never actually used by the operating system and applications running in the VM.

According to a recent paper published by Microsoft Azure, Microsoft Research, and Carnegie Mellon University, DRAM memory can account for more than 50 percent of the cost of building a server for Azure, which is a lot higher than the average of 30 percent to 35 percent that we cited last week when we walked the Marvell CXL memory roadmap into the future. But this may be more of a function of the deep discounting that hyperscalers and cloud builders can get in a competitive CPU market, with Intel and AMD slugging it out, and that DRAM memory for servers is much more constrained and that Micron Technology, Samsung, and SK Hynix as well as their downstream DIMM makers can charge what are outrageous prices compared to historical trends because there is more demand than supply. And when it comes to servers, we think the memory makers like it that way.

Memory stranding is a big issue because that capital expense for memory is huge. If a hyperscaler or cloud builder is spending tens of billions of dollars a year on IT infrastructure, then it is billions of dollars on memory, and driving up memory usage in any way has to potential to save that hyperscaler or cloud builder hundreds of millions of dollars a year.

How bad is the problem? Bad enough for Microsoft to cite a statistic from rival Google, which has said that the average utilization of the DRAM across its clusters is somewhere around 40 percent. That is, of course, terrible. Microsoft took measurements of 100 clusters running on the Azure cloud that is clusters, not server nodes, and it did not specify the size of these clusters over a 75 day period, and found out some surprising things.

First, somewhere around 50 percent of the VMs running on these Azure clusters never touch 50 percent of the memory that is configured to them when they are rented. The other interesting bit is that as more and more of the cores are allocated to VMs on a cluster, the share of the memory that becomes stranded rises. Like this:

To be specific, when 75 percent of cores in a cluster are allocated, 6 percent of the memory is stranded. This rises to 10 percent of memory when 85 percent of the cores are allocated to VMs, 13 percent at 90 percent of cores, and full loading of cores it can hit 25 percent and outliers can push that to as high 30 percent of DRAM capacity across the cluster being stranded. On the chart on the right above, the workload changed halfway through and there was a lot more memory stranding.

The other neat thing Microsoft noticed on its Azure clusters which again have VMs of all shapes and sizes running real-world workloads for both Microsoft itself and its cloud customers that almost all VMs that companies deploy fit within one NUMA region on a node within the cluster. This is very, very convenient because spanning NUMA regions really messes with VM performance. NUMA spanning happens on about 2 percent of VMs and on less than 1 percent of memory pages, and that is no accident because the Azure hypervisor tries to schedule VMs both their cores and their memory on a single NUMA node by intent.

The Azure cloud does not currently pool memory and share it across nodes in a cluster, but that stranded and frigid DRAM memory could be moved to a CXL memory pool without any impact to performance, and some of the allocated local memory on the VMs in a node could be allocated out to a CXL memory pool, which Microsoft calls a zNUMA pool because it is a zero-core virtual NUMA node, and one that Linux understands because it already supports CPU-less NUMA memory extensions in its kernel. This zNUMA software layer is clever in that it has statistical techniques to learn which workloads have memory latency sensitivity and those that dont. So, workloads dont have such sensitivity, they get their memory allocated all or in part out to the DRAM pool over CXL and if they do, then the software allocates memory locally on the node and also from that core-less frigid memory. Here is what the decision tree looks like to give you a taste:

This is a lot hairier than it sounds, as you will see from reading the paper, but the clever bit as far as we are concerned is that Microsoft has come up with a way to create CXL memory pools that doesnt mess with applications and operating systems, which it says is a key requirement for adding CXL extended memory to its Azure cloud. The Azure hypervisor did have to be tweaked to extend the API between the server nodes and the Autopilot Azure control plane to the zNUMA external memory controller, which has four 80-bit DDR5 memory channels and multiple CXL ports running over PCI-Express 5.0 links that implements the CXL.memory load/store memory semantics protocol. (We wonder if this is a Tanzanite device, which we talked about recently after Marvell acquired the company.) Each CPU socket in the Azure cluster links to multiple EMCs and therefore multiple blocks of external DRAM that comprise the pool.

The servers used in the Microsoft test are nothing special. They are two-socket machines with a pair of 24-core Skylake Xeon SP-8157M processors. It looks like the researchers emulated a CPU with a CXL memory pool by disabling all of the cores in one socket and making all of its memory available to the first socket over UltraPath links. It is not at all clear how such vintage servers plug into the EMC device, but it must be a PCI-Express 3.0 link since that is all that Skylake Xeon SPs support. We find it peculiar that the zNUMA tests were not run with Ice Lake Xeon SP processors with DDR5 memory on the nodes and PCI-Express 5.0 ports.

The DRAM access time on the CPU socket in a node was measured at 78 nanoseconds and the bandwidth was over 80 GB/sec from the socket-local memory. The researchers say that when using only zNUMA memory the bandwidth is around 30 GB/sec, or about 75 percent of the bandwidth of a CXL x8 link, and it added another 67 nanoseconds to the latency.

Here is what the zNUMA setup looks like:

Microsoft says that a CXL x8 link matches the bandwidth of a DDR5 memory channel. In the simplest configuration, with four or eight total CPU sockets, each EMC can be directly connected to each socket in the pod and that cable lengths are short enough so that the latency out to the zNUMA memory is an additional 67 nanoseconds. If you want to hook the zNUMA memory into a larger pool of servers say, a total of 32 sockets then you can lower the amount of overall memory that gets stranded but you have to add retimers to extend the cable and that pushes the latency out to zNUMA memory to around 87 nanoseconds.

Unstranding the memory and driving up overall utilization of the memory is a big deal for Microsoft, but there are performance implications of using the zNUMA memory:

Of the 158 workloads tested above, 20 percent had no slowdown using CXL memory, and 23 percent had a slowdown of 5 percent or less. Which is good. But as you can see, some workloads were hit pretty hard. About a quarter of the workloads had a 20 percent or greater performance hit from using zNUMA memory for at least some of their capacity and 12 percent of the workloads had their performance cropped by 30 percent or more. Applications that are already NUMA aware have been tweaked so they understand memory and compute locality well, and we strongly suspect that workloads will have to be tweaked to use CXL memory and controllers like the EMC device.

And just because we think all memory will have CXL attachment in the server over time does not mean we think that all memory will be local and that CXL somehow makes latency issues disappear. It makes it a little more complicated than a big, fat NUMA box. But not impossibly more complicated and that is why research line the zNUMA effort at Microsoft is so important. Such research points the way on how this can be done.

Here is the real point: Microsoft found that by pooling memory across 16 sockets and 32 sockets in a cluster, it could reduce the memory demand by 10 percent. That means cutting the cost of servers by 4 percent to 5 percent, and that is real money in the bank. Hundreds of millions of dollars a year per hyperscaler and cloud builder.

We are counting on people creating the PCI-Express 6.0 and 7.0 standards and the electronics implementing these protocols to push down to reduce latencies as much as they push up to increase bandwidth. Disaggregated memory and the emergency of CXL as a universal memory fabric will depend on this.

Link:
Microsoft Azure Blazes The Disaggregated Memory Trail With zNUMA - The Next Platform

In this guest post, Mischa van Kesteren, sustainability officer at HPC systems integrator OCF runs through the wide variety of ways that large-scale computing environments can be made to run more energy efficiently.

Supercomputers are becoming more energy hungry. The pursuit of Moores Law and ever greater hardware performance has led to manufacturers massively ramping up the power consumption of components.

For example, a typical high performance computing (HPC) CPU from 10 years ago would have a thermal design power (TDP) of 115 Watts today that figure is closer to 200.

Modern GPUs can exceed 400 Watts. Even network switches, which used to be an afterthought from a power consumption perspective can now consume over 1KW of power in a single switch.

And the race to achieve exascale has pushed the power consumption of the fastest supercomputer on the planet from 7.9MW in 2012 to 29.9MW in 2022.

In this era of climate chaos, is this justifiable? Ultimately, yes. Whilst 29.9MW is enough electricity to power 22,000 average UK households, the research performed on these large systems is some of the most crucial to how we will navigate the challenges we are facing and those to come, whether thats research into climate change, renewable energy or to combat disease.

It is vital, however, that we continuously strive to find ways of running HPC infrastructures as efficiently as possible.

The most common method of measuring the power efficiency of a datacentre is through its power utilisation efficiency (PUE). Traditional air-cooled infrastructure blows hot air through the servers, switches and storage to cool their components and then air-conditioning is used to remove the heat from that air before recirculating it. And this all consumes a lot of power.

The air-cooling often has a PUE in excess of two, meaning the datacentre consumes twice as much power as the IT equipment. The goal is to reduce the PUE of the HPC infrastructure as close to one as possible (or even lower).

A more efficient method is to cool the hot air with water. Water transfers heat over 20 times faster than air making it far better for cooling hardware. Air cooled components can use water through rear door heat exchangers which place a large radiator at the rear of the rack (filled with cold water), cooling all the hot air that is exhausted by the servers.

Get the flow rate and water temperature right and you can remove the need for air conditioning all together. This can get the PUE down to closer to 1.4.

Alternatively, components can be fitted with water blocks on the CPU, GPU, networking etc, which directly cool the components, removing the need for air cooling all together. This is far more efficient, bringing the PUE down further, possibly to less than 1.1.

Ultimately, we need to do something with the waste heat. A good option is to make use of free cooling. This is where you use the air temperature outside to cool the water in your system. The highest outdoor temperature recorded in the UK was 38.7 C.

Computer components are rated to run at up to double that so as long as the transfer medium is efficient enough (like water) you can always cool your components for just the energy used by the pumps. This is one of the reasons why you hear about datacentres in Norway and Iceland being so competitive they can make use of free cooling far more judiciously due to their lower temperatures.

Taking things one step further, the heat can be used for practical purposes rather than exhausted into the air. There are a few innovative datacentres which have partnerships with local communities to provide heating to homes from their exhaust heat, or even the local swimming pool. The energy these homes would have consumed to heat themselves has in theory been saved, which can bring the PUE of the total system below one.

The next step which is being investigated is to store the heat in salt, which can hold it indefinitely, to make allowances for the differences in heating requirements and compute utilisation. Imagine the knock-on effect of the traditional Christmas maintenance window where IT infrastructure is turned off just when those local households need heat the most.

One thing you may have noticed about all of these solutions is they are largely only practical at scale. It is not a coincidence that vast cloud datacentres and colocation facilities are the places where these innovations are being tested, that is where they work best. The good news is the industry seems to be moving in that direction anyway as the age of the broom cupboard server room is fading.

However, in the pursuit of economies of scale, public cloud providers are operating huge fleets of servers, many of which are underutilised. This can be clearly seen in the difference in price between on demand instances that run when you want them to (typically at peak times) and spot instances which run when it is most affordable for the cloud provider.

Spot instances can be up to 90% cheaper. As cloud pricing is based almost entirely on the power consumption of the instance you are running, there must be a huge amount of wasted energy costed into the price of the standard instances.

Making use of spot instances allows you to run HPC jobs in an affordable manner, and in the excess capacity of the cloud datacentres, improving their overall efficiency. If you are running your workloads on demand, however, you can make this inefficiency worse.

Luckily HPC workloads often can fit the spot model. Users are familiar with the interaction of submitting a job and walking away, letting the scheduler determine when the best time to run that job is.

Most of the major cloud providers offer the functionality to set a maximum price you are willing to pay when you submit a job and wait for the spot market to reach that price point.

This is only one element of HPC energy efficiency, there is a whole other world of making job times shorter through improved coding, right sizing hardware to fit workloads and enabling power saving features on the hardware itself to name a few.

HPC sustainability is such a huge challenge that involves everyone who interacts with the HPC, not just the designers and infrastructure planners. However, that is a good place to start. Talking to those individuals that can build in the right technologies from the start ensures that they will provide you with a sustainable HPC fit for the future.

Read more:
Building the sustainable HPC environments of the future - ComputerWeekly.com