Cloud Hosting

Due to the increased frequency of extreme weather, businesses have to make sure they protect themselves. In January, Brge Brende, president of the World Economic Forum, warned that climate change and environmental risks are the biggest threats facing society.

Even businesses that are not directly affected by extreme weather events may face risks if accurate weather forecasting is not built into their business assumptions, with Peter Giger, group chief risk officer of Zurich Insurance Group, recently asking: If weather patterns change, how do I look at the probability of events occurring?

The floods in 2013 were reported as the worst December and January rainfall on record. Two years later, in 2015, Cumbria and North East England faced Storm Desmond, which broke records for the most rainfall in a single day. This week, residents of Cumbria are again facing the big clean-up in the wake of Storm Ciara.

Weather Source provides enterprise clients with meteorological data to support their businesses. Such information is becoming business critical, due to the effects of climate change.

What we are seeing is climate change is in full effect, says Mark Gibbas, CEO of Weather Source. Weather information is important to many companies that have sensitivity to weather, which can impact their operations and their revenue.

As weather becomes more extreme, accurate forecasting becomes increasingly important in helping companies to quantify and manage climate risks.

Weather Source has created a business to provide such information to businesses. According to Gibbas, what makes the service unique is that it offers a continuum of weather over space and time, which provides a history of normal weather conditions for any hour of the year and the variants in weather.

In the early years of the company, around 2015, Weather Source used MySQL on optimised hardware for speed. As it expanded, it began hosting the service on Amazon Web Services (AWS) and Google Cloud Platform (GCP). While public cloud hosting is still being used, Gibbas says the company wanted to achieve greater speed and efficiency.

Due to the complex dataset it requires a time series database which spans the planet Gibbas says the company needed something that could handle petabytes of data.

Weather Source selected Snowflakes Data Exchange to provide its clients with a better understanding of climate change risk and to increase its customer portfolio. This data includes hyper-local weather information and climate data from around the world.

Snowflake is extremely efficient and fast, and has powerful technology for data sharing, he says. This allows us to store information once and share with many different clients, cutting down on the total storage we require and avoid duplicating information.

The Weather Sources service runs on Snowflake, which is hosted in AWS. Moving historical data into the cloud is time-consuming, but once there, updates to the database can be achieved relatively quickly. For instance, migrating half a petabyte of historical data into the public cloud can take several days.

But once the data is in Snowflake, operationally, it only takes seconds to update the latest weather information, according to Gibbas. The Weather Source has two ways to upload data: either via an AWS S3 bucket, which can automatically load data into Snowflake, or via the Snowflake application programming interface (API), which Gibbas says is very fast.

Snowflake is also very efficient computationally. Only a few hundred Gbytes of new data per day is processed, he adds. For comparison, the company needed to process a terabyte of data on GCP or AWS.

Such efficiency gains are essential in accurate forecasting. Each days forecast is made up of several forecasts and is projected over 15 days, leading to petabyte scale of data processing. But once the data is in Snowflake, Weather Source does not have to move the data.

The Weather Source data service is available in Snowflakes Data Exchange market. Snowflake users who require weather data are able to access different views of the weather data provided by Weather Source, without the need to download anything.

Traditionally, organisations both had to make a copy of the data locally on their own systems when they needed to share data, which Gibbas says dramatically expands the amount of data they need.

However, data sharing through the Snowflake Data Exchange market provides a single place where the data is stored. They just access one common database, Gibbas adds.

This also cuts down the speed of data transfer as there is no need to load the data into their own systems, says Gibbas. Since there is only a single copy of the weather data, as and when the forecast is refreshed, users can immediately access the new data.

More:
Weather Source uses Snowflake to keep ahead of extreme weather - ComputerWeekly.com