Managing data can be challenging in any environment. But data management in the cloud is especially difficult, given the unique security, cost and performance issues at play. With that reality in mind, here are some tips to help IT teams optimize cloud data management and strike the right balance among the various competing priorities that shape data in public, private or hybrid cloud environments.
Before delving into best practices for cloud data management, lets briefly discuss why managing data in the cloud can be particularly challenging. The main reasons include:
Those are the problems. Now, lets look at five ways to tackle them.
A basic best practice for striking the right balance between cloud storage costs and performance is to use data storage tiers. Most public cloud providers offer different storage tiers (or classes, as they are called on some clouds) for at least their object storage services.
The higher-cost tiers offer instant access to data. With lower-cost tiers, you may have to wait some amount of time--which could range from minutes to hours--to access your data. Data that doesnt require frequent or quick access, then, can be stored much more cheaply using lower-cost tiers.
For many teams, object storage services like AWS S3 or Azure Blob Storage are the default solution for storing data in the cloud. These services let you upload data in any form and retrieve it quickly. You dont have to worry about structuring the data in a particular way or configuring a database.
The downside of cloud object storage is that you usually have to pay fees to interact with the data. For instance, if you want to list the contents of your storage bucket or copy a file, youll pay a fee for each request. The request fees are very small--fractions of a penny--but they can add up if you are constantly accessing or modifying object storage data.
You dont typically have to pay special request fees to perform data operations on other types of cloud storage services, like block storage or cloud databases. Thus, from a cost optimization perspective, it may be worth forgoing the convenience of object storage in order to save money.
One of the key security challenges that teams face when managing cloud data is the risk that they dont actually know where all of their sensitive data is within cloud environments. It can be easy to upload files containing personally identifiable information or other types of private data into the cloud and lose track of it (especially if your cloud environment is shared by a number of users within your organization, each doing their own thing with few governance policies to manage operations).
Cloud data loss prevention (DLP) tools address this problem by automatically scanning cloud storage for sensitive data. Public cloud vendors offer such tools, such as Google Cloud DLP and AWS Macie. There are also third-party DLP tools, like Open Raven, that can work within public cloud environments.
Cloud DLP wont guarantee that your cloud data is stored securely--DLP tools can overlook sensitive information--but it goes a long way toward helping you find data that is stored in an insecure way before the bad guys discover it.
Data egress--which means the movement of data out of a public cloud environment--is the bane of cloud data cost and performance optimization. The more egress you have, the more youll pay because cloud providers bill for every gigabyte of data that moves out of their clouds. Egress also leads to poorer performance due to the time it takes to move data out of the cloud via the Internet.
To mitigate these issues, make data egress mitigation a key priority when designing your cloud architecture. Dont treat egress costs and performance degradations as inevitable; instead, figure out how to store data as close as possible to the applications that process it or the users who consume it.
In addition to allowing you to store data, all of the major clouds now also let you process it using a variety of managed data analytics services, such as AWS OpenSearch and Azure Data Lake Analytics.
If you want to analyze your data without having to move it out of the cloud (and pay those nasty egress fees), these services may come in handy. However, youll typically have to pay for the services themselves, which can cost a lot depending on how much data you process. There may also be data privacy issues to consider when analyzing sensitive cloud data using a third-party service.
As an alternative, you can consider installing your own, self-managed data analytics platform in a public cloud, using open source tools like the ELK Stack. That way, you can avoid egress by keeping data in the cloud, without having to pay for a third-party managed service. (Youll pay for the cloud infrastructure that hosts the service, but that is likely to cost much less than a managed data analytics service.)
The bottom line here: Managed cloud data analytics may be a useful tool, but deploy them wisely.
Like many other things, data management is just harder when you have to do it in the cloud. The good news is that, by being strategic about which cloud storage services you use, how you manage data in the cloud and how you factor data management into your cloud architecture, you can avoid the cost, performance and security pitfalls of cloud data.
Read more from the original source:
5 Ways to Improve Data Management in the Cloud - ITPro Today
- re:Invent recap: Amazon showcases cloud computing innovation - VentureBeat - December 3rd, 2021
- Cloud computing is changing everything about IT skills. Here's what that means for your job - ZDNet - December 3rd, 2021
- What is 'The Cloud' and How Does it Pervade Our Lives? - Scientific American - December 3rd, 2021
- Cyber Security Today, Dec. 1, 2021 - The FBI seizes alleged ransomware gang member's funds, a cloud computing security report from Google and more -... - December 3rd, 2021
- How are Organizations Doing with Cloud? - InformationWeek - December 3rd, 2021
- Amazon Way Behind Microsoft, Salesforce, and Oracle in Cloud Software - Business Insider - December 3rd, 2021
- Sidero CTO: Why multi-cloud strategies are on the rise - Siliconrepublic.com - December 3rd, 2021
- Everyone's been talking about AWS, and Google Cloud is jealous - Techradar - December 3rd, 2021
- Industrial Cyber Security Market: High Investment in Infrastructure as a Service and Cloud Computing Tools to Augment Growth During Coronavirus -... - December 3rd, 2021
- Cloud Security Alliance Issues Guidance to Help Organizations Optimize Business Outcomes When Using Cloud-native Key Management Services with External... - December 3rd, 2021
- Running business on Cloud - The Hindu - December 3rd, 2021
- EMBL-EBI Selects Google Cloud as Strategic Partner to Accelerate the Pace of Research - PRNewswire - December 3rd, 2021
- Varcoe: Amazon 'bet the right way' choosing Calgary for new data hub and tech training - Calgary Herald - December 3rd, 2021
- Joint Cloud Computing: How Can Organizations Benefit From This New Trend? - Toolbox - November 28th, 2021
- AI and Machine Learning, Cloud Computing, and 5G Will Dominate in 2022 - IBL News - November 28th, 2021
- EU companies issue formal complaint against Microsoft OneDrive Windows integration - ZDNet - November 28th, 2021
- One of the world's largest supercomputers lived for only 10 minutes - TechRadar - November 28th, 2021
- 3 Reasons to Buy Ankr - Motley Fool - November 28th, 2021
- Tencent and Green Packet firm up cloud deal - The Star Online - November 28th, 2021
- Attackers are using AWS instances to launch attacks on WordPress sites - Techradar - November 28th, 2021
- Atos and OVHcloud strengthen their partnership by leveraging Atos OneCloud Sovereign Shield - GlobeNewswire - November 28th, 2021
- Microsoft Partners With the Global Manufacturing and Industrialisation Summit To Fast-track Digital Manufacturing Through Cloud Computing - Al-Bawaba - November 28th, 2021
- Global Micro Server IC Market to Reach $6.1 Billion by 2027 - Yahoo Finance - November 28th, 2021
- Smart Cloud: What role will AI play in the evolution of cloud technology? - Global Banking And Finance Review - November 28th, 2021
- AVM Cloud VMware collaboration: Committed to customers' end-to-end cloud journeys - Malaysiakini - November 28th, 2021
- Cerebras Systems and G42 to Partner to Bring High-Performance AI Compute to the Middle East - Financial Post - November 28th, 2021
- What is Cloud Computing? | Oracle - November 24th, 2021
- The Top 10 Hottest Cloud Computing Startup Companies Of 2021 - CRN - November 24th, 2021
- An Overview of Cloud Computing Technology in the Era of Digital Transformation - Analytics Insight - November 24th, 2021
- What Is Edge Computing? Why It's Important and How It Works | eWEEK - eWeek - November 24th, 2021
- LPU Launches BTech in CSE Cloud Computing in Collaboration with Google Cloud - News18 - November 24th, 2021
- Internet of Things (IoT) Cloud Trends 2021 - Datamation - November 24th, 2021
- Artificial Intelligence and Machine Learning, Cloud Computing, and 5G Will Be the Most Important Technologies in 2022, Says New IEEE Study - Dark... - November 24th, 2021
- How SOAR Helps to Hold Up Your Part of the Cloud Security Shared Responsibility Model - Security Boulevard - November 24th, 2021
- Global Virtualization Security Market Expected to Generate a Revenue of $6,986.3 Million, Growing at a Healthy CAGR of 13.6% During the Forecast... - November 24th, 2021
- Cohere partners with Google Cloud to train large language models using dedicated hardware - VentureBeat - November 24th, 2021
- Scalable Modular Data Centers and the Race to ROI - Data Center Frontier - November 24th, 2021
- Stocks making the biggest moves after hours: Nordstrom, Gap, VMware, HP and more - CNBC - November 24th, 2021
- Why Dream11 wants to float in the cloud - ETCIO.com - November 24th, 2021
- Beginners Guide to Cloud Computing - An Introduction ... - November 11th, 2021
- 12 Risks, Threats, & Vulnerabilities in Moving to the Cloud - November 11th, 2021
- What is Cloud Computing? | Oracle South Africa - November 11th, 2021
- Cloud Computing in Automotive Market : Qualitative And Quantitative Research Incorporating Impact Of Economic And Non-Economic Aspects | Intellias... - November 11th, 2021
- Should You Invest in the First Trust Cloud Computing ETF (SKYY)? - Yahoo Finance - November 11th, 2021
- The 3 biggest challenges of SASE in hybrid cloud environments - CSO Online - November 11th, 2021
- Microsoft to further expand cloud services in China - Chinadaily USA - November 11th, 2021
- Global Cloud Computing for Business Operations Market 2021 Supply Chain Analysis, Structure, Industry Inspection, and Forecast 2026 - Energy Siren - November 11th, 2021
- Global Cloud Computing for Business Operations Market Assessment 2021-2028| Industry Size, Share and Top Key Players: Amazon Web Services, Microsoft... - November 11th, 2021
- Cloud Computing Optical Component Market 2021: Applications, Market Assessments, Key Players Analysis, Trends, And Forecasts To 2027 - Energy Siren - November 11th, 2021
- Why a Wider Net is Now Needed to Find the Right Software Engineer - Entrepreneur - November 11th, 2021
- VMware salaries revealed: How much the cloud computing firm is paying engineers, managers, and salespeople as it navigates a future without Dell -... - November 11th, 2021
- IIT Jodhpur along with WileyNXT invites applications for its first-ever online PG Diploma with academic credits in Data Engineering and Cloud... - November 11th, 2021
- Cloud computing | Shaping Europes digital future - November 6th, 2021
- NIST Cloud Computing Program - NCCP | NIST - November 6th, 2021
- Home | Federal Cloud Computing Strategy - November 6th, 2021
- Cloud Computing Security | Trend Micro - November 6th, 2021
- The global cloud computing market size is expected to grow from USD 445.3 billion in 2021 to USD 947.3 billion by 2026, at a Compound Annual Growth... - November 6th, 2021
- Spotlight on FINRA's latest report on cloud computing in the US securities industry - Lexology - November 6th, 2021
- Microsoft rolls out new tech to connect its cloud to rivals - Yahoo Finance - November 6th, 2021
- Chicago on a Revenue Roll From Cloud and Netflix Taxes (1) - Bloomberg Tax - November 6th, 2021
- Healthcare Cloud Computing Market Projected To Represent Strong Growth, Expanded Technology By 2021 To 2030 | Scoop News - Scoop - November 6th, 2021
- Riding the wave of edge computing growth - ITProPortal - November 6th, 2021
- Creatio Partners with Whale Cloud to Accelerate Digital Transformation for Telcos - PRNewswire - November 6th, 2021
- What is Cloud Computing? | Oracle India - November 3rd, 2021
- Cloud Computing Architecture - javatpoint - November 3rd, 2021
- 10 Future Cloud Computing Trends To Watch In 2021 - November 3rd, 2021
- Filings buzz in the power industry: 18% increase in cloud computing mentions in Q2 of 2021 - Power Technology - November 3rd, 2021
- Cloud computing spending is at a record high. But the global chip shortage could create some problems ahead - ZDNet - November 3rd, 2021
- How to go hybrid: The right recipe for mixing on-prem and cloud computing - VentureBeat - November 3rd, 2021
- Microsoft rolls out new tech to connect its cloud to rivals - Reuters - November 3rd, 2021
- Cloud computing grows but agencies lacking resources, auditor general hears - The Mandarin - November 3rd, 2021
- Why cloud bugs don't get CVEs, and why it's an issue - TechTarget - November 3rd, 2021
- Top Hybrid Cloud Computing Companies to Watch in 2021 - Analytics Insight - November 3rd, 2021
- We're still just scratching the surface of the cloud's potential - TechCrunch - November 3rd, 2021
- Cloud-fuelled European IT outsourcing spend hits record high - ComputerWeekly.com - November 3rd, 2021
- Beginner computer classes online: What can you take, and where? - ZDNet - November 3rd, 2021
- Microsoft Rides Cloud Computing Boost to Nearly Overtake Apple as Most Valuable Company | Technology News - Gadgets 360 - November 3rd, 2021
- Secrets to success with cybersecurity hiring and retention - Healthcare IT News - November 3rd, 2021
- Ekinops acquires SixSq and steps up its presence in Edge Computing - PRNewswire - November 3rd, 2021
- Alibaba launches new server chip to boost its cloud business in challenge to Amazon and Microsoft - CNBC - October 24th, 2021