Kubernetes storage is useful for managing multiple forms of persistent and non-persistent storage in a cluster to cater to both stateful and stateless workloads in a containerized environment. Proper management of Kubernetes storage options allows us to dynamically provision the most suitable storage resources for multiple applications with minimal administration overhead.

There are various storage concepts you can leverage in Kubernetes, as well as typical use cases and pro tips for running Kubernetes on AWS from a storage perspective.

The storage architecture for Kubernetes is based on volumes as the core abstraction. Volumes can be ephemeral (non-persistent) or persistent, depending on their intended use cases. Kubernetes also allows for the dynamic provisioning of storage resources using volume claims.

Volumes are the basic storage entities in Kubernetes. A process in the container sees a filesystem view, which includes a root filesystem that matches the initial contents of a container image and the volumes mounted inside the container (if defined within the container specifications).

Volumes are mounted at specified paths within the container image, so you must independently specify the mount path of each volume used for each container in the pod. While volumes cannot be mounted within other volumes, a volume can be shared between multiple containers in a pod using sub-paths within the volume mount paths in the pod specifications.

Kubernetes supports multiple storage volumes, including local storage devices, network file systems (NFS), and cloud storage services like AWS Elastic Block Store (EBS) volumes. Developers can also create custom storage plugins to support specific storage systems deployed on Kubernetes clusters as extensions.

While support for cloud storage services is currently available within the core Kubernetes project, the Kubernetes Storage Special-Interest Group (k8s-sig-storage) is slowly shifting toward providing storage support through external Container Storage Interfaces (CSIs). For example, the native Amazon Elastic Block Store volume has been deprecated as of v1.17 in favor of the Amazon EBS CSI.

Unless specified within the container specifications, each container in a pod is created with an ephemeral volume by default in Kubernetes. This means there is a temporary storage directory on the machine that hosts the pod. Ephemeral volumes are removed after the pod ceases to exist.

The data in ephemeral volumes is safe in the case of a container crashing, as this does not remove a pod from a node. However, the pod with the crashed container may subsequently be deleted due to rescheduling or any reason that causes the pod to be evicted to another host, causing the data in the ephemeral volumes to be removed in the process.

Kubernetes uses persistent volumes, and persistent volume claims to allow pods to use storage in a portable manner while abstracting its implementation away from how storage is consumed.

A persistent volume (PV) is a storage entity within a cluster that is either allocated manually by an administrator or dynamically allocated based on a storage class. PVs define the details of the storage implementation, such as capacity, access modes, storage class, and reclaim policy. As they are cluster resources, PVs are not portable between clusters.

A persistent volume claim is a storage request used by developers to describe an applications storage requirements, for example, a containers storage size and access mode. Since the storage request is separate from the storage creation, Kubernetes can enforce access control mechanisms based on the container or pod credentials and the available PVs in the cluster.

A PVs lifecycle is independent of pods. The lifecycle of a PV and PVC consists of 5 stages:

For Kubernetes on AWS, EBS volumes are persistent volumes hosted in the same region and availability zone as the EC2 instance nodes running on the EKS cluster. When a pod is removed from an EC2 instance node on the EKS cluster, the data in the EBS volume mounted to the pod is persisted and the volume is unmounted.

Cluster administrators can configure storage classes in the cluster and assign PVs to each class. Each class represents a particular type of storage that users can request in their PVCs, depending on varying workload requirements within the cluster.

Dynamic volume provisioning is a feature in Kubernetes that lets you create storage volumes on-demand without requiring cluster administrators to create new ones manually.

Each storage class specifies a volume plugin called a provisioner and the parameters required for it to allocate storage volumes dynamically. When a user configures a storage class in their PVC, the provisioner automatically creates a storage volume based on the required specifications.

While some provisioners are internal and shipped alongside the Kubernetes project, you can also use external provisioners by following the specifications defined by Kubernetes.

A Container Storage Interface (CSI) is a Kubernetes extension that provides an extensible plugin architecture for vendors to create compatible storage plugins.

There are custom storage plugins in the form of CSI drivers for arbitrary storage systems external to your Kubernetes project (e.g., Amazon Elastic File System).

After you deploy the CSI driver on the Kubernetes cluster, you can use these CSI volumes with other Kubernetes storage API objects such as PVs and storage classes. For example, you can create a storage class that references the CSI provisioner, assign them to PVs, and reference the PVs in PVCs to mount the CSI volumes to a pod.

With the core Kubernetes projects gradual shift toward distributing control over the provisioning of non-native storage to the respective providers, external vendor-based CSI drivers are currently the preferred approach for managing the lifecycle of the external storage system in Kubernetes clusters. As part of this gradual shift, numerous migration features are slowly being released within the core project to ease the transition from the in-tree storage plugin to the corresponding vendor-based CSI driver.

With various options available for Kubernetes storage, you need to know which type to choose for different use cases.

For transient applications that require data storage (e.g., applications that extract read-only input data in files), you can use ephemeral volumes for storing data within the lifetime of the pod without being limited to the location and availability of some persistent volume.

Persistent volumes will store data beyond the lifetime of the pod for applications requiring data persistence across restarts (such as a database).

For use cases that require managing storage for various workloads within the cluster, you can opt for storage classes with various storage levels, backup policies, or any arbitrary policies defined by cluster administrators.

When it comes to running Kubernetes on AWS from a storage perspective, there are five pro tips you should keep in mind.

Using PVCs in the container config as part of the deployment IaC template lets users request persistent storage across clusters. This, in turn, enables storage configuration portability that is not tightly coupled with cluster resources. This tip applies to ephemeral Amazon EKS clusters created on-demand and Amazon EKS clusters deployed across multiple regions and availability zones.

Including PVs in the container, config is not recommended if you want to avoid tight coupling with a specific volume and prevent failure in binding storage volume when instantiating the container in the pod.

Instead, use PVCs as volumes while the cluster provisions storage by finding the PV bound to the claim and mounting that volume to the pod.

Cluster administrators can specify a default storage class for PVCs that do not have specific requirements for the storage class they need to bind to; they can also create separate storage classes that represent varying workload requirements. This allows users to request storage in PVCs based on the name of the storage class without manually specifying the volume within each pods specifications. Otherwise, PVCs without a specified storage class will fail to provide a PV.

Cluster administrators can provide users with a storage class when instantiating the config template. If the user provides a storage class name, the value should be defined in the StorageClassName within the PVC specifications so that the PVC can match the correct storage class. Otherwise, a PV can be automatically provisioned for the user using the default storage class in the cluster.

When monitoring your Kubernetes cluster with Prometheus and configuring alerting rules via Alertmanager, keep an eye out for PVCs that remain unbound for a prolonged period. This could mean your cluster lacks dynamic storage support or a storage system, where users wont be able to deploy configs requiring PVCs. If this is the case, the user should create a PV that matches the requirements defined in the PVC.

Implementing best practices for Kubernetes storage enables you to apply optimal storage configurations and dynamically provision suitable storage resources to multiple containerized applications without significant administrative overhead.

To reap the benefits of Kubernetes storage options on AWS, you can gradually adopt these best practices by creating storage classes and enabling dynamic volume provisioning in the cluster. You can choose suitable storage types for your containerized workloads based on the given use case.

Which best practices have you considered for Kubernetes storage on AWS? Let us know on LinkedIn, Twitter, or Facebook. Wed love to hear from you!

Read more here:
A DevOps Guide to Kubernetes Storage on AWS - Spiceworks News and Insights

Google, which started as a simple search engine in 1997, is now a major player in just about every facet of our digital lives. A commensurate increase in scrutiny has followed. From data privacy issues to antitrust claims, the Do No Evil company has been accused of myriad unsavory practices. On top of that, it has been known to somewhat abruptly cancel services and products.

Owing in part to these trials and tribulations, there are several services now seeking to draw customers away from Google. The chief value add in many of them is privacy: limited data collection and retention and improved encryption. This should be music to tax professionals ears, as it is incumbent on us to safeguard not only our own data but also our clients data. We dont keep our hard copy tax return documentation in a storage location where third parties have access, so why would we hold our digital retention to less stringent standards?

In light of this, please consider this short primer on privacy-focused alternatives to the Google offerings frequently used by tax professionals. Most of these services either wont have a free tier or have a free tier only for limited personal use. In many cases, they are substantially more expensive, especially for heavy users. One thing to note is that when youre not paying for a service, the company is monetizing the service somehow. And more often than not, youre a product being packaged and sold to an advertiser.

Googly Eyes on Cracked Sidewalk Making a Smiley Face

Photographer: Stock photo via Getty Images

If Google has one banner product, its the search engine. Truth be told, if youre using a modern browser, your connection is secure, and you arent signed in to a Google account that is keeping track of your search history, you dont have too much to worry about when performing the odd search here or there. All the same, if youre beefing up security at your house, and you install new locks on all the doors and a new alarm system, do you leave a window open just because its mostly out of reach?

The clear alternative to Google Search is DuckDuckGo, which doesnt track searches or tie an individual to a specific search query. And the search results are pretty darn good. DuckDuckGo has been around since 2008 and sees about 3 billion monthly searches. It monetizes by showing ads tied to the individual search that is performed rather than by assembling a dossier on a user and showing ads relevant to the users perceived interests.

Nonetheless, care should be taken when running searches that contain identifiable information for a client. Thought should be given as to what kind of picture could be painted by an aggregation of all the searches you run that contain a given clients nameyour research might be giving more away than you realize.

If Google has a product that rivals its search in ubiquity, its Gmail. Privacy-focused folks may remember that in 2017, Google had a bit of a scandal when it was revealed the company scanned the content of emails to better target advertisements to customers. For ordinary email users, that is a privacy violation. For professionals who may have other peoples personal and financial information in their inboxes, that might be an ethics problem.

Luckily, there is no shortage of Gmail alternatives, and many have privacy as a top line feature. Among the most oft-recommended services is Tutanota, a service out of Germany that boasts end-to-end encryption. This means, at least in theory, no one can read the contents of your inbox. Similarly, Proton Mail encrypts everything related to your account and signs your emails using PGP (Pretty Good Privacy) keys so recipients can be certain an email from you is indeed an email from you.

When client data is sent by email, the email itself and any attachments that contain personal information should be encrypted and put behind a password, regardless of what email service you use. You dont know who may be listening in on the receiving end.

Google Sheets is a popular tool with folks who spend their days crunching numbers but has many of the same privacy concerns tied to other Google services. Someone who has access to your Gmail account has access to your Sheets and anything else stored in the larger Google Drive service. A spreadsheet that is accessible from any browser can be useful for tracking client data, and there are even templates for doing things like estimating quarterly tax paymentsbut convenience comes with a security tradeoff.

Many privacy-focused users are looking at services such as CryptPadnot just because it claims end-to-end encryption and is open source, but simply because keeping everything in the Google ecosystem seems a bit like that old saw about eggs and baskets. CryptPad allows users to remain completely anonymous, which may reduce ones risk of exposing client data in a targeted attack.

The use of Google Drive grants relatively permissive terms of use to Google for your documents. As free cloud hosts go, that isnt bad, but it may be a nonstarter for folks that need to store private or sensitive information. Additionally, and as mentioned above, the tying together of the various Google services behind one Google account is convenient but creates one point of entry for an attacker to gain access to your entire digital world.

Finding alternative private cloud providers to the big players (such as Google Drive, Microsoft OneDrive, and Dropbox) is not as simple as with email and spreadsheets. The big players are best in a position to provide storage services for rates that reflect their ability to pay for storage at scale. As such, the privacy-focused answer for using cloud storage is a bit different from the above.

As with all things, inexpensive storage comes with a tradeoff for tax professionals: file retention and destruction issues. The more cloud storage you have, the less motivated by space constraints you will be to periodically prune your client data, and that may be a problem when client data is retained for periods longer than necessary.

In sum, use one of the major players, but encrypt your data prior to uploading it and regularly delete client data you no longer need to retain. Cryptomator offers an open source and free tool that streamlines the process. The result isnt perfectwith enough time, any encryption method can be crackedbut its the best solution that isnt simply not using cloud services.

If the above hasnt convinced you to make a privacy move, at least make sure you are using a secure password for Google that isnt used anywhere else, and turn on two-factor authentication. If possible, do not have your two-factor authentication codes sent by SMS to your cellphone, and use a code-generating application like Google Authenticator or Authy. If you do use SMS to receive your codes, call your cellphone provider and ask for a PIN code on your account for all changes.

Finally, when handling sensitive informationespecially someone elsesbe thoughtful about when, where, and why you add a piece of data to the cloud. Best practices for data backup include local and off-site backups. Your clients wont thank you when their data isnt divulged in a breach, because that would be weird, but you can sleep soundly knowing that you arent going to have to have an uncomfortable conversation with them the next time a big breach makes the news.

This is a regular column from tax and technology attorney Andrew Leahey, principal at Hunter Creek Consulting and a sales suppression expert. Look for Leaheys column on Bloomberg Tax, and follow him on Twitter at @leahey.

Read the original post:
Privacy-Focused Alternatives to Google Services for Tax Pros - Bloomberg Tax

In many jurisdictions, organizations are now required to store user data in the country where its collected and users live, which creates an enormous technical problem. A sovereign cloud might be the solution.

In a spate of announcements over the summer, Red Hat and Accurture said theyd extended their 12-year cloud partnership to pursue more innovations in hybrid cloud. But the smallest part of their announcementthat theyll collaborate on new products for the sovereign cloudis one of the most exciting, as its an area of data privacy thats churning with regulation and controversy.

In a statement, Raj Wickramasinghe, emerging platform lead at Accenture, said: Organizations are increasingly turning to hybrid cloud to help overcome complex challenges around core business functions like customer service and supply chain and to drive growth and innovation. Through our expanded alliance with Red Hat, we can further help clients embrace the cloud continuum to enable greater operational efficiency and drive innovation.

That sets the stage nicely for the idea of the sovereign cloud, which addresses how multinational organizations handle user data when deploying applications on one of the public cloud providers, like Google, Amazon, or Microsoft. In many jurisdictions, organizations are now required to store user data in the country where its collected, and users live, which creates an enormous technical problem: How do you possibly aggregate and analyze user data from multiple or dozens of different data stores?

The sovereign cloud doesnt relate to those data stores as much as it does the interconnectivity between thema place where data can flow freely and legally in a way that still enables viable applications for global audiences.

Red Hat and Accenture are adding a layer of open-source technologies and services to their sovereign cloud research and development, making it appealing to regulators and governments in the years to come.

See also: Solving for Sovereign Data with Edge AI

As soon as an organization accepts users from beyond a single country, they must start paying attention to the complex network of data privacy/security regulations. Aside from having their reputation on the line, failure to comply with data regulations can result in massive fineswell-well companies like Amazon, WhatsApp, and Google have all been fined hundreds of millions of dollars for GDPR violations.

The US, UK, and EU already have multiple regulations, with precedent-setting rulings, updates, and new initiatives always on the horizon.

The US has the CLOUD Act, which forces US public cloud providers to hand user data over to a government identity or law enforcement agency if they request it via a warrant, subpoena, or court order. That sounds simple enough, but the data requested might also be stored in another country, which creates conflict with that jurisdictions own requirements.

Thats one of the ways the CLOUD Act conflicts with the EUs General Data Protection Regulation (GDPR), which is perhaps the most well-known regulation, as its already caused lots of headaches for developers around storing personally identifiable information (PII) and is why you see so many cookie consent banners on the websites you visit, particularly if theyre based in the EU. Under the GDPR, cloud providers can only disclose personal data for legal requests made under EU law.

With these regulations at odds with one another, its not entirely clear what a cloud provider would do, for example, if the Security and Exchange Commission got a court order to hand over user data thats stored in Germany.

The landscape is dotted with other regulations that make it even harder to track and understand. Schrems II, a legal judgment published in July 2020 for the EU and UK, requires that organizations must individually and manually assess all data thats to be transferred to a non-EU country. The goal was to ensure the target country adheres to EU standards on data and privacy, but it also destroyed any opens of an open US-EU data highway, known then as EU-US Privacy Shield.

Like the CLOUD Act, the UKs COPO Act 2019 also allows UK law enforcement to compel non-UK companies to fork over user data.

Staying compliant with these conflicting rules is an enormous challenge for organizations and the public clouds they use to deploy applications and store user data, and an area thats overdue for concerted R&D.

Organizations can deploy a homegrown sovereign cloud today by working with smaller regional storage providers in target jurisdictions or deploying their own on-premises private cloud storage in every country where they do business.

Virtual data spaces are another burgeoning solution to this problem. If multiple trusted organizations partner together on establishing and maintaining the same high standards for storing and sharing data, they can safely share data without running afoul of any US, EU, or UK regulation. User data is never stored centrally in these spaces and only shared between partners when absolutely necessary.

On that front, GAIA-X is developing a federated European data infrastructure through a network that links many public clouds together. The goal is a European public cloud that respects the digital sovereignty of its users based on transparency and openness. Public cloud providers will have to commit to participating, but GAIA-X already has support from BMW, Deutsche Telekom, SAP, Siemens, Scaleway, and others.

As the general public gets more awareness of digital sovereignty and fines continue to stack up, this issue will only get messier, more complicated, and a lot more popular. Considering how Red Hat and Accenture have nothing concrete to announce, theyre likely years away from having a plug-and-play solution, which would do wonders for startups and small- or midsize-businesses who dont have enterprise-sized wallets.

Read more:
Can Red Hat and Accenture Solve Data Sovereignty? - RTInsights

There are all kinds of digital subscriptions you can sign up for nowadaysfor cloud storage and fitness plans and music streaming and audiobooksand you simply cannot sign up for every single one. Even if you had the money, you wouldn't have the time to get your money's worth.

This brings us to YouTube Premium, the paid plan for the video-sharing site that you and billions of other people happily use for free. So why would you pay $12 a month for it? We're going to outline the perks here, so you can decide whether it's worth the price.

No Ads

Sitting through YouTube ads isn't the worst experience you're ever going to havemost of them can be skipped after a few seconds anywaybut you might be surprised at just how much better the YouTube experience is without the interruptions and hassles.

You can try YouTube Premium for freewhich we encourage you to doand that will give you an idea of the difference. It actually saves a significant amount of time, especially if you're a frequent YouTube user and spend a lot of time on the site or in the app.

Also, if you're using YouTube for music for a party, you won't have your tunes interrupted every few minutes by an advert. It also helps when you're presenting clips to other peopleat work or in a classroom, for example.

You can download videos to your device with YouTube Premium.

Built-In Video Downloads

YouTube Premium subscribers can also save videos for offline playback. This works in the YouTube apps for Android and iOS, as well as in desktop web browsers, so that you can take your videos anywhere.

No longer do you have to put up with a dodgy internet connection or a lack of Wi-Fi ruining your video streaming. Whether you're heading to a park or hopping on a plane, you're able to queue up a selection of clips for your viewing entertainment.

In the mobile apps, there's also a feature called Smart Downloads. This automatically downloads clips that YouTube thinks you will like (based on your recommendations), so you've always got something to watch, whether you're online or off.

YouTube Music

YouTube Music, which ordinarily would set you back $10 a year, is included with YouTube Premiumso you're getting a full music streaming service in the style of Spotify or Apple Music as one of your perks.

While YouTube Music might not be quite as feature-rich or as comprehensive as some of its rivals, it has everything that most people need. You can run it through a web browser or through the dedicated mobile apps, and there's the option to sync tracks to your devices for offline listening.

Read more from the original source:
YouTube Premium Has Its Perks. Here Are Some to Consider - WIRED

Dell Technologies has announced co-engineered developments with VMware covering VxRail hyperconverged appliances, special edge site VxRail systems, APEX cloud subscription offerings, and validated AI systems with automated machine learning.

The announcements were made at VMwares Explore event against the backdrop of Broadcoms pending acquisition of the virtualization company.

Jeff Boudreau, president of Dells Infrastructure Solutions Group, said: Dell Technologies and VMware have numerous joint engineering initiatives spanning core IT areas such as multicloud, edge and security to help our customers more easily manage and gain value from their data.

There are actually three VxRail announcements: VxRail with DPUs, VxRail and vSAN ESA, and VxRail modular edge nodes.

VxRail systems now support vSphere 8 software that has been designed to run on server-attached DPUs (data processing units) such as Nvidias BlueField-2. ThisvSphere Distributed Services Engine software is part of the Nvidia-VMware Project Monterey that offloads hypervisor, networking, security, and storage tasks from the CPU to the DPU. No configurations have been announced yet but we performance stats with some anticipation.

Dell told us that PowerEdge servers will support onboard DPUs as well.

Certain VxRail configs support VMwares new vSAN Enterprise Storage Architecture (ESA), which can deliver an up to 4x vSAN performance improvement through using NVMe technology. This vSAN ESA is a single-tier HCI storage setup and all NVMe VxRail models are optimized for it.

Dell has also announced rugged and modular VxRail systems for edge locations, the smallest form-factor VxRail systems to date.

We were told in a briefing that these new ruggedized modular nodes can be racked, stacked, and mounted for use anywhere in almost any environment and scale from two to 64 nodes. Theyre specifically designed with half-width blades and are the industrys first onboard integrated and purpose-built hardware witness.

A hardware witness is an ESXi host in a stretched cluster that stores virtual machine (VM) metadata, known as the witness components, for each VM object. If a host in the stretched cluster fails, the other sites may not be able to decide which one is the master or authoritative site. The witness provides a so-called vote to build a quorum and decide which is the main site. Then VMs from the failed site can be vMotioned to the new main site and restarted.

The onboard hardware witness removes the need for remote datacenter infrastructure power and administration thats typically required with a separate witness node.

We are envisaging non-rackmount chassis, and thinking of wall-mount systems and table-top designs. Dell envisages these mini-VxRail systems being used for healthcare, energy and utilities, and digital cities with deployments in high-latency, low-bandwidth locations.

Scale Computing has had a strong focus on edge location HCI systems for some years. For example, it has a cigarette box-sized HCI edge appliance, the HE150. This uses Intels NUC micro-PC processing hardware. We dont know if VxRail will come this far down the small system scaling curve.

APEX Cloud Services with VMware Cloud adds managed VMware Tanzu Kubernetes Grid services, meaning customers canprovision Kubernetes clusters and containerized workloads through the vSphere user interface.

APEX Private Cloud and APEX Hybrid Cloud offer new compute-only options so customers can scale compute and storage independently. They can connect compute-only instances to Dell storage such as APEX Data Storage Services.

Dell Validated Designs for AI Automatic Machine Learning (AutoML) uses automated machine learning models to help data scientists develop AI-powered applications. It includes validated configurations of VxRail hyperconverged infrastructure with H2O.ai, Nvidia, and VMware software with automation that delivers up to 18x faster AI models. This is according to an April 2020 Total Economic Impact Study Commissioned By Dell and Intel.

Link:
Dell boosts VxRail with DPUs running vSphere Blocks and Files - Blocks and Files