Why To Run Open Source Data Technologies in Your Own Cloud Account

Organizations have an increasingly important choice to make when enlisting managed services or database-as-a-service (DBaaS) providers for their open-source data technologies: go with one of the many providers that require data management to be hosted within their own domains, or insist on running in your own (cloud) provider account (RIYOA). 

In general, most DBaaS providers offer the simplicity of having just one bill to pay, which covers all cloud costs and instances. However, that simplicity comes at costs beyond the monetary—reduced visibility of what’s going on with your data, limitations to developer flexibility when trying to utilize data-layer technologies and restrictions on access to data itself. While less of an out-of-the-box option, retaining RIYOA control avoids these shortcomings and makes it significantly easier to shift to a new provider if a change becomes preferable at some point.

Here are five reasons why RIYOA is an option worth considering for enterprises leveraging open-source data technologies:

Efficient Data Access

Developers that rely on externally-hosted data technologies lose efficiency by placing their own data within the black box of their DBaaS provider. Whereas RIYOA developers can decide to pull a quick query and examine a specific relationship in a table at their convenience, developers with hosted data have to ask the host for access and accept whatever latency is involved. Without direct access to pull ad hoc queries efficiently, organizations miss out on timely data delivery and insights. For example, use cases where a business must rapidly communicate detailed information to customers may become impossible. 

Many high-value use cases also require the ability to harness previous data and compare snapshots in time to understand how data has changed. While that information is available in data backups, teams with externally-hosted data find that access to those backups is severely restricted. The team simply wants to grab backup data to load on another cluster and run analytics or other processes on it. However, doing so means submitting a ticket to the host, getting approvals to remove backup data from where it resides, and finally waiting to receive that data. These procedures are especially cumbersome because DBaaS hosts keep backup data for the sole purpose of restoring it themselves, not for data utilization or any more flexible purpose (even though that past data can have myriad use cases for organizations). In these scenarios, requested data can take days or weeks to arrive…and, oftentimes, some pestering. Give the same team RIYOA ownership of their own data environments, and open access to backups will have them running queries and putting data to work without delay. 

Visibility for Optimization

Direct RIYOA access to data and data environments can inform operational efficiency and optimization as well. If, for example, a team running Apache Kafka cannot access the schema to visualize data and understand data transitions, developers won’t be able to recognize optimization opportunities. It may be that reducing a deployment’s node count is ideal, but the monitoring required to reveal that insight is simply unavailable.  

With RIYOA, monitoring data is available immediately — bringing transparency to workload optimization opportunities. By enlisting a managed services provider that acts as a flexible partner, enterprises can work in collaboration to right-size data usage and maximize operational efficiency.

Flexibility To Change (Just Like With Open Source Itself)

Open-source data technologies offer teams the freedom to use solutions however they wish—and switch to new ones as necessary — while avoiding proprietary limitations and vendor lock-in. However, DBaaS hosting providers often have “open core” derivates that place layers of proprietary black box software on top of true open-source solutions in order to achieve vendor lock-in and make it difficult to move away from their services. In some cases, open-core DBaaS services require teams to connect to an API and then to their own database, or offer serverless models. This forces teams to change their data models, from tables to keys to data partitions, and for developers to adapt how they write applications and ping the cluster.  

RIYOA preserves the freedom and flexibility intrinsic to open source, allowing enterprises to pursue the best strategies for their own products rather than catering to their DBaaS host’s needs. RIYOA also means avoiding vendor lock-in, and always having the freedom to pick up your data and go. With RIYOA, a managed services provider simply provides a control plane for data monitoring and management assistance. If an organization sees fit to move on, the provider removes that control plane and the organization can take its same open-source cluster and environment elsewhere. 

Avoid Security Paperwork and the Appearance of Risk

RIYOA can be particularly appealing to enterprises because it allows them to avoid a lot of security paperwork and related customer communications that may be poorly received (or interpreted). When a business leverages a DBaaS hosting provider, it stores its customers’ data in that host’s data center. The business is therefore required to notify all customers that their data is hosted by a third-party company, which can make many end users uncomfortable. 

Enterprises leveraging a RIYOA approach with managed services provider security support can still get requisite security protections but without the potentially-worrisome customer communications. Enterprises can also move faster with new deployments, or even a shift to a new provider, with no need to make announcements or meet the legal burdens of moving data to a new third-party entity. 

Harness Multiple Database Technologies

Hosted DBaaS products feature prebuilt control plane software designed to let organizations use a particular database, with no in-built flexibility to connect to other databases. Enterprises must therefore utilize connectors and APIs to achieve database connections, substantially slowing progress on their own applications. Teams attempting to combine multiple black-box DBaaS products can face extremely complex challenges. 

In contrast, a RIYOA strategy allows organizations to easily leverage multiple database technologies in their pure open-source forms in combination, with no black box limitations getting in the way. I’ve seen, via Instaclustr by Spot, that this database flexibility component of RIYOA is increasingly important to businesses.

Run Open-Source Data Technologies Your Way

Reliance on DBaaS hosting remains rife with limitations and lock-in risks that many organizations are unclear about before they make critical decisions. By running your own cloud provider account on a managed service, your organization can still receive all the benefits of partnering with an expert technology provider, while retaining all the freedoms and flexibility needed to make the most of your open-source data solutions.