Executing against data in the multi-cloud

Author: Beatrice

By Mark Balkenende, Vice President of Product Marketing at Matillion

To call multi-cloud a mere trend is an understatement, as it is quickly becoming central to an organisation’s digital playbook. Multi-cloud strategies are described by 21% of businesses as their ‘most important initiative’, according to IBM’s State of Multicloud report, with nearly one third of organisations now using three or more clouds.

Most organisations are tapping into the speed and agility of multi-cloud architectures to get apps into production quicker and innovate faster. But what about doing the same for data?

When it comes to data warehousing, it’s generally a case of strength in numbers (two or more clouds are better than one) given the variety and complexity of modern data. Most multi-cloud networks can accommodate a combination of cloud platforms and cloud data warehouses (CDWs), which promises businesses increased flexibility and diversification. In fact, there are several common use cases and reasons that we see every day:

● Data recovery. Multi-cloud architectures allow organisations to distribute their platforms and warehouses, meaning they can be safeguarded against cloud outages.

● Tools for different needs. A diversity of cloud environments is key to meeting the needs of data and IT teams, ensuring that those using different technologies and datasets to support different lines of business are fully supported.

● Vendor diversification. Should price, storage or compute offerings change on the vendor’s side, or should there be an increase in data demand, multi-cloud enables your organisation to be more flexible. Those more closely tied to legacy, data-intensive applications and platforms will also find multi-cloud helps to avoid vendor lock-in.

● Meeting different regional needs. Being able to meet data compliance and sovereignty requirements, and scale resources up and down, in different markets that make most sense for your business.

However, multi-cloud isn’t a silver bullet solution and, like any technology, comes with its own data risks and challenges to navigate. Storing and effectively executing against data in the cloud, and safeguarding against serious data challenges, starts with a solid underlying strategy.

Breaking down data siloes

Often, a multi-cloud configuration produces data silos which can lead to inconsistencies in output. Data is stored in various data warehouses and lakes, spread across multiple platforms and locations, with which individuals apply their own rules. As people work on different (and an increasing number of) streams of data, it becomes more difficult for businesses to have that ‘single source of truth’ that they so desperately seek, as well as become truly data driven.

Portability is another contributing factor to creating data siloes. With a growing amount of data stored in different formats across a variety of technologies, organisations find it increasingly difficult to navigate this lack of interoperability between providers. Vendor lock-in is therefore prominent, and while there are portability solutions on the market, they remain expensive to obtain and maintain.

While data silos and lack of portability are ongoing issues for individual teams, moving data between platforms or regions poses a company-wide data security risk without the correct governance and security controls in place. To prevent escalation, business leaders must ask themselves, how can the company protect itself from these data risks while maximising the freedom, adaptability, and cost savings that come with a multi-cloud architecture?

Understanding the available combinations

A key consideration when devising a sound multi-cloud strategy is that no two architectures are the same, and nor are business problems. They can comprise a unique mix of public and private cloud infrastructures or use several CDW/Cloud Data Platform (CDP) providers, such as Amazon Redshift and Snowflake. It may also be that you host operational data stores in AWS but are migrating and analysing data in Azure, for example. In the case of multi-cloud, you might be doing all of this simultaneously!

What remains important regardless is that you have a unifying data management layer, which allows for the secure passage of data across storage layers, warehouses, platforms and environments. It lays the foundation for ‘cross-cloud’ data sharing, whereby organisations deploy a single type of CDW that can operate on multiple cloud data platforms. By way of example, Snowflake customers can launch their CDW/CDP on AWS, Google Cloud and Azure. Flip the configuration, and you can use a single underlying cloud platform with multiple CDWs.

Maximising value from your data

Multi-cloud infrastructures are not one-size-fits-all, but whatever solution is opted for, a strong data transformation and loading process is key to extracting optimal value from your data. Although complex without the right integration strategy, these architectures help data teams be more productive, flexible and efficient when demands to scale your business up and down are high. When supported by a robust data strategy, companies have more opportunity with the data they extract and can make more informed decisions across the organisation – the end goal for any business investing in the cloud.