For many in the technology industry, big data is joined at the hip to a second major trend – artificial intelligence (AI). By harnessing AI tools to examine large data sets and uncover new insights, proponents believe significant value can be added in new and exciting ways.
However, herein lies a big challenge as most organisations are not in a position to use AI tools to analyse the big data reserves they already have because much of it is stored in silos and not readily accessible. Industry research shows most organisations can only analyse a small portion of the data they are holding. Research also shows the majority of the data within most organisations is not used for analytics activities at all.
Taming the data beast
There are a range of reasons why organisations have not been able take full control of their data. Restricted budgets, legacy technology, lack of skilled people, and security concerns are among them.
One of the biggest, however, has been dubbed “mass data fragmentation”. This term describes the large volumes of enterprise data that are siloed, scattered, and copied both in on-premises locations and within cloud platforms. This, in turn, leads to an incomplete view of the data and an inability to extract real value from it.
Fragmented data often sits in siloed infrastructure used to “try” and manage backups, archives, object storage, and file shares. These legacy products frequently do not integrate. And, that issue can be exacerbated by duplication of data between on-premises facilities and cloud platforms. This frequently leads to data sets that are hard to locate, let alone analyse.
This problem also creates other issues. A lack of visibility of all data can put an organisation at risk of non-compliance with strict data privacy regulations. And, with laws such as GDPR now in place, it makes the challenge a board-level issue.
Confronting the fragmentation challenge
Overcoming the challenge of data fragmentation can be difficult, but it’s certainly not impossible.
Because increasing numbers of organisations are making use of hybrid IT infrastructures, where cloud resources are utilised in conjunction with solutions on-premises, there is a need for a software platform that supports both environments. The platform needs to be able to manage long-term retention, full life cycle disaster recovery, and backups while also delivering the ability to bring critical applications and analytics directly to the data – versus having to move the data to the applications.
Once such a platform is in place, organisations will find it easier to realise the true business advantages of big data and AI. And, being able to run applications directly on top of an organisation’s entire data set can empower organisations to gather insights that previously have not been possible.
The bottom line is that data has to be accessible and manageable, regardless of where it is being stored. By taking the steps now to ensure this occurs, organisations can position themselves in the best possible way to take advantage of opportunities in the future.
Rather than being held back by data fragmentation and silos, organisations will be able to unleash the real value that exists within their big data stores.
Steve Coad, ANZ country manager, Cohesity
