Metadata = Data Intelligence

A lot of us may never pay attention to it but metadata has been around for the longest time. Metadata is usually defined quite simply as "Data about the data". Metadata tells us about the data that is stored, displayed or used. It gives purpose to the content by telling what the content of the data is; who is the owner; when was it created; when was it modified; what version is it; where did it originate from and so on.

Metadata provides intelligence to the content of the data that we have and yet, metadata is rarely grabbing the limelight. Well, my entry this time is to promote metadata because metadata enhances the quality of the data and making it easy to search.

This week I am in the UK and attending Enigma Data Solutions training in Uckfield, United Kingdom. And I have been enlightened how critical metadata is to Enigma Solutions such as PARS (Project Archiving and Retrieval System), and IDS (Intelligent Data Store). It is the metadata that gives the value of the seismic data in the Oil and Gas industry.

Enigma Data Solutions HQ in Uckfield, East Sussex, United Kingdom

And with the capacity of the data growing larger and larger and the number of files involved moving into the billions, the data management environment in Oil and Gas can no longer work with IT backup such as Tivoli Storage Manager (TSM) or Symantec NetBackup. That is because these IT backup solutions backs up or archives files.

Unfortunately in a seismic environment, geo-scientists do not deal with files because seismic projects work in terms of applications. And these applications work with not just files, but tablespaces, links, URL and other proprietary data structures. But IT backup and archive works almost exclusively with files and no other data structures.

This means that the seismic applications and geo-scientists have to accommodate the confines of the file data structure. Hence this is the case of the business having to suit IT, not the other way around. Such practices make the data management in the Oil and Gas industry extremely inefficient. Dealing with files, especially retrieving information from archived data can take days, weeks or even months. In an industry where the sudden spike of USD per barrel of oil means millions of dollars of profit, the Exploration and Production (E&P)section cannot afford to retrieve their data the IT way. It is just too slow and too inefficient.

That is why when seismic data is archived, metadata input either manually, automatically or even extracting for seismic applications greatly enhances the quality of the data. Metadata enhances the search quality and speed. It allows data to be restore to its exact environment quickly whereas the IT way of using files will require reconstruction of the working environment and this could take days and weeks.

Therefore, do not underestimate the power of metadata. And with more and more data online and going to the cloud, the reason to have better quality of data lies with metadata. Even though there will be arguments and different schools of thought regarding metadata, it will continue to enhance our user experience when dealing with massive amount of data. There is no denying metadata makes a lot of sense.

In the end, we get intelligent data by applying metadata.