Plumbing the Depths of Unstructured Data

Can collaboration and sharing of data, within the context of adhering to a standard when devising data governance plans, help solve issues with handling unstructured data? This question arises when hearing how firms try to build data foundations to produce higher quality data analytics.

Data governance and analytics can be improved by building mechanisms to share data, says Leif Hanlen, a business development executive at Data61, a data requirements organization that is part of CSIRO, a digital development agency backed by the federal and state governments of Australia. 

Yet, however advanced a data sharing system is, the rules that data vendors and exchanges have about their data make it challenging to join such disparate sets together, says John Denheen of Tyler Capital in London. “One of the big issues with getting good reference data is that you need to be able to tack it on to other data sets,” he says.

Before compatible standards can be achieved, firms have to identify the business practices that differ, says Sydney Hassal, a director at Scotiabank. Along with this, standards can vary among jurisdictions, observes Allie Harris of Bank of Montreal. 

The US Office of Financial Research (OFR) points to “microdata,” which OFR chief data officer Cornelius Crowley defines as granular, targeted and specific information about entities, instruments, transactions or products. He advises that breaking down standards efforts to individual data elements like those he calls microdata can increase accuracy. In this context, one wonders whether unstructured data can be organized with a “microdata” approach—and what that can mean for following standards.

Harnessing unstructured data could be most useful to support consistent application of data standards.

Analytics Goldmine

Unstructured data is emerging as a “goldmine” for the application of data analytics, CSIRO’s Hanlen says, although it is nothing new—it is akin to “50 million Word documents saved over the last 10 years and put in a box.” 

Firms should now understand the two types of unstructured data, says Hanlen. “Unstructured data sitting inside the enterprise—in the customer relationship-management system, in fields called ‘other’—is like a hole in the ground that’s yet to become a goldmine,” he says. External unstructured data from sources such as social media and market surveillance is already being tapped for its golden value, he says.

Analytics can come into play for the former type of unstructured data. “The task for analytics of unstructured data is not to build a brand new goldmine, but to extract elements of information from that unstructured data,” he says.

CSIRO and Data61 have worked on analyzing payments, documents predicting market outcomes and price signals based on the likelihood of future events. The challenge Hanlen finds in working with such unstructured data of a predictive—rather than measuring—nature is placing that data in a credible spectrum, which can be used to anticipate the future or a likely outcome.

Unstructured data may be a goldmine, but it is one that industry experts are just figuring out how to tap, for a variety of purposes. In March, ISITC executive Jeff Zoller pointed to unstructured data about investment behavior and patterns, as well as industry analysts’ commentary, and social and economic behavior, potentially supporting the disruption of data-management technology. In May, MUFG Canada Branch chief information and operations officer Ron Lee, speaking about how his firm is consolidating systems, noted this effort would increase the capability to work with unstructured databases.

Before any of these more aspirational purposes, however, harnessing unstructured data could be most useful to support the consistent application of data standards. The legal entity identifier (LEI) is ripe for this, as Karla McKenna, head of standards at the Global Legal Entity Identifier Foundation, observed in June, saying entity legal form information—a data element in the LEI repository—remains in “unstructured form.”