Opening Cross: Mo’ Data, Mo’ Problems
Big Data: Everyone’s talking about it, but what is it, and more importantly, can it make money? Basically computation of such large, numerous and sophisticated datasets that the process requires new paradigms of storage and analysis, Big Data presents big challenges to be able to address it correctly, but creates big opportunities for those who can do so.
In the past, a firm’s value was in the experience of traders, and their ability to assimilate information and spot, recognize and react to events. This task became onerous as data volumes increased, prompting firms to translate that experience into algorithms with greater capacity, which can react faster and more often.
But even the best algorithms risk being overwhelmed by the amount of data out there—especially as firms seek to incorporate new inputs, unstructured data, information on supply and demand, among numerous other content types.
For example, this week, we report on how StarCompliance—a provider of software that monitors the personal trading activity of employees of financial firms to ensure they are complying with the firm’s code of ethics—is using content from news aggregator Acquire Media to reconcile trading activity against news activity to spot potential insider trading by individuals or the firm, using a weighted trailing average news impact score calculated by Acquire on pre-filtered content, so as not to overwhelm compliance officers.
Of course, there are some projects so mind-bogglingly complex that they make this important service look like child’s play. And key to making these understandable (i.e. valuable) at the consumer level—whether that consumer is a human monitoring compliance or an algorithm poised to fire off trades—is reducing complexity early in the process. For example, in this week’s Open Platform, Jeff Wootton of SAP recommends using event stream processing technology to perform “smart capture”—which performs some analytical processes within the data stream itself to reduce the amount of work that must be performed to gain valuable insight once the data reaches its destination.
In a real-time environment, this could be used to create new derived inputs to support trading, but risk is an obvious sweet spot for the application of Big Data. With so many initiatives underway, the best way for firms to address these and ensure compliance is to capture as much data as possible to cover all their bases.
As Waters reporter Steve Dew-Jones reported from last week’s Big Data Online Summit, hosted by WatersTechnology, the “holy grail” of Big Data is “continuous risk analysis,” said independent information architect Tom Dalglish. “Pre-trade risk analysis is where regulation is driving us. If that’s taking 30 minutes, it’s too long; we need it to be in seconds,” Dalglish said.
And Big Data is key to achieving this across datasets as diverse as market data analytics, intraday risk assessment, and real-time data on liquidity and funding, but breaking down traditional business silos to leverage the skills and knowledge trapped in those silos across the enterprise, reported Tim Bourgaize Murray, citing remarks at the Summit by SAP’s Stuart Grant.
But before this can begin, firms must address the people problem of Big Data: finding the right engineers to build a Big Data architecture that will deliver the desired results, and to develop the analytics that will reap those results, reported Anthony Malakian from the summit, citing Howard Halberstein, lead solutions architect for Unix at Deutsche Bank, who said assembling and storing the data is less problematic than knowing how to use it. “When you have that many variables… to correlate… where do you start?” Halberstein said. “I’ve seen things get mired when they get to the analytics piece [because] they can’t get any usable information from this large pool of data.”
In short, while Big Data will be a requirement going forward, the most advantage will be gained by those who can overcome these challenges and be first to use it to their advantage across their lines of business.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
Nasdaq to market new options strike listing tech to other exchanges
The exchange operator is experimenting with emerging technologies to determine which options strike prices belong in a crowded market, with hopes to sell the tech to its peers.
Waters Wavelength Podcast: Bloomberg’s Tony McManus
Tony McManus, global head of enterprise data division at Bloomberg, joins the podcast to talk about the importance of data in the context of AI and GenAI.
Putting the ‘A’ in CDO: The rise of the chief data and analytics officer
As data and analytics become more intertwined, banks and vendors are creating a new role—the chief data and analytics officer—to help them take advantage of the opportunities it presents. It may sound easy, but rethinking data can be a gargantuan task.
The IMD Wrap: Talk about ‘live’ data, NAFIS 2024 is here
This year’s North American Financial Information Summit takes place this week, with an expanded agenda. Max highlights some of the must-attend sessions and new topics. But first, a history lesson...
Waters Wavelength Podcast: S&P’s CTO on AI, data, and the future of datacenters
Frank Tarsillo, CTO at S&P Global Market Intelligence, joins the podcast to discuss the firm’s approach to AI, the importance of data, and what might be in store for datacenters in the coming years.
Breaking out of the cells: banks’ long goodbye to spreadsheets
Dealers are cutting back on their use of Excel amid tighter regulation and risk concerns.
BMO’s cloud migration strategy eases AI adoption
The Canadian bank is embracing a more digital future as its cloud strategy makes gains and it looks to both traditional machine learning and generative AI for further augmentation.
Waters Wrap: GenAI and rising tides
As banks, asset managers, and vendors ratchet up generative AI experiments and rollouts, Anthony explains why collaboration between business and tech teams is crucial.
Most read
- Waters Wavelength Podcast: Bloomberg’s Tony McManus
- IMD & IRD Awards 2024: All the winners
- Waters Wavelength Podcast: S&P’s CTO on AI, data, and the future of datacenters