The Next Big Data Debate Emerges
![michael-shashoua-waters michael-shashoua-waters](/sites/default/files/styles/landscape_750_463/public/import/IMG/317/167317/michael-shashoua-waters.JPG.webp?h=acfe3244&itok=ceJMABf4)
The ongoing discussion about big data, which continued last week in Waters' Big Data Webcast, appears to be turning away from a debate between using cloud computing or the Hadoop standard to a concern with rapidly increasing volume and velocity of data creating a need for greater use of big data systems.
An unspoken context underlying the webcast discussion, which had participants from Credit Suisse, BNY Mellon, Intel, IBM's Platform Computing and Sybase, is that the industry already seems to be leaning or moving away from Hadoop and toward cloud as being more effective for handling big data.
"The cost per gigabyte of storing that transaction over time is pushing us into cheaper, non-SQL, big data-type solutions," said Ed Dabagian-Paul, a vice president at Credit Suisse who works on setting strategy and direction for technology infrastructure at the firm. "The traditional big data solutions haven't mapped to our problems. We can answer most of our existing problems with existing data analytics or very large databases."
Daryan Dehghanpisheh, global director of the financial services segment at Intel, identified "volume, variety, value and velocity" as the four pillars of big data. He had already noted volume, and processing speed and time as key areas for big data when speaking with us in November.
Intel works with partners to produce solutions for operational issues such as big data. According to Dehghanpisheh, the company aims to achieve complex machine learning, statistical modeling and graphing of algorithms within big data, rather than the traditional business intelligence of query reporting and examining historical data trends. Orchestrating use of metadata and setting data usage policies are important parts of administering big data operations, he adds.
An extensible framework is needed to manage the volume and velocity at which big data now pours forth, as Dennis Smith, managing director of the advanced engineering group at BNY Mellon, sees it. "There are tremendous cost benefits to this from a scale standpoint and particularly looking at volume use cases," he said. Cloud computing inherently offers greater scale, of course, and analytics can be layered onto it or attached to it. As Smith also explains, Hadoop-related technologies, or standalone analytics infrastructures and traditional data warehouses as staging areas may all be ways to manage big data in tandem with cloud resources.
The question to ask, or the discussion to have, now, is how to marry big data, sourced from or processed through the cloud, with analytical systems that can derive actionable meaning from the data, for all its increased volume and velocity.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
US banks seek to open vendors’ black box on green data
Inaugural Fed climate scenario analysis flags lack of transparency around third-party models.
IEX Cloud closure forces fintech clients to seek data alternatives
IEX says it is ditching its unprofitable data arm to focus on its core exchange business, but other vendors believe they can turn a profit from its former client base of fintechs, retail investors and some institutions.
The IMD Wrap: Déjà vu as exchange data industry weighs its options
Max highlights some of WatersTechnology’s recent reporting on data costs and capacity issues facing the options industry, and asks, haven’t we seen this before somewhere?
FRTB data quality issues persist amid shifting implementation dates
Banks are finding market and reference data challenges posed by the FRTB’s standardized model tricky, compounded by uncertainty over when the regulation will take effect.
Cboe pushes rule change to make way for proprietary Opra alternatives
As US options data has grown in volume and cost, Cboe says changing the public feed's governing document would make way for more competition from private alternatives, including its Cboe One Options Feed, launched in 2023.
Waters Wrap: What’s going on ‘Here’? Examining interop’s next move
OpenFin is now known as Here. Anthony explains what the rebrand might signal for the application interoperability space.
This Week: MSCI, Tradeweb, FactSet, LTX, MarketAxess, TS Imagine, and more
A summary of the latest financial technology news.
Regulators urged to promote cyber security investment
Public interest in stopping cyber attacks that could trigger bank runs, says Bundesbank researcher