Anthony Malakian: The Swimmers in the Lake
![antony-malakian antony-malakian](/sites/default/files/styles/landscape_750_463/public/import/IMG/001/339001/antony-malakian.jpeg.webp?h=e7ef40b7&itok=8dKJfh17)
The arms race for programmer talent has been a long-discussed topic in Waters, and there’s no reason to think it won’t continue to be a challenge for the foreseeable future—at least until our robot overlords finally wrest control of civilization from our hands.
While at the Buy-Side Technology North American Summit, held in mid-October in Manhattan, one panel discussed the merits of deploying data lakes versus data warehouses. The debate was interesting and led me to write a feature on the subject. It also got me thinking about the talent challenge, but more on that in a minute.
Two of the panelists—one from MetLife, the other representing JPMorgan Asset Management—were fairly evangelical on the subject of data lakes. The funny thing is, however, that their definition of a data lake would, I imagine, differ from some hardcore programmers. JPMorgan’s Scott Burleigh, in fact, said that his firm governs the data before it enters the lake, which I think many would argue makes the lake more of a warehouse than a pure lake.
But I also think that this is a natural progression of new technologies and strategies: The definition is morphed, stretched and molded to fit a particular industry. The capital markets space is likely to have a different view of data lakes than that adopted by music streaming services, in the same way that Wall Street had a different definition of the cloud compared with Silicon Valley where the technology first emerged.
Limited Skillset
I was speaking with Gartner’s Nick Heudecker on this topic, who has written a research paper on data lakes. He has also noticed that the term data lake is being thrown around to describe a lot of things. It’s even been used interchangeably with Hadoop. The key, he says, is to look at the underlying architecture if you really want to see what it is.
You’ll need innovative programmers sitting alongside the data scientists and quants to help extrapolate value from these lakes.
“If I’m building a very detailed semantics layer and add security in the way that I would do in my data warehouse, and I’m optimizing data for storage or reuse, is it really a data lake, or is it a different implementation of a data warehouse architecture? People confuse the physical implementation of the data warehouse with the architectural considerations,” he said. “And that’s fine; call it what you want. But I want to know about the architecture and from there we can figure out what characteristics you’re going to need.”
The real question is, who is going to build that architecture? As Dan Graham of Teradata told me, a data lake is “limited to what a programmer can do. It’s up to a programmer to do a lot of do-it-yourself work in the data lake.”
At some point, this raw, unstructured data is going to have to be cleaned and used, otherwise it’s just a data dumping ground without a purpose. And the cleaning part might not be as easy when it’s on such a massive scale.
“All of these data cleansing techniques are a foreign idea and a new area of exploration for the Hadoop data lake people,” Graham noted.
The problem for capital markets firms is they don’t want to have to hire separate cleansing specialists to go along with their Hadoop specialists.
Data Needs Programmers
I’m tired of the term “big data.” It’s too all-encompassing and vague. But as the volume of data grows at exponential rates—according to IBM, 2.5 quintillion bytes of data is created daily—a term needs to be created and accepted, so for the time being let’s just go with big data.
If firms want to find purpose and value in that sea of information, data lakes will prove to be a starting point. It’s like oil refining: you collect it, separate it, convert it, and treat it. You want to collect as much as possible, but the art is in the downstream stages.
You’ll need innovative programmers sitting alongside the data scientists and quants to help extrapolate value from these lakes. So, if you think the talent race was going to slow down anytime soon, you’re wrong.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@waterstechnology.com
More on Emerging Technologies
Reading the bones: Citi, BNY, Morgan Stanley invest in AI, alt data, & private markets
Investment arms at large US banks are taken with emerging technologies such as generative AI, alternative and unstructured data, and private markets as they look to partner with, acquire, and invest in leading startups.
Startup helps buy-side firms retain ‘control’ over analytics
ExeQution Analytics provides a structured and flexible analytics framework based on the q programming language that can be integrated with kdb+ platforms.
The IMD Wrap: With Bloomberg’s headset app, you’ll never look at data the same way again
Max recently wrote about new developments being added to Bloomberg Pro for Vision. Today he gives a more personal perspective on the new technology.
LSEG unveils Workspace Teams, other products of Microsoft deal
The exchange revealed new developments in the ongoing Workspace/Teams collaboration as it works with Big Tech to improve trader workflows.
IBM report finds ‘shadow’ data significant contributor to data breaches
As AI and cloud take on greater importance in the capital markets, firms need to consider their threat impact zones.
Bloomberg adds AI earnings summaries to Apple Vision Pro app
The vendor continues to add content and functionality to its Bloomberg Pro for Vision app, which sits at the convergence of spatial and mobile computing.
SS&C continues Blue Prism rollout, eyes other acquisition targets
The company is focusing on organic growth while keeping its eye on potential acquisitions.
CME: CFTC OKs clearing move to Google Cloud
The CFTC has given the Chicago-based exchange approval to run its clearing and settlement infrastructure on the Google Cloud Platform, while the exchange and vendor have extended their partnership to last until at least 2037.