Big Data Webcast: Analyze This

Aside from the various technologies that firms can implement to support their big data initiatives, they still need to develop a workable, coherent big data road map, which, even for the most battle-hardened participants, is a significant technology, time, and financial challenge. And then there are the regulatory implications to consider as regulators, irrespective of their locations and domains, continue to squeeze market participants under their purview, requiring them to store and retrieve on demand data across the organization for regulatory compliance purposes. It’s a big ask for even the best resourced and funded organizations, although there are significant business benefits to be gleaned for those more proactive firms.            

Technology and Operational Challenges
Recently, Waters gathered together a panel of specialists for a virtual roundtable where the big data technology and operational challenges facing financial services firms were discussed, along with the specific business applications for big data platforms and how analytical techniques can assist organizations when it comes to processing, storing, and managing both historical and real-time data more effectively. Also on the agenda were the issues around the best ways for firms to take their first tentative steps down the big data road, and the potential pitfalls they need to negotiate so that they don’t make the same mistakes experienced by the technology’s early adopters.

Roundtable participants included Alasdair Anderson, global head of architecture at HSBC; Gary Bhattacharjee, executive director, enterprise information management at Morgan Stanley; Patrick Angeles, director, field technical services at Cloudera; and Peter Simpson, senior vice president of research and development, data visualization at Datawatch. Victor Anderson, editor-in-chief of Waters magazine, moderated the event.   

Tools
Providing analysts on the buy side with a more extensive set of tools to underpin their research and investment decisions appears to be one of the most obvious direct business benefits offered by big data platforms, according to Cloudera’s Patrick Angeles, who adds that from a sell-side perspective, the low-hanging fruit will have more of a compliance flavor. “Through the ingestion of more data and the provision of  analytical capabilities that have until now not been available—I’m talking not just about traditional business intelligence but also machine-learning tools—they are able to generate or perform research in ways that have not been possible before,” Angeles says. “This is specific to the buy side, but on the sell side, the requirement to store transactions and provide them for auditing purposes and detect activity and behavior—that is more compliance-driven.”    

Datawatch’s Simpson echoes Angeles’ sentiments, although he cites a number of additional areas where buy-side and sell-side firms might utilize a big data platform to help them more accurately monitor any number of business processes in order to generate something akin to a “big picture” across the business.

“It’s about monitoring what’s happening now—intraday and historically—and that’s across the trade lifecycle from pre-trade analytics through portfolio and risk order monitoring, and then surveillance and compliance, both from a broker and a regulatory point of view,” Simpson says. “And then there is also latency monitoring—it’s that kind of view.”

According to Simpson, the analytics underpinning big data platforms are both “trivial and complex,” which, at face value might seem paradoxical, but on hearing his qualification of that statement, is logical. “It’s very simple things like users requesting historic open-high-low-close (OHLC) charts and data-volume bars, or providing them with the 95th percentile returns so that they can calculate their Value-at-Risk (VaR) numbers,” Simpson explains. “That can be very different on certain platforms and trivial on others. Our customers are looking for the kind of calculation to go from having things five years ago that calculated over a nightly run, to calculating sub-second.”

Past Mistakes
Morgan Stanley’s Bhattacharjee explains that the firm’s big data strategy is approximately two-and-a-half years old, and during that time, he and his colleagues have learned a number of lessons from the technology. “Lesson number one is not to treat a big data project as a science experiment—it has to be aligned to the business,” he says. “Yes, it’s new technology and it’s exciting to work with, but if it’s not aligned to a specific business driver, it becomes a bit of an albatross.”

The second lesson, according to Bhattacharjee, is that if firms want to embark on big-data projects, they should employ an agile methodology, which means that they typically won’t go through the traditional development cycles that they ordinarily might have.

“It will be a more iterative fashion where you try to ingest a lot of the data and drive some analysis out of it, provide that to the end-user in a quick timeframe, get some feedback, and then change your process,” he says. “One of the advantages of using things like Apache Hadoop is that it allows you to do that in a much more efficient way because you don’t have to define the schema before you start your process—you can ingest all the data you have across all its different sources and then define some analysis once all the data is in Hadoop.”

According to HSBC’s Anderson, his firm’s big data journey is about six months behind that of Morgan Stanley, and that HSBC took what he describes as a “very cautious” approach, prefering to learn about the technology before committing to it. In this respect, Anderson says HSBC conducted a number of proofs of concept in order to get a feel for the technology.

“We, like many other financial institutions, are globally dispersed and we wanted to ensure that this technology we were rolling out wasn’t the stuff of rocket scientists,” he says. “We used the hype of the technology and staged a hack-a-thon event where we built our big data platform and allowed all the developers to build applications that had, within a business context, used the value of the information within the big data platform on an app basis. We didn’t restrict them in terms of what they could build—we let them go at it without a single business use-case—and out of that event we had multiple applications built using significant amounts of the analytical processing, and that allowed us to demonstrate the potential value of the platform to the business stakeholders.”

Possible Pitfalls
According to Anderson, one of the pitfalls awaiting unsuspecting developers is the velocity of change within the industry, which, from a technology and a vendor perspective, has the potential to cause headaches for user firms when it comes to choosing the most appropriate technology for their needs and sticking with that decision.

“I think you need to plan in order to evolve and this is not going to stop anytime soon,” he says. “If anything, we seem to be in a diverse phase of technology and you need to be able to adapt. Going all-in on a certain platform strikes me as very risky right now. Also, building a big data platform and throwing everything into it—there are no magic beans here. Your stuff doesn’t automatically integrate, and therefore you have to make a lot of investment in developing your information and those analytics. It’s easier and quicker to do but it doesn’t happen by itself.”