Regulations either recently implemented or about to come into force aim to keep pace with the increasingly complex and globally interconnected financial markets, which have created many issues for regulators, not least around misconduct. Manipulation techniques are constantly evolving, and traders seeking to manipulate the market to their advantage can try to hide misconduct by spreading orders across multiple trading venues, using different markets or assets, or by exploiting automated functions in the markets. 

In July 2016, the revised EU Market Abuse Regulation came into force to tackle insider trading and market manipulation practices such as spoofing—where a trader creates a false interest in buying or selling, then profits when others are misled into buying or selling away from the real market price. Over the coming months, Mifid II rules will force banks and fund managers to collect much more data at the trading desk level than is currently required, enabling regulators to pinpoint order flows by trader, and even by client, to better spot market abuse and manipulation. Officials at French regulator Autorité des Marchés Financiers (AMF) say that the trading data it will receive following the implementation of Mifid II in January will likely increase tenfold.  

In the US, securities exchanges and Finra will next month begin reporting all of their transactions data through the US Securities and Exchange Commission’s Consolidated Audit Trail system, with broker-dealers to follow over the next two years. This, SEC officials say, will result in the availability of market data for surveillance on “an unprecedented scale.”

Several factors are making effective market oversight more important than ever: the high cost of misconduct and pricing manipulation—in aggregate, around $20 billion in the wake of recent Libor and foreign exchange rate-fixing scandals; financial institutions’ desire to restore trust and confidence in the capital market; and an evolving regulatory landscape that will sanction any firm that fails to report misconduct uncovered at a later date. 

Market participants and regulators are seeking smarter, more proactive approaches to spot rogue activity and to better anticipate any malfeasance that might arise, and are increasingly turning to artificial intelligence technologies—not only to better manage the sheer volume of data entering their systems, but also to provide additional context around specific behavior and to discover new patterns in data.

Where AI Comes In

Artificial intelligence is a catch-all that combines everything from machine learning and deep learning to robotics, natural-language processing and behavioral science. In broad terms, systems and applications that deploy AI are able to make logical decisions based on analysis of structured and unstructured data they consume. Machine learning, a branch of artificial intelligence, enables systems to automatically learn and improve from experience without being explicitly programed to execute on a specific function. In market surveillance, this could mean learning which trading patterns might lead to a prosecution, so a system can automatically alert compliance professionals to these in the future.

All the key ingredients are there for the use of artificial intelligence to flourish in market surveillance: the availability of massive amounts of data, much of it unstructured; the exponential increase in computer processing power; and the declining price and growing convenience of data storage solutions, such as the cloud. 

This year, several regulators and exchange groups have announced plans to use artificial intelligence—in particular, machine learning, cognitive computing and behavioral science tools—to manage and analyze massive amounts of data, pinpoint anomalous behavior and anticipate threats. Many major banks are also either implementing this technology or are piloting schemes.

Technology giant IBM is one company ramping up its AI product line to support the financial services sector with its Watson cognitive supercomputer, which became a household name in 2011 after beating long-time champions on the quiz show Jeopardy.

Marc Andrews, vice president of Watson Financial Services Solutions, says traditional approaches to identifying market manipulation have tended to exist in data silos, where organizations monitor trading activity with analytics, then use different systems to monitor communications, and voice and audio traffic. This, he says, leads to three major problems: false positives becoming a common occurrence; identified incidents not being correlated or substantiated; and other incidents being missed.

“A lot of alerts are based on rules that look at things like: ‘Has there been a significant movement in price shortly after someone made a trade?’ or ‘Did the price of an asset go down significantly right after they traded?’ which alone might not indicate a problem, because good traders that know the market will make those same types of trades. At the same time, financial firms are generating alerts and monitoring traders through lexicon-based approaches, using word searches or keywords in their communications channels, but these can lead to a lot of false positives and extra effort in investigations. Importantly, people can disguise what they are saying and use other terms that might not be picked up,” Andrews says.

IBM Watson’s surveillance offering combines unstructured data, including chat transcripts, email communications and voice recordings, with structured trade data and other types of information, such as HR information and performance data, to create a more unified, machine learning surveillance system that is trained primarily by data as opposed to rules. 

Collective Knowledge

Andrews says the system goes beyond traditional rules- and lexicon-based alert systems by using advanced analytic and cognitive technologies to identify behavioral patterns associated with misconduct. As it identifies patterns, these patterns become part of its collective knowledge. 

For instance, the system can set a baseline for employees’ normal trading activity, taking into account factors like types of trades, assets traded, trade volume, dollar amounts traded, and communication style, then use capabilities such as personality insights and tone analyzer features to identify anomalous behavior, trading patterns or communications that fall outside normal patterns. 

Andrews says IBM Watson is already working with some exchanges, while a large, undisclosed UK bank has been piloting the system for voice surveillance of its FX traders to identify potential misconduct. Another large global bank is piloting the system to reduce false positives and streamline the identification of “risky actors” who they should focus on, aided by alerts generated by a lexicon-based system.  

Other technology vendors offer similar services. AI technology vendor Digital Reasoning also analyzes natural-language communications using its Synthesys system—a machine-learning platform that can understand how people communicate by analyzing the context, content and relationships within communications, while semantically revealing concealed insights. It aims to decipher traders’ “code” to separate innocuous chatter from illegal concealment, manipulation or collusion.

The technology is used by Goldman Sachs, UBS and Point72 Asset Management, among others, as well as by US intelligence analysts tracking terrorists, and law-enforcement agencies to combat human trafficking. For financial firms, along with lingo and colloquialisms, Digital Reasoning trains its algorithms on certain word patterns that could set off a compliance alert. The company says its technology generates person-centric insights and detects anomalies or changes in traders’ behavior or risk profiles, reducing false positives by around 50 percent.

“Digital Reasoning allows us to leverage electronic communications monitoring technology to tie in ‘intent’ with trading behavior,” says Valerie Bannert-Thurner, senior vice president and head of risk and surveillance solutions at Nasdaq, which  took part in Digital Reasoning’s Series D investment round last year—which raised $40 million and valued the company at more than $200 million—and has integrated Synthesys into its Smarts Trade Surveillance system.

Nasdaq’s partnership with Digital Reasoning is significant because it forms part of what the exchange group calls its 360-degree market surveillance offering, which combines machine intelligence, cognitive computing and behavioral science tools with Smarts surveillance alerts, allowing it to analyze data beyond trading records to better spot pricing manipulation. Smarts, which has a library stocked with 210 preconfigured alert scenarios, is already used  by 47 marketplaces, 17 regulators and over 140 market venues globally.

In July, Nasdaq also acquired Sybenetix, a London-based regulatory technology company that uses algorithms to learn the behavior patterns of individual traders at an asset manager or hedge fund and detects when they do something out of character.

“We are moving toward a more holistic approach to surveillance, which considers several different data points in addition to trading data,” Bannert-Thurner says. While Digital Reasoning allows the exchange group to understand intent and meaning behind trading behavior, the acquisition of Sybenetix pushes Nasdaq further into the realm of conduct. 

“This provides the deepest level of insight into the specific profiles associated with individual traders or entities. This was a logical sequence for us. By further enhancing our data—structured and unstructured—we are able to extract more signals to better understand human behavior. … Through the use of sophisticated machine learning, we are able to filter through all of the noise and unstructured data to provide only the most relevant information,” Bannert-Thurner says, adding that the exchange has developed ranking and scoring capabilities, which allow users to prioritize tasks and reduce false positives.

The London Stock Exchange is also blazing a trail in the AI space, complementing its rules-based alerting engines with AI-powered algorithms. “This enables us as practitioners to adopt a fresh approach to detecting suspect activity where we are able to proactively engage in looking at behavior that may cause an issue in the future and potentially give us the tools to detect fraud before it happens,” says Liam Smith, head of market supervision at the LSE.

The exchange group, which operates two equities markets and a derivatives market, worked with cyber-security firm SparkCognition and IBM Watson to give it “a head start” on AI and related technologies. “Following a successful proof-of-concept with our partners, our internal teams have been working on solutions for surveillance. The new enhanced fraud detection mechanisms are already in place, and we are currently working on putting them into full production as well as exploring several other approaches to AI,” Smith says. “These new technologies allow us to deploy efficient algorithms that can detect problems with systems, latency, security threats and market manipulation at their source very quickly, allowing us to act on issues instantaneously.”

The LSE plans to expand its surveillance capabilities by including formerly separate datasets, where patterns of behavior have not been explored. “The ability of our surveillance team to integrate unstructured/semi-structured data with market data will be a powerful tool to detect or deter not-compliant behavior,” Smith says.

Old Watchdog, New Tricks

In addition to financial firms and marketplaces exploring the use of new technologies for surveillance, regulators are also turning to machine learning and AI as they prepare for the swathes of information that will soon be flowing into their systems, generated by new regulatory requirements. 

In the US, self-regulatory body the Financial Industry Regulatory Authority (Finra), which monitors an average of 37 billion stock orders, modifications, cancellations and trades a day—sometimes increasing to as much as 75 billion records—has been developing machine-learning software to enhance and accelerate the computer programs it uses to sift through these massive amounts of data to detect manipulative behavior and uncover schemes that it doesn’t even know about yet. 

Finra runs more than 175 independent patterns, which detect a variety of compliance issues and suspicious conduct across markets. As Finra president and chief executive Robert Cook noted in a speech last month, a new pattern takes months to develop and test, requiring close coordination between surveillance analysts and data scientists to identify the right conduct and to set the appropriate parameters for the program to examine. 

“Recently, we have been working to enhance and accelerate this process through the use of machine learning, by which surveillance analysts identify specific instances of behavior of interest and allow algorithms to ‘train’ themselves to identify additional instances in new datasets,” he said. 

A Finra spokesperson says the regulator’s first pattern incorporating a machine-learning model with a feedback loop is expected to be ready to test by year-end, while other patterns utilizing machine learning are in various stages in the development lifecycle.

While embracing new technologies and working with exchanges to aggregate data and coordinate cross-market oversight, Finra will also soon begin submitting data to the new Consolidated Audit Trail (CAT), which comes into force next month (see related story, this issue). 

The CAT is a single central repository that will collect and track data on all trading activity in the US equity and options markets. Not only will it collect more data than is currently collected by Finra, which presently canvasses 99 percent of US stock market trading volume and about 65 percent of US options trading activity, it will require the submission of more information than is currently aggregated, such as personally identifiable information of individual investors. National securities exchanges, Finra and the SEC will have access to the data held on the CAT system.

As Finra prepares to transition to CAT, Cook underscored the importance of embracing new technologies to adapt to the new regulatory environment. “US securities markets are incredibly dynamic, and surveillance in the post-CAT world must be no less quick to embrace new developments in technology, such as machine learning, in order to innovate and adapt to these developments, including new products and market practices,” he said.

The spokesperson says Finra plans to run its current cross-market patterns and new surveillance patterns that are in experimental mode against data from the CAT when it becomes available. 

In Europe, France’s AMF announced earlier this year that it has developed a new surveillance monitoring system called ICY, built with the support of IT vendor Neurones. Based on big data technologies, the system allows AMF to run analyses on larger batches of datasets than its previous monitoring system to detect market manipulation and suspicious activity. The regulator is rolling out ICY to start receiving Mifid II data from January 3, 2018.

The regulator is also developing open-source machine learning and AI libraries that will allow it to perform various types of data analysis, such as clustering, neural networks and natural-language processing to better detect market manipulation. “The implementation of this platform constitutes a transformation program for the AMF,” says Alexandra Givry, head of AMF’s market surveillance department.

“The first set of algorithms we will implement aim to supplement and improve existing programs in our in-house alert system by reducing the number of false positives and facilitating data quality management and clustering information from different data sources. We have also set up some pure machine-learning algorithms to make very granular data easy to analyze, such as clustering to categorize trading behaviors. The implementation of the AI algorithms is still in progress, but we hope that this will improve the capabilities of market abuse detection,” Givry says.

 In addition, the AMF’s data science team is investigating natural-language processing techniques to monitor messages from forums and social networks to determine whether the context in which a market event occurred is relevant for analysis. “We expect this new technology to help the AMF analysts expand their surveillance capabilities in line with the new challenges of the market and regulatory environment,” she says.

Baby Steps

Though more organizations are going public with their initiatives, IBM’s Andrews says the application of machine learning and artificial intelligence in market surveillance has significant potential to efficiently tackle market manipulation, but is still in its infancy.

“What is likely to happen over the next year is that pilots of prototype applications of these technologies will prove out their effectiveness, and we will start to see a shift from experimentation to implementation. We will see organizations plan for how they will acquire or leverage these technologies, and this will involve getting their data lined up and ready—and, organizationally and culturally, getting the skills ready to be able to take advantage and leverage these types of capabilities,” Andrews says.

LSE’s Smith shares this sentiment, saying that most major financial firms are focusing efforts on collecting data and beginning the analysis process with AI or other machine-learning techniques. “The industry is at the stage where AI is rapidly developing, so it’s fair to assume that adoption rates will be very high in the coming years. This rapid development means that technologies that just a few years ago were considered emerging technologies are now widely available as open-source products,” he says.

While expounding the benefits of AI, market participants also stress the importance of the human element in market surveillance. Rather than replacing humans, sources generally agree that AI and machine-learning technologies will handle the more mundane work of sifting through data in search of activity that may warrant closer scrutiny and further investigation by compliance officers, who will continue to play the lead role in trade surveillance.

This is a view expressed by Scott Bauguess, deputy chief economist and deputy director of the SEC’s Division of Economic and Risk Analysis (Dera) in a speech at the OpRisk North America conference in June. He told the audience that “while the human brain will continue to lose ground to machines, I don’t believe it will ever be decommissioned with respect to the regulation of our financial markets.”

The SEC has utilized both machine learning and big data technologies to extract actionable insights from its massive datasets, and while machine-learning algorithms may help by pointing examiners in the right direction in identifying possible fraud or misconduct, “machine-learning algorithms can’t then prepare a referral to enforcement. And algorithms certainly cannot bring an enforcement action. … So while the major advances in machine learning have and will continue to improve our ability to monitor markets for possible misconduct, it is premature to think of AI as our next market regulator. The science is not yet there,” Bauguess said. “This does not mean that staff won’t continue to follow the groundbreaking efforts that are moving us closer to AI … [but] I expect that human expertise and evaluations always will be required to make use of the information.” 

Think of AI in the surveillance space as a motion sensor alerting compliance professionals to where they should focus their flashlights, leaving nowhere for potential price manipulators to hide.