Agile: More Mentality Than Methodology

Part one of this two-part feature focused on the premise of the Agile software development methodology and explained what such strategies might look like on a practical, day-to-day basis within buy-side and sell-side firms. Part two outlines the business processes most suited to Agile, the communication tools available to capital markets practitioners that make the methodology particularly suitable for delivering immediate value through frequent software releases, and the notion that Agile entails more than a pure software development methodology—it is, according to sources in this feature, a culture or indeed a mentality that needs to be adopted throughout the organization in order for its significant benefits to be fully realized.

“It’s more of a culture than a development methodology, and there’s a real danger that people get too attached to the practices and methodology and they lose sight of the underlying principles,” explains Barney Dalton, CTO at London-based Aspect Capital, a $5 billion systematic hedge fund. “If you go back to the original Agile Manifesto, it’s a set of principles, not a set of practices. That’s one thing you need to be really careful about when adopting Agile—you need to make sure you stay true to the principles and don’t just look at the practices that some people have chosen to adopt, because they might not be relevant to you.”

Focusing on the Minutiae 

Rizwan Saeed, head of technology at UBS Delta, the Swiss bank’s buy-side-focused portfolio analysis and risk management platform, agrees with Dalton’s sentiment, in as much that he warns against focusing too much on the minutiae relating to the technical aspects of the methodology at the expense of the people responsible for driving it, irrespective of whether they are technology producers or consumers. “I think there’s a danger that people pick up a book on Agile and say, ‘Scrum is Agile’ or ‘Kanban is Agile,’ and they become wedded to the idea of a particular methodology,” Saeed explains. “Agile is more about the people—you need to ensure that your processes adapt as the organization adapts and evolves. The key with Delta having been successful from an Agile perspective is that it’s had strong backing and engagement from the business side. Ian [Lumb, head of sales, UBS Delta] and Lindsey [Matthews, head of risk at UBS Delta] bought into the idea that the technology side isn’t just something that gets shunted off to a different part of the organization that speaks to them via email or a business analyst—they are actually willing to invest the time to engage with developers. That engagement is definitely a cultural and people aspect to the Agile process.”

According to Michael Radziemski, CIO of Lord Abbett & Co., a Jersey City, NJ-based equity and fixed-income investment manager with approximately $125 billion under management, Agile is “definitely” a mentality and a cultural stance that must be acknowledged and fully embraced—preferably throughout the entire organization—if it has any hope of fully delivering on its potential. “There is a whole cultural approach that has to go with the Agile methodology,” Radziemski explains. “Our guys have embraced that and that’s why it’s working well. By contrast, if you found some folks deeply wedded to a Waterfall approach, they would find Agile difficult,” he says. 

“There is a whole cultural approach that has to go with the Agile methodology. Our guys have embraced that and that’s why it’s working well. By contrast, if you found some folks deeply wedded to a Waterfall approach, they would find Agile difficult.” Michael Radziemski, Lord Abbett

Suitability

So, if Agile is the answer to delivering value to end-users through early and frequent releases, can it be applied to all business processes, and crucially, if it isn’t deemed appropriate, why exactly is that? Porter Eubank, director of enterprise architecture at Lord Abbett, explains that from his firm’s perspective, the back office tends to favor the Waterfall methodology, while the front and middle offices are more appropriate for Agile. “It varies from area to area,” he says. “In the back office, things tend to be more formal, more Waterfall, but in the front office there is greater demand for incremental deliverables, although there are times when things can’t be delivered that way, particularly if you are implementing a whole new architecture. Then, what usually happens is the first release will be a bit more Waterfall and enhancements will be more Agile. So you see a transition in the front office from the first release to the enhancement queue, which Agile is very good for managing—we even see our vendors using it to deal with client requests.”

Radziemski adds that from a purely “functional” standpoint, there are no business processes that he considers unsuitable for development work from an Agile perspective. “In fact, most of the success we’ve had with Agile has been with our front-office investment systems and the reason for that is that we’re iterating constantly with the investment folks—we work very closely with them,” he says, emphasizing one of the keys to any successful Agile initiative: close collaboration between technology producers and consumers.

Keith Shiner, vice president, financial services at Toronto-based consultancy Intelliware Development, goes a step further than both Radziemski and Eubank, arguing that in his experience when working alongside capital markets clients, the Waterfall methodology is the preferable model to opt for when it comes to what he describes as “well understood, not rapidly changing legacy products and systems,” such as those associated with the back office, echoing Eubank’s sentiments. “However, we feel that most other processes are suitable candidates,” Shiner adds. “At the heart of the matter lies the management of risk inherent in any IT deliverable and we’ve demonstrated that Agile is the best way to manage those risks—it is essentially process and project agnostic. However, if you have an engaged and empowered product owner, it dramatically increases the success of an Agile delivery engagement.”

Considerations 

UBS’ Saeed explains that from his experience there are certain instances when Agile is not a feasible option for software development, especially when regulatory considerations come into play. “The characteristics are anywhere where you’ve either got the regulators involved or you need external sign-off,” he says. “So, if you’re changing pricing and valuation models or risk models, or something that you’ve published, and then are changing where you need external sign-off, those are harder to adapt to Agile.”

Ian Lumb, head of sales at UBS Delta, expands on Saeed’s summary, explaining that in the event of unforeseen circumstances, certain high-priority development work needs to leapfrog other scheduled tasks, in which case a modified or hybrid-Agile methodology is preferable. 

“From the client’s perspective, sometimes if they, or we, spot a bug, or there are issues—regulatory or otherwise—that need resolving within a tight timeframe, this may sometimes be better suited to a hybrid-Agile process,” Lumb says. “This can run alongside the regular sprint planning and allow for high-priority development items to drop in at the top of the sprint queue. Sometimes you do need to override the longer-term plan with unexpected changes of priority—that’s why Agile is great at enabling a nimble business.”

Clearly, as Lumb intimates, flexibility is a key criterion for the success of any Agile strategy, allowing developers the leeway to pivot whenever and wherever necessary, unencumbered by a rigid, prescriptive framework that demands development work be carried out in a pre-determined, regimented sequence. 

“If you go back to the Agile Manifesto, it says that an Agile process is something that evolves—it’s not something that is prescriptive,” Saeed says. “It’s about the people first. If you take the core principles from the Manifesto, there’s probably not much that can’t be fitted into that paradigm, but obviously you have to tweak the implementation depending on the context you’re in.” 

Full Embrace

Aspect Capital is a good example of what is technically possible when an organization fully embraces Agile for all its development work, an initiative that, according to Dalton, the firm embarked on in early 2013. “We had quite a specific change process to move to Agile,” he says. “We didn’t know what it was going to look like—we knew some of the things we wanted to achieve and we got various external people to help us through that transformation. It’s taken a bit of experimentation to get to where we are today, and even today it continues to evolve. The key is that the team feels it’s their process and if they think something is not working properly, they will propose changing it or they will go ahead and make the change.”

Dalton says Aspect’s Agile strategy has two distinct layers: adapting to what the business needs through running two-week sprints, providing agility in terms of what the teams are currently working on, breaking down that work into small stories, and delivering value with each sprint so that they can change what they are working on without leaving half-finished work; and the process itself, which, by its very nature, is agile and which can change. “We run retrospectives every two weeks and discuss what worked and what didn’t from a technical perspective, but also from a process perspective. We then make changes based on those retrospectives. It means that the team genuinely does own the process,” Dalton explains. 

Where does Dalton stand on the appropriateness and feasibility of Agile underpinning every business process within the fund? In other words, is there anything that Aspect does on a day-to-day basis that he would consider “off-limits” from an Agile perspective? Apparently not, although naturally the firm’s trading strategies—its secret sauce responsible for differentiating it from other similar funds—are the only exceptions. 

“In terms of delivery to the business, I think most things are quite well suited,” Dalton says. “We had some questions raised by the business when we first moved to Agile—they had much more of a long-term, milestone-based perspective, and they found the idea that they might get small bits of work as they went along quite difficult to understand. But for us, Agile has worked well across all areas, although the one area that we don’t want to change every two weeks is our trading strategies themselves. They operate on a longer time horizon and frequent changes would just create unnecessary trading churn.”

Communication 

As previously mentioned, communication of one of the keys to any successful Agile initiative, a factor acknowledged by UBS Delta, not only through the location of its development teams, but also by way of the technologies it has adopted to foster transparency, efficiency and idea sharing. “A lot of our team is co-located in London, although we do have people out in the Asia-Pacific (APAC) region as well,” Saeed explains. “Obviously, communication is key to the whole process and this is where the tooling plays a big part. We make use of all the standard development tools—Jira and Confluence and so forth—and we’ve shifted away from email and now use tools like Jive, which UBS has implemented at an organizational level. UBS has also implemented Lync Group Chat and my development team in London has a daily call with their counterparts in APAC. Again, it’s all about information flow and transparency, and a lot of that is down to how people buy into the process.” 

Lindsey Matthews, head of risk at UBS Delta, underscores the value of face-to-face communication in Delta’s Agile framework, explaining the importance of close proximity for key staff members: “Just being able to get up and walk 15 feet across the room is extremely important and so having a lot of the development team here in London is very useful,” he says. 

Targeting Transparency 

Like UBS, Aspect Capital places a premium on clear, efficient and transparent communication, an emphasis that has led the fund to adopt a range of Agile-associated tools and technologies. “At the code level we use various unit-testing frameworks—all open source—and we mostly develop in Java,” explains Dalton. “At a higher level we use Jira to manage the sprint processes, we use TeamCity to manage the integration of all of our builds, and behind that we use Atlassian Stash for storing all of our source code.”    

UBS’ Lumb explains how his firm’s Agile strategy, along with the various frameworks and communication technologies underpinning it, has allowed it to improve not only the efficiency of the entire development lifecycle but also the transparency of the processes that play such a vital role in its healthy functioning. 

“Before, when we were on a more Waterfall project-based program, we often didn’t have a great degree of visibility into what was happening, how it was happening, what stage developments were at, and whether what was being delivered was actually what was required in the first place,” Lumb says. “Under Agile, we’ve got much better visibility of future developments, we know what’s coming, when it’s coming, and we know with a high degree of certainty what is scheduled for the next three months or so. Then, for the following three to 12 months or longer, we have a longer-term roadmap and vision. From a business perspective, we’re far more involved in the whole process, from discussing and specifying the original requirements through to reviewing progress and testing pre-release.”

Secrets of Success

A lot has been written about the use of Agile in the capital markets in recent times—not least by this publication—which has naturally seen a number of prerequisites emerge determining the success of such initiatives. Saeed at UBS measures success by the value he and his colleagues are able to deliver to the firm’s end-users, a process that is continually in a state of flux and therefore has no end point. “Agile is all about the concept of delivering value and not necessarily about delivering a specific feature,” Saeed says. “I think the facets that have made us successful are our short and continuous releases providing a steady stream of deliveries, and our ability to adapt to and change direction as the market changes or as our clients’ needs change.” 

Lord Abbett’s Radziemski, on the other hand, cites three criteria: “You have to have business people willing to engage with you in a close and constant manner; we tend to do our projects with small groups of well-rounded people, typically five programmer-analysts who can do everything across the project lifecycle as opposed to large teams of specialists; and finally, knowing which projects to apply Agile to.” 

And finally, what has Dalton learned during his three-year Agile odyssey that he would commend to other capital markets CIOs considering similar initiatives? “Agile must come from the team—you can’t impose it,” Dalton advises. “And having external input was really valuable for us. We had three or four coaches (consultants), all of whom had slightly different skillsets. Having them work with the team over an extended period was really effective. They didn’t need to be in the office every day, but just having that guidance—that coaching—was crucial.” 

Salient Points

• Agile is more than just an alternative to the Waterfall development methodology—it is a culture and a mentality that capital markets firms need to fully embrace in order to realize its significant potential.
• Some practitioners argue that there are certain instances, especially those related to the back office, where Agile is not suitable. However most believe that just about every business process stands to benefit from the Agile treatment. 
• A number of business communication and process management tools are particularly well-suited to an Agile environment, streamlining information flow and making the entire development process more auditable and transparent.