Exploring the challenges and opportunities of an ever-expanding data universe

Chanice Henry: For today’s financial services leaders, trustworthy data is the cornerstone of decision making. But data integrity and accessibility can be difficult to pin down in a volatile macroeconomic climate.  

So, how can financial services leaders ensure their digital transformation enables innovative decision making for better business outcomes? 

I’m Chanice Henry, senior editor at FT Longitude. And joining me to discuss this is Birgitte Bryne - or Bridget, as everyone calls her - who is the Chief Technology and Operating Officer at Norges Bank Investment Management.  

Bridget, thanks for joining me. 

Birgitte Bryne: Thanks for having me.

Chanice Henry: You've been at the forefront of operations in your organisation for a number of years now. 

Of course companies today are operating against a radically different backdrop than we’ve seen for many years—one with far more volatility and, as a result, instability, and it’s critical that they make the most of opportunities as well as seeing risks as clearly as possible. 

So it would be good to hear from you Bridget on how the contextual climate is reshaping how you are approaching digital transformation at Norges Bank. 

Birgitte Bryne: In general, the global macroeconomic climate has been a challenge for all investors the last few years. For us, to be faster to market and also possibility to do forward simulation and stress tests is crucial. We have to make sure that our portfolio managers and traders have the tools to meet these requirements, and in a world where these things can change very fast. So we are building systems internally for managed control and also for portfolio management and trading. So we want this to be tailor-made and we want to have full control over that kind of development.

So in general, we underwent a significant digital transformation when we migrated all of our technical infrastructure and all our systems to the cloud. I believe that it has been, and still is, our most critical transformation to meet the future requirements going forward and also to facilitate scalability and security above all in a cost-efficient manner is our best approach, I would say, to tackling future challenges.

We have decided a few years ago to bring everything in-house. I see in many investment firms, they are outsourcing things, but we went the opposite way and that has given us full control and the ability to change rapidly, if that's necessary. We also strive to contribute to others by being as transparent as we can as a fund. So we share data externally through modern mechanisms and that includes sharing our proxy voting, our holding lists, and more. And we hope that that can be beneficial for other investors.

I think last, but not least, I would like to mention cyber security. It has become more and more critical. I believe that more and more advanced threats and threat actors and information that is needed to be absorbed into our infrastructure in a meaningful way is something that we spend a lot of resources on. Analysing the constant shift in the security environment is something that we're spending more and more resources on and in a macroeconomic and globalisation of the world today, more and more important.

Chanice Henry: Absolutely. And also, feeding that into your staff training, right? So making sure that of course you've got your technical folk that are aware and scanning the landscape, but also, everyone in the business, despite their level of acumen and skills with cyber awareness, it becomes as heightened as it can be through the training that you give. The risk landscape with cyber is always evolving, there are always more threats that are becoming more sophisticated, so it's about making sure that that awareness filters all the way through the business.

And I'd like to keep digging into that area of risk that we're mentioning here. Not only on the cyber side, but also in terms of compliance or wider operations, and specifically around data. So how is data playing an important role to identify, manage and mitigate risk at Norges Bank?

Birgitte Bryne: Our business is 100% driven by data, whether it's research or generating investment strategies, ideas, trading, the entire confirmation and settlement process and the reporting process is dependent upon data.

It's crucial that data has the right quality from the start and we handle it with as little influence as possible, only adding information along the way with high quality, so as clean as possible. Central to this is how we source data, where our focus continues to be on obtaining this as directly from the source as possible.

And if we talk about latency, which is often discussed, it's a crucial aspect. It's easy to put into perspective as, for instance, we have a long-term horizon to our investments. It's a generation fund. It's a fund for generations, so in that sense, latency is not important. But if you then put it into perspective of trading operations, latency is very crucial. Particularly when we come to algorithms and the algorithm tradings, it's extremely important.

And, of course data is important in the AI revolution that we are looking at currently. It has become much easier and much more efficient to both development and to operate in the recent years when it comes to data. What used to take months can now only take days. The fact that it's easier means that we're less dependent on relatively few key individuals as opposed to what we used to be, and they're also allowing us then to do more things in parallel. 

Chanice Henry: And like you were saying there, without data at the heart of all of those things, it's going to really damage the quality of the outputs. With that in mind I think it would be interesting to look at front, middle, back offices: so you've got the more revenue-generating front of house roles, and then those thread through to be supported by the operational middle and back roles. 

So how would you say that data and tech is enabling better decision-making in the front, middle and back offices roles at Norges Bank?

Birgitte Bryne: I would say that simplicity is the key word here. To make data available to your organisation and all downstream systems and access to clean data, and the same data sets in all processes to avoid duplication and deviations between various data sets. And this is not easy.

At the current stage, we are doing a big cleanup to get there: 25 years of history. It's a huge undertaking, but it needs to be done to be efficient going forward and to cater for future AI projects. I always like to highlight our cloud strategy and our focus on one global custodian bank, one system architecture with as few interfaces as possible, and core systems that support the entire management. I would say that now it's easier than ever to get systems to talk to each other. Our approach to this is to build a platform for integration with core systems, which will enable us to build simple front-end solutions targeted towards end users going forward. So for us, front, middle and back office systems is integrated on one platform or maybe two platforms. 

Chanice Henry: It would be good to hear more about any particular lessons you’ve learned from your experience on the data analytics modelling techniques or technologies that you’ve used in your time Bridget. I know you were mentioning about the power of the cloud that financial services companies should look to lean towards or maybe avoid when they're looking to identify opportunities and risks, especially around this volatility we have. So any dos and don'ts that you have from your experience around data modelling techniques or technologies?

Birgitte Bryne: New technology has made the development and operations of systems and data much more efficient. It has enabled some democratisation of both applications and developments internally. So, a smaller proportion of the total is now centrally controlled, much more is sent to the business lines. It increases the capacity and it proves the fit of what we're building to support revenue generating tasks. It reduces also the bottlenecks.

The lowest hanging fruit is probably related to operational gains from automation by traditional programming techniques. At the other end of the scale, high profile access return creation is often linked to an element of creativity and it's difficult to automate entirely. But I think one key insight is that one of the greatest potential is to avoid dealing with those extremes in separation, and instead focus on the interface between the two and work holistically on the totality of the organisation's activities. So collaboration, absolutely, investment-focused operations and operationally efficient investment decisions.

One example would be with corporate actions with generally high volumes of low quality information. Operational differences according to investment objectives is very important. wWhile lots of portfolio management time can be saved by assistance of language models found in generative AI, for instance. Optimization is another example with the potential to improve quality and save time, but only if done right and supported by high quality forward-looking data. So, in general, it's a tall order for off-shelf front-end systems to combine the generality and efficiency necessary to meet complex investment organisations.

Chanice Henry: And on that note of complexity, I wanted to pick up on a point you made slightly earlier of having a standardised language or just a general rule for standardisation within your data and also your platforms. A lot of companies are looking to do this, to get started or dial that up but it’s not always easy to know where to begin. So do you have any advice on that, where to get started with this standardisation mission that a lot of companies will be on? 

Birgitte Bryne: I think that trying to start with the smaller things, the easier things, the standardised things has at least been beneficial to us, and then build on top of that as you go along. We have started more from the practical angle, and then built the solutions and the governance around the systems as we went along. 

I also think that you have to spend a lot of time on tidying up in the infrastructure and the data sources, otherwise, it's just going to be just another platform or just another system where you have to maintain deviations between the new things that you're building and what you already have in other places of the firm. You can, of course, build interfaces in between them and that is something that we have done as well, but we try to make as little interfaces as possible. That has been beneficial for us and we're still continuing on that journey. Simple architecture; one platform for everything: that is important within the portfolio management; has been beneficial for us.

But I realise it's a big journey for most companies and has been also for us and it still is, but I think it needs to be something that the organisation decides to do, decides to spend money on, decides to spend resources on. Again, we have a big advantage for being the kind of fund as we are, a fund for generations, also has made it possible for us to prioritise these things. We don't have to really worry about the reporting of next quarter when it comes to this. We have the means that we need to do these huge investments in these kind of platforms because that's what it is, huge investments.

Chanice Henry: And finally Bridget, it would be great if you could share your golden rule on how financial services companies should be looking to make sure that their digital transformation efforts are fit for purpose in today's current volatility.

Birgitte Bryne: Collaboration between the various areas is probably the best starting point and, as I mentioned previously, to agree on the strategy and what to prioritise is probably the best advice I can give.

I think it's important to get people on board, to hire the right people that already have some kind of expertise that have been through some kind of transformation already, particularly to the cloud, particularly to transformation over to modern data warehouses, and the possibilities that these kind of setups can make. I think it's important to go through that, first of all, and then take away the things that are not the most important things. Start with core. Try to get the core and to get the most important key processes in order before you move forward. That would be my advice at least and that has been our learning point that if we did it all again, we would have prioritised a bit differently probably, and we would have made sure that we had in-house competence before we went along.

That said, we were eager and started without going through all of these things before we started, and I think that in some sense that also made us move much faster because we didn't do all the analysing work beforehand. We didn't go through all aspects and all things that we can fail on as we went along. We kind of jumped into it and we decided that, "This is the deadline. We're going to make it within this deadline." And then, after we made the jump, we tied it up in all of things afterwards.

It's probably not ideal, but it has made us move forward very quickly. I think that when we talk to our vendors, they say that a lot of the questions and a lot of things that we want them to make, or make available for us, is something that pushes them forward because we have come quite far when it comes to using the possibilities that these modern tools and modern platforms can make available. So in that sense, it has been a good idea, but it has been stressful for the organisation also.

Chanice Henry: I can imagine. I think there's something to be said for the fact that in businesses there can be a lot of agonising, a lot of overthinking. So sometimes learning as you go and getting started, there’s a lot to be said for that, and when it’s done well it can be a great thing with a lot of rewards. And it definitely sounds like that's the case with Norges Bank and your experience there. 

Well, it was incredibly fascinating to speak with you today Bridget. Thanks for joining us and we’re wishing you and Norges Bank all the best with your digital transformation and other endeavours.

Birgitte Bryne: Thank you so much for having us on board.

Chanice Henry: Artificial intelligence, machine learning and large language models bring the promise of a competitive edge for financial services firms in today's volatile macroeconomic climate. But in the race to unlock this opportunity, organisations must work to overcome system limitations and avoid a growing wealth of misinformation. 

So how can financial services organisations derive new value from these emerging technologies without compromising on quality, integrity, and trust? 

I'm Chanice Henry, senior editor at FT Longitude, and joining me to discuss this is Andrew Chin, Head of Investment Solutions and Sciences at AllianceBernstein. Thanks Andrew for joining me today.

Andrew Chin: Hi Chanice, it’s great to be here. Thank you for inviting me.

Chanice Henry: Andrew, you've got a wealth of experience across data science and risk management in financial services. Could you start by painting a picture of how, in your opinion, the sector has changed in recent years and in particular the impact that emerging technologies, so AI or artificial intelligence, machine learning and large language models, are having?

Andrew Chin: So, I would say I started my journey about seven years ago when I first created our data science efforts here within AllianceBernstein. What's really changed over the last four or five years is that we have a lot more data. So we have a lot more data that we're dealing with. There's the higher value, more variety, it's coming at us much faster on my iPhone. So what that means is that there's a lot more information that I need to consider. And then on top of that, the tools by which we can use to synthesise the data, to understand the data, to analyse the data, has really exploded. Large language models, AI, machine learning, as you've said. And so the opportunity to analyse this data much more quickly in a much more efficient way, that's also changed. So I would say as I think about the industry, it's really those two aspects which has really impacted our organisation. 

Chanice Henry: Yeah, absolutely. There's more variety in data than we've ever seen before, the volume is growing exponentially, and that can bring in difficulties in that, what do we trust? What information points are reliable enough, fit for purpose for a high stakes decision? Or what should just be taken as contextual and needs to be verified? So this idea of misinformation is a conversation I'm seeing coming up more and more. Fake news, what can you trust? And of course with these, a sophisticated system comes in such as AI or LLMs or ML. It's great that they can crunch massive data, but you need to make sure the data being fed in is trustworthy and reliable. Have you in your experimentations figured out any dos and don'ts with safeguarding the trustworthiness of the outputs of these technologies based on the input of the data that's going in?

Andrew Chin: Yes, you're certainly right. The data quality, that is the most important thing. People say ‘garbage in, garbage out’. But it's a lot more important today, because when I think about the small amount of data we had before, you can easily, visually see the outliers in the data, right? So that's how you know, "Oh, actually that's probably bad data and now I can think about whether I want to use that dataset or not." With the volume of data we have today, it's impossible for me to visually see or visually understand whether the data is an outlier or misinformation. So we have to think of other ways to ensure that the data quality is high. So there are a variety of different things that we can do. One is obviously work with a vendor, work with a source that you can trust.

Second thing is, even as you have this data, you will need to cleanse it, you will need to massage it in a way that hopefully works for you. So especially when using large language models, there are lots of things that you need to do to mitigate what are called hallucinations in the data, the tendency for these models to potentially answer, using irrelevant or actually just incorrect data. 

What we do on our side is we first, we finetune these models. Usually these large language models are not fit for purpose for what we want to do in our specific domain. We're in finance. I may have a specific need in equities or in fixed income. So finetuning the models with examples to work in your specific sector is really important. 

Secondly, is prompting. Prompt engineering has become a whole new field. But it's important that you ask the question correctly, just like the way you're asking me questions to have me explain something better. It's the same thing. You have to ask good questions to prompt a good response.

I would say the third thing that we really try to do is we really still try to hire subject matter experts. Just because you have more powerful tools doesn't mean anybody, a dummy like me, can potentially use these models. If you're using these models specifically for mortgage-backed securities, as an example, you still need a subject matter expert in that particular domain so that they can ask the appropriate questions and when they get an answer, ensure it's the right quality and it's in the domain that you're looking for. So you can't just have anybody operate these tools. So I would say that domain experts, subject matter experts, will remain important in this industry.

In our organisation, we ensure that it's the humans, it's the employees that have final accountability. And while these tools are helpful, you know, we want to make sure that the ownership still lies with the individuals.

Chanice Henry: There's a lot of excitement about how we can be using these technologies and people are going from sandbox experimentations and now trying to roll things out so they can see some returns and some wins. 

And of course the question is, well, what is the low hanging fruit of these powerful technologies? I think to dig into this, it would be good if we broke this down into a few steps, maybe into front, middle and back office within financial services. 

So if we start with front office in financial services, what are the quick wins that AI, machine learning and large language models can bring to the performance and decision of the front of office employees in financial services? Again, any dos and don’t to make sure you’re not doing more harm than good. 

Andrew Chin: I would say broadly speaking, the lowest hanging fruit is around natural language processing. And the reason that's the case is because in our industry we work with so many documents, whether in the front office, back office, it's always documents, always text. So these natural language tools are really perfect for task. So I would say broadly speaking, that's where the opportunities are. 

Within natural language processing or AI, I would say that these tools are typically used for a variety of different tasks. One is interpretation, trying to understand something. Secondly, it's around summarisation. Can I summarise this article very quickly? Third is chatbots. Fourth is content creation. Can I create content based on what I know? And then I would say the last one is prediction. So I would say those are the broad categories by which these large language models are typically being used.

But the main benefit that I see today on the investing side is really around making the analysts more efficient. Instead of listening to earnings calls from 50 companies today, I can synthesise it very quickly. Instead of trying to see what all the corporate filing from my energy sector said, I can just synthesise it and say, "Well, on a macro level these are what all the energy companies are saying." So a lot more efficient from that perspective. 

And then maybe an example from the operational side. So on the operational side, we're reading lots of documents usually for compliance reasons. "Can I invest in this security?", "Are there restrictions in this security that it makes it harder for me to invest?" Now you can sift through the 200-page document very quickly and operationally it makes the process a lot easier. So we've certainly seen gains in both of those areas. On the operational side, we've easily seen productivity gains of approximately 50% or so, really helping our operational workflows tremendously.

Chanice Henry: I'd like to dig in a little bit more into the operational side because I feel that the front of office side of things tends to get a lot of limelight. But I think there's still a lot of gains to be made on the operational side, so I'd like to pause there a little more. I'd be interested to know more about the risk management side of things. Also looking at liquidity management as well, where AI can be utilised there to fast-track decision-making, reduce gaps in understanding. Any observations that you've had in those areas that you'd like to share?

Andrew Chin: Certainly. We look to news organisations, like the FT, to cover stories all around the world. The difficulty for us is, how do you read all these news sources all around the world in an efficient way? So from a risk monitoring perspective, one of the things that we try to do is we try to highlight whether there are any potential risks surrounding any of our companies which are coming up all around the world. So we are using large language models to sift through all the news articles which impact the holdings that we have, which impact the companies that we hold. And one example specifically is that we look for ESG-related issues. Slave labour for example, is a big issue for us, and we want to ensure that we understand potential instances of slave labour around the world and how they may impact companies that we hold.

So these filters act as really quick ways to get to the things that you're looking for, get to the themes that you're looking for. That's just around ESG, but you can imagine this being relevant for lots of different things. For risk purposes, for compliance purposes, for operational issues, for new opportunities. You may want to invest in a certain market, you might want to invest in a new technology, so you can imagine these tools being very helpful for that.

Chanice Henry: There's a lot of discussion going on at the moment around how to safely but productively scale up the adoption of these technologies, how they're being rolled out and used, what guardrails you are embedding, but also not to a point where you stifle too much productivity. Any insights that you've seen on your side around what works in those areas and what you see could have good potential moving forward?

Andrew Chin: I would say that it starts with having accountability, ownership for the decisions. So you have to say at the very top, "I am responsible for these decisions.", k Knowing that you're responsible now, what does that imply? For me, what that usually implies is, "Okay, I'm going to customise these models and make them fit for purpose for the task that I have at hand." So what I suspect going forward is even though we have some core models, ChatGPT, Gemini and things like that, I think each of us, we're going to customise these models and make them fit for purpose for very particular things. For example, I may say, "I have a large language model just to summarise earnings transcripts.", "I have a large language model just to find ESG risks in news articles." So what that means is those models can now be smaller and easier for me to use -. Tthat means they run faster, less expensive, things like that. - bBut also more importantly, I can now finetune these models for these specific use cases and make sure that they work in that area.

I think it's very difficult for any one of us to really create a mega model like a GPT-based model, but we can certainly use some of these tools to make them fit for purpose for these smaller problems that we have. That's the commercial side where we use some of these large models.

And then finally, as I think about organisations and how they may prepare for the future, one of the things that they need to do is really prepare the employees and train the employees. And I'm not just talking about the data scientists. Of course you have those. But employees who are making investments, employees who are doing the operational tasks, you want to make sure that they know how to use these tools effectively in their domain.

Chanice Henry: Yeah, absolutely. As we close, we've chatted about a lot of topics today, but if you could boil it down to one golden rule that you'd share with other business leaders around embracing these sophisticated technologies, what would that be?

Andrew Chin: I guess what I would say is, there is probably more hype right now than warranted, but the long-term potential is there. As we look at financial services over the coming years, we probably will be disappointed by what's going to happen in the next couple of years, but I think we'll be awed by what we will see in the next 10 years. And I think the rationale for that is because, as I said, we're so excited about what GPT has, can potentially do, and so we're all like, "Okay, okay, let's see what's next." But we're going to be disappointed because these tools need that calibration, need the finetuning, need the attention and care, need the subject matter experts. So some of us will lose attention and that's disappointing. But come 10 years, people like our firm, you hopefully will make a lot more progress and people will be like, "Wow, you've made that much progress with a 10-year period."

So I would say don't be too disappointed by what you see in the next few years, because there's a lot of promise and potential in what we're trying to do here.

Chanice Henry: Yeah, absolutely right. And that's a natural part of an adoption curve with a new concept or a new tech. You have the hype that skyrockets and everybody gets a real fever pitch, and then you start to see things taper down and some people fall away. But then it comes back up and there's then a plateau of what's the real value here, which is shifted through the noise. What do we actually have? So you're quite right, persevere through that point that is inevitably going to come when the dust starts to settle, to hold firm and keep looking for that value because it is there, I think you’re spot on with that.

Andrew Chin: You said it really well, Chanice, Thank you for that.

Chanice Henry: Well, thanks so much for your time today, Andrew. It's been great for us to chat. It's been really insightful for us to go through these topics today.

Andrew Chin: Thank you for the opportunity.

This interview is part of a research study, produced by FT Longitude, in partnership with London Stock Exchange Group.