Finserv Has More Kinds of Data Than You Think, and It’s a Compliance Nightmare
It should surprise no one that today’s corporations generate a lot of data. And they will continue to do so at an increasing rate: From 2020 to 2022, the amount of data the average enterprise stores more than doubled, from one petabyte to roughly 2.022 petabytes. That’s over 100% growth in just two years.
Financial services (“Finserv”) firms create more than their fair share of that data. Even a modest-sized regional bank will likely traffic in as much data as a company ten times its size. But what few experts have come to grips with is the sheer variety of data that finserv companies must manage.
All that variety creates a huge hurdle for data management and compliance simply because most solutions on the market specialize in certain types of data only. This fact has forced most finserv companies to cobble together several disparate solutions…or to forego any sort of data management whatsoever.
And that is creating an extremely large but hidden source of risk for finserv firms.
The Varieties of Data in the Average Financial Institution
Consider for a moment all the sources of data that, say, a regional bank traffics in every day:
- Transactions at all physical locations
- Transactions carried out online and via a mobile app
- Client personal data (name, address, birthday, social security number, etc.)
- Account information (account numbers, transactions, balances)
- Spending categorization
- Credit information
- Mortgage and loan information
- Contract information
- Emails (to the tune of 128 messages sent and received each day, on average, per employee)
- Employee personal information and pay information
- Employee logs
- Analytics data (based on customer spending patterns, segments, new products, customer feedback, etc.)
- Marketing data (email open rates, website visits, direct mail sent, cost of obtaining a new customer, etc.)
- Customer service data (tickets, rep notes, inquiries, dates)
- Network usage and access statistics
- General data on markets, commodities, and prices
A similar exercise works for other finserv companies (insurance companies, wealth management firms, etc.).
Looking at this list, it’s clear that all this data is gathered, stored, and used by different departments within the organization. In part because of that, data is probably also spread across several systems—for example, an OLTP database for online transactions, an OLAP database so that marketing can do interesting analytics work, an email server maintained by IT, etc.
It’s also clear that this data differs a lot in and of itself. For example, emails are a popular example of unstructured data: Individual emails can vary widely in terms of length and kinds of information, and there is no real formatting that lends itself to classical database storage. On the other hand, transaction data are a good example of structured data: The information is organized into specified fields of known structure and length.
The Problems that Come from Scattered Data
Who cares that there are so many kinds of data being tossed around? Compliance officers should, for one thing. Having different kinds of data in different places can be a complete nightmare when it comes to things like data privacy and compliance. For example:
- What happens when transaction data is appropriately encrypted in a transaction database but fails to be encrypted when that data is aggregated for analytics purposes?
- How can appropriate access be maintained? For example, how can institutions ensure that clients have access not only to their account information but also to things like customer service correspondence?
- Which bits of data are covered by the company’s privacy policy, and which aren’t? Which are included in state and federal privacy laws, and which are not? How would someone even know?
- What mechanisms are in place to ensure that all kinds of data in every location is destroyed once it reaches the end of its data lifecycle?
Again, keeping track of all that data is, from a compliance standpoint, exponentially more difficult with each kind of data and each new “data home.”
Moving Forward: A Singular Data Archiving Solutions?
To be clear, the issue is not a lack of solutions. The idea of data archiving—that is, moving data from its more readily-usable formats to a kind of “deep storage” for long-term preservation—has been around probably since the library of Alexandria (roughly 222 B.C.). Today, there are literally dozens of data archiving and data storage alternatives on the market.
The real issue is that most of these tend to be one-trick ponies. For example, there are some great examples of email data archiving platforms, and they do really well with unstructured data. There are also document management systems, backup, archival software, monitoring systems, and more. But each one has its own specialty; few can act as a central repository for everything while still managing access, logging, and data destruction as needed for compliance.
Indeed, this patchwork landscape of solutions is precisely what drove our engineers to create the Omni Archive Manager. We saw that there was a need for a single tool that could archive all data, maintain appropriate records management across the data lifecycle, and monitor and control access. The need for such a tool happened to be greatest for financial institutions—precisely because of the amount and variety of data they were generating every day.
It might be the case that some institutions can get along with a “cobbled together approach.” But, with increasing regulatory legislation around data privacy, and increasing sophistication of cyber attacks, those days are numbered. No longer can finserv rest easy, assuming that siloed information will be their saving grace. Soon, even smaller companies will need to archive and monitor their data as if they were huge international companies. Then the question becomes: How quickly can they do it?