Credit unions are increasingly investing in newer software and technologies. In fact, research shows that more than a third of credit unions in the range of $50 million to $100 million now have mobile apps.

Embracing new technology is helping credit unions to develop innovative new services that give them a competitive edge. Basing these services in the cloud is helping to lower their costs.  Both of these trends mean considering new software systems, which is a challenge to IT departments. One of the biggest issues is data integration — that is, mapping years of data from old software systems into fields in the new system.

Data migration issues were blocking one credit union's move to a new banking system and keeping it from modernizing its services, until managers discovered extract, transform, and load (ETL) data integration technology.

Years ago, the credit union internally developed a core banking system to store and maintain member account information. As its membership and product offerings grew, the homegrown solution soon grew to hundreds of tables storing hundreds of gigabytes of data.

Because of the legacy system's design, the credit union's ability to develop modern, member-centric software applications was severely hindered. Fitting new products and services into the system took excessive amounts of time and the team was already burdened with completing many regulatory changes to keep the core system compliant, which prevented them from focusing on developing productive software applications.

Making the Move with Data Integration

Ultimately, a decision was made to convert to a third-party core banking platform. However, given the sheer enormity of the data to be converted, credit union leadership soon realized that converting to the new platform would be a daunting data integration task that would require a dedicated team to get the job done.

The team was charged with converting legacy data into the new core banking system, developing conversion logic/business rules, identifying opportunities for data cleanup, and integrating newly selected ancillary software with the core system.

One key issue was moving from account-centric systems to member-centric format. Converting entity relationships from the old system to the new system was very challenging, as data in the legacy system would need to be converted to the new formats.

For example, information that resided in one table in an account-based system would exist in multiple tables in the new system. And in some situations, the reverse would also be true, where three records in the old system might be merged into a single record in the new system. Converting between the two would take complex normalizations of the data, increasing the risk of an excessively cumbersome migration project.

Implementing a Solution

The team knew it needed to find a data integration solution, but any time spent learning a new tool or paradigm was time not spent working on the actual business problem at hand.

Therefore, data integration performance and ease-of-use were top priorities for the credit union. Furthermore, an overly esoteric tool would prevent newcomers from effectively participating in the project down the road.

Finally, the credit union wanted a solution with a performance threshold high enough to realistically test in a timely manner, so that the data integration didn't take any more time than absolutely necessary.

The candidates for the solution boiled down to the classic "build vs. buy." Coding a system in-house was initially considered, but was ultimately not a viable option because the development effort would have been too extensive and the run-time performance would likely not scale to the amount of data to be migrated.

So, the credit union determined the best option would be a third-party ETL tool.  They specifically required a software platform that was user friendly and not too complex to operate so that their business analysts as well as developers could use it. The learning curve needed to be fairly simple so developers could use the tool for a period, move to another issue, and then come back without having to relearn it.

The data integration software of choice, in addition to its ease of use, offered SQL source and SQL lookup table components, enabling developers to leverage and translate existing SQL skills.

The name parser component was an unexpected bonus, saving additional development time. The solution allowed the credit union to modernize and automate its data systems to be more efficient, while freeing up its most valuable asset, human resources, for more important responsibilities.

Technology is enabling credit unions to offer more and better services — more cost effectively — than ever before. Making sure that they can integrate their existing data into these new systems however, can still be a challenge. With ETL data integration, though, this challenge can be overcome.

Jay Mishra is vice president of development at Astera Software.