4 Core Aspects Of A Successful Data Integration Project
Data integration is a fairly easy thing to define, as it’s the process of drawing together and combining data from different sources to provide the user with a unified and accurate view of the information.
However, the process of integrating data between separate systems can be far more complicated than it first seems, especially when you take into account the differences between systems in terms of the way they store and present data, the way they are built, legacy software that doesn’t easily integrate with newer systems, and the sheer volume of data being created and drawn in from many disparate sources.
With this in mind, whenever you venture down the path of a data integration project, there are certain things you must do to increase your chances of success, a number of which are listed below.
Design a clear strategy for your data sharing model
Integrating two systems and setting up a straightforward sync between the two of them is relatively simple, because there are only two possible directions in which information can travel – data goes in, and data is exported out. However, as you start adding other systems into the mix, things can become infinitely more complex with each new application that is added.
That’s why having a clear strategy for your integration projects in place from the outset should help you to determine which systems are going to be integrated, and in turn that means you can be proactive in implementing and planning out the overall data sharing model.
However, if you instead choose to take a reactive approach to implementing new systems, you run the risk of having to rebuild and redesign your integrations from the ground up each time, which is clearly a very costly and time-consuming approach.
Maintain Parent-Child relationships and avoid orphaned records
Whenever you undertake any kind of data integration project, it’s vital that you maintain existing Parent-Child relationships when the information is synced.
If you don’t spend enough time thinking this through and start transferring data without a solid plan in place, you risk ending up with orphaned records – data that is in the target system but which isn’t tied to anything, be it a business owner or parent record.
As you can imagine, orphaned records such as these become a major problem when you attempt to clean up the data later down the line.
Determine if you need a System of Record
The whole point of creating integrations between different systems is to easily move information between them. When they are connected, data will be moving between them constantly.
This means that, before you get into the true detail of a data integration project, you need to have clear data governance processes and definitions in place for what will happen in certain situations – for instance, when data from each system tries to overwrite corresponding information from another.
This could be something as straightforward as a company account which is created in your customer relationship management (CRM) system or supplier information management (SIM) system, which will then be synced with your ERP system.
It’s probable that you will want the data from your SIM or CRM system to overwrite the data in your ERP, not least because the people entering the data into those systems are likely to have much more detailed knowledge of the vendor or customer they are working with than the finance people on the ERP side.
However, as is always the case it’s not quite that simple, because there might be certain information being passed from the ERP to the SIM, CRM or other systems that is only editable within the ERP itself. This could be something like an account ID or account number that is transferred across to the supplier information management system, but can only be edited within the ERP.
The reason for this is to stop someone from altering the data in a way that breaks the link between the ERP and SIM systems. As part of this, you will need to define fields that are specifically owned by one application or the other, otherwise called your System of Record.
Define your conflict resolution rules
While you can set up one-way integration by establishing your System of Record, when it comes to two-way integration and the potential problems this can cause in terms of similar pieces of data from each system trying to overwrite each other, you will need to put ‘conflict resolution’ rules in place.
This is done by defining rules that essentially say that if there is a conflict involving ‘X System’, then that system will always win and have the final say. This means that, even if you or someone else attempts to update information locally within a system, if that same data isn’t reflected in X System, the update won’t actually take place unless the conflict resolution rules are changed.
This is important because it stops data in different systems from being independently edited and therefore leading to errors and confusion down the line.
For instance, if John Smith is listed as working for Federal Express in your SIM system, then you don’t want the same contact to be listed as working for FdEx in your ERP because the record has been updated – you need the information to be carried through. This means either defining your SIM system as the System of Record, or setting up a conflict resolution rule that means the SIM system will have the final say.
Hopefully the points listed above have given you an insight into the questions you should be seeking to answer before pushing ahead with a data integration project.
Here at HICX we support many leading global organisations in improving and transforming their supplier management and procurement processes by utilising the power of great supplier data. You can see plenty of example case studies of projects we’ve worked on with market-leading organisations here.