How to migrate an ERP and ERP Data Migration resources for 2021

This article is a source of work for anybody interested in ERP Migration and especially a focus on Data Migration. I prepared for a Job interview. It is never a bad idea to refresh your mind and pick new sources about an interesting subject.

ERP Migration

According to Why ERP Migration fails published on Infopulse,  45% of companies that have proceeded to an ERP migration reported a budget overrun during ERP migration, and 58% had to deal with timeline overruns. On average, these were 11% over the original timeline. More grave ERP implementation failures usually originate from another source — subpar architectural decisions, data inconsistencies, and lack of visibility.

Such technical blunders (if missed during the QA stage) can cause significant failures after go-live and lead to severe financial and reputational damages.

In 2019, Revlon faced a class action lawsuit as they failed to fulfill product shipments of an estimated $64 million in net sales due to a technical problem with their SAP ERP implementation.

Oleksii Masharov

Establish a core team of business champions

A decade ago, many ERP software projects failed due to inadequate involvement of business staff. The right approach is to involve business at every step as ‘leaders’ — not just as ‘users.

According to this article,

Build a core team by inducting two to three members from each business function. Assign them with a fulltime ERP implementation responsibility. These should be functional experts from departments such as production, procurement, planning, finance, accounting, R&D, and sales. These business champions shouldn’t be given any function-specific work, but be used as change agents to drive the software migration project. The core team should define key business requirements, test the new ERP solution as per these requirements, and finally train the users after project completion.

The recruitment of your ERP Migration team is the key factor for the future success of your migration.

Your ERP implementation team should consist of :

• a business-level layer of people: you need to understand the business processes, the business rules, and the data entity ( term of TOGAF)
• a source system layer: you need to compose a set of experts from the source systems with application programmers, application analysts that understand the business rules or the data structure. Since the source system can be very old and the knowledge lost, we need to keep a stable team with the remaining experts of our system
• a target system layer: the target system is better designed however the knowledge is localized with the vendor. We need to educate our colleagues on the new system and understand the concepts, principles behind the new solution. We will integrate to our team, vendor specialists with a good knowledge of the principles and the data model of the new ERP.
• migration specialists layer: such specialists bring their experience in the design and the implementation of tools to automate as much as possible the migration tasks especially the data migration.

The team will split their responsibilities into three main activities :

1. Data owners, who input or understand the data design and structures in the legacy system as well as business requirements.
2. Functional team (usually from ERP system provider), who understands the ERP system data design and targeted business requirements.
3. Data migration team (usually from ERP system provider), who understands data profiling, cleansing, transformation, and migration techniques. The capability to document the migration processes at a functional and technical level is a requirement for this team

These employees will be our future ERP experts. Any employees that will likely use the ERP platform should undergo extensive instruction on how to use the technology. This can prevent human error from interfering with important data. Appoint a few individuals in the organization to be designated ERP experts, points of contact for employees to use if they have any questions or issues with the new ERP.

• Rule 1: Select candidates based on their skills, not on their position. The team needs to be motivated and interested in the company’s goals and understand migration purposes.
• Rule 2: The team should be able to make decisions and express their own vision on the migration process.
• Rule 3: Manage communication between the team members and ensure it runs smoothly.  Conduct daily meetings, especially after the completion of important stages, before moving to the next ones. Discuss all possible obstacles and difficulties. 

One of the biggest challenges with an ERP migration is the Data Cleansing and migration to the new system. In the next section, we will address some common recommendations to execute an ERP Data migration.

ERP Data migration strategy

When it comes to ERP implementation, data migration is one of the most important, and time-consuming aspects of the implementation process.

Case in point: In 2016, the UK division of Vodafone was fined £4.6 million by a regulator and carried additional financial losses after an ERP data migration failure. The telecom provider was attempting to move over 28.5 million customer accounts containing over one billion individual data fields from a legacy billing system to a new ERP platform. Due to a series of errors, some of the records were not appropriately migrated, and the issue was discovered only a year later when multiple prepaid subscribers did not receive service after conducting a payment.
Oleksii Masharov, Infopulse

Data migration is often the foundation or basis to gauge the success or failure of an ERP implementation. Since ERP manages and works primarily with data, efficient data migration is fundamental to an ERP implementation project. Unless the beginning balance of accounts is correct, you cannot expect the system to churn out accurate information.

 

Synergetix defines two different approaches for the Data migration of an ERP, the Transaction driver ( or incremental approach ) and the Table drive ( the one we see the most frequently). Interestingly enough, The former approach allows only active data extracted from a legacy system and converted into a file which then, will be loaded into an ERP system. The latter method is table-driven as it allows migration of the whole tables from the legacy system, with no selection criteria, to the new ERP system.

 

Challenges in Data Migration

Although important, migration is more than just copying data from source to destination; you have to validate the data being provided and make sure it is clean. If you have been creating your own ways of inputting information into legacy systems, or if processes haven’t been standardized across functional groups, your data is going to be dirty. Here are some key challenges with respect to data migration:

1. Ensuring credibility of data
   1. Spending insufficient time studying and understanding data
   2. Running the risk of inaccurate conclusions basing analysis of just a small sample of data
   3. Process complexity leading to large volumes of inaccurate, complex data
   4. Insufficient data cleansing, making it irrelevant to the context of its requirement
   5. Working with inaccurate, incomplete, and even false data
   6. Unnecessary time and cost spent on remedial actions to cleanse data, and directing additional resources to carry out this work to stay on track towards pre-determined project timelines
   7. Privacy: some data could leak outside the project. A good privacy policy is mandatory.
2. Lack of knowledge of working with data
   1. Limited knowledge of how the source systems work
   2. Little knowledge of how the target system works
   3. Inaccurate or insufficient input from the business teams
   4. Not giving enough time and attention to data quality and data discovery
3. System and role-related challenges
   1. Insufficient access to the target system
4. Project management lack of maturity
   1. Start too late the data migration
   2. Ignoring or under evaluating the data cleaning effort
   3. Time wasted on devising new strategies for workarounds to be implemented and deploying additional people to carry out this activity
   4. The potential cost of missing out on deadlines if data cleansing and migration overshoots the allotted time
   5. Lack of communication across different stakeholders will further aggravate the faulty data migration strategy
5. Lack of knowledge of the target system
   1. Substandard performance of the new system which will not deliver expected results or aid in recouping costs in the time specified
   2. Unsatisfactory user experiences and a decline in confidence in the management
   3. Degeneration of relationship with your ERP vendor and partner
   4. Inability to use the ERP system to its full capacity or potential, and reap the benefits for which it was intended

A basic recommended strategy

1. Plan for the data
2. Develop a migration strategy
   1. Data Volumetry
   2. Data identification
   3. Modeling ( Physical, Logical, Business)
   4. Data value analysis
   5. Data cleaning
3. Stick to data standards
4. Carry out sufficient testing

ERP Data migration best practices

Though people assume that all of the data in the legacy system should be transferred to the new one, the truth is that not all of the existing data are valuable to your business later on. Based on your organization’s specifications, this can be an opportunity to filter and remove irrelevant, wrong, or duplicated data in your system. No-longer-used data can also be identified as it is unnecessary and invaluable to your business. These data will not be lost permanently since the legacy system can be maintained in read-only mode for review when needed. Usually, the new data model varies a lot, and the choices made on the ancient system, sometimes 10, 30 years ago cannot be migrated easily.

Here is a list of tips to help you in data migration :

All experts are convinced that the data migration strategy must take into account the following aspects or else the project will fail:

• Putting the target system at the center: the new system is certainly more demanding on the quality of the data inserted into its system. Do not force data from the source system at all costs, but modify and clean up the data so that it is compatible. It is therefore imperative to train experts on the data model, concepts, and management rules incorporated into the new system.
• Anticipate problems with data: depending on the volume of data transferred, any error in the implementation of the data migration strategy is extremely costly. Good preparation and an iterative approach are the keys to success. It is not possible to anticipate all migration rules before the project. It is necessary to iterate, migrate, test, and reach 80% of the work done as quickly as possible.
• Automate what can be automated: Manual data migration should be limited to very small volumes of data. Indeed, any manual data transfer is prone to numerous data entry errors and is not sustainable in the long term. Many tasks can be automated with the participation of your team’s modernization experts, such as data analysis, mapping, data cleansing, etc.
• Share information with all stakeholders: to keep the team at peak efficiency, it is important to share information and to have indicators and reports that closely follow the progress of the migration.
• Start with less critical and small components and data: constantly moving to the more complex ones. Such a strategy will make it possible to constantly control and check the changes after each stage.
• Spot your data weakness :
  • Data Purging And Storage: a huge amount of data have been stored over the years, and as the business rules changed some data are invalid or unused. An exhaustive assessment of your data has to be performed to maintain a new system as clean as possible.
  • Poor Organization Of Data: If those data aren’t organized, then it will certainly cause failure and the organization will crumble.
  • Inconsistent Formatting: without the right formatting the system will struggle to distinguish between those data. Therefore it will cause the system to fail
  • Corrupted System: System corruption is a nightmare for users. Bad data can wreak havoc on the systems. It can also cause unexpected behavior to the system.
  • Integrations Failure: In that case, the data must be passed back and forth between ERP solutions. But, if the third party is anyway corrupted or unclean, then it can cause the integration to fail.

Resources to read more

• Best practice migration checklist
• Migration Checklist
• Checklist ERP Focus
• Best practices ERP Checklist
• Common questions about ERP Software