There are many reasons why an organization may face a large ECM data migration. Most of the times companies start thinking of ECM data migration because they want to modernize workflows. Other times, this may be because of a recent acquisition, a part of a strategic effort, or a push to minimize expenses in maintaining old systems.
Whatever the reason, a large ECM data migration is a daunting thought. And it often fails. These failures are usually the direct result of poor planning. This is why researching for available migration tools and processes is such a crucial part of ECM data migration. We’ll mention the tools at the end. For now, here are some guidelines to help you plan a proper ECM data migration process.
1. Study the Source of Data
The first step in any ECM data migration project should be studying the source of data. This data is the key subject of the entire process, so it goes without saying that teams need to know what they want to migrate. Here are a few steps to help your team get to know the data better:
a. Ask the right questions:
- What types of documents are stored in your current system?
- What indexes and metadata are used for describing each document type?
- What are the relationships between individual documents?
b. Analyze the enterprise content management system in which documents are stored:
- How can you extract documents from your current ECM system?
- How the documents relate to each other?
- How is the metadata stored?
c. Understand the business environment:
- How can the team migrate all the data without disrupting the daily operation of the entire system?
- Are there regulations that require your business to store documents in a long-term archive?
- What are your organization’s’ business requirements?
Obviously, these questions lists are neither exhaustive or following a workflow-specific order. Each organization will have specific situations that the ECM migration team should discover before starting any data migration.
2. Extract Data and Metadata
Extracting the whole data and metadata from an enterprise content management system is a challenging process. Usually, ECM systems provide a mechanism for pulling out individual documents one at a time. However, pulling out every stored document is an entirely different situation. As an option for simplifying the whole process of extracting document by document, you may consider the following:
- API – Application Programming Interface. Even though API is given for retrieving multiple or individual documents for a presentation, this interface can be used for extracting data for migration purposes. The positive aspect of migrating data via APIs is that you have excellent control over what gets extracted and where it is stored in the new system. The downside though is speed. Extracting documents and metadata via API can become unsustainable when there are a large amount of data to move.
- Batch Tools. Some enterprise content management systems are providing batch tools for loading and extracting documents of large-scale directly to and from the system. If your ECM system has such tool, you can easily use it for extracting data and metadata.
- Direct Database Access. Some enterprise content management systems can extract binary data and other metadata directly from a data source. The data migration team will need to have a clear understanding of the data storing process in order to perform a proper data migration.
- Engaging a vendor. Engaging the right vendor that has a set of proven practices will help you easily manage and simplify your data migration to or between ECM systems. ECM Data Migration is a well-developed industry and there is no need for in-house teams spending the time to reinvent processes and tools that are already out there. This is why it can be a great idea to simply hire the right team for the job. Learn more about How To Pick The Right Vendor For Your ECM Migration Needs.
Keep in mind that most of the times, large ECM systems use various methods of data compression and data encryption. This further complicates the ECM data migration process and this is yet another reason why hiring a vendor may be an excellent route. Your in-house team will then be there just to make sure that the vendor is doing things properly.
3. Recording of an Audit Trail
When the data migration is finished, you must have detailed information to ensure that every document has been migrated. You can easily achieve this by the recording of an audit trail during the conversion process and employing a series of checks and balances during all phases of the migration.
To determine if the data migration process was successful, it is necessary to provide an audit trail documenting the following:
- Where did the document come from?
- What metadata were connected to the document?
- Did the document change? If yes, why, how?
- Can the document be located and accessed in the new system as in the previous system?
- Are all the documents from the source ECM system accounted for in the target ECM system?
Because of the high volume of data, it is not recommended to perform manual spot checks of each document. Manual verification should be performed for a batch of documents for every given type.
4. Enrich or Add New Indexes
Indexes contain key information about the document such as customer name, statement date, and an account number. With the usage of this metadata, a search can be made to access some particular document within the ECM system.
The available indexes depend on the document type and the information that can be found on each page. You may be able to extract this metadata from your enterprise content management system as it is and preserve it in some form. If this is not the case, you will have to re-index the documents.
Either way, the data migration process gives you an opportunity to enrich or add new indexes so you can create new ways of accessing your documents.
Once documents have been loaded, some ECM systems don’t allow new indexes to be added, so this can be an excellent opportunity to enhance the value of documents stored in your enterprise content management system.
5. Loading Documents
Now when you have gone through the previous steps, your data is ready to be migrated to the new system.
Usually, the inputs that you will supply to your new enterprise content management system consist of the document data, indexes, metadata, and print resources. If the documents were extracted from the existing ECM system as individual files, file system performance might become a problem. In order documents to be loaded efficiently, some ECM systems enable document files to be joined into a single stacked file. This is a file that contains thousands of conjoined document files. The metadata of these documents is located in a separate index file that contains lengths and byte offsets, pointing to the location of one individual document in the stacked file.
ECM systems are providing few mechanisms for loading documents in large batches:
- API – Application Programming Interface: Some ECM systems are providing an API which you can use to write custom applications to load documents.
- Loading Utility: Some ECM systems are providing an app that will load the content if they are provided with a document data file and index info.
The end goal of every ECM data migration is to transform the business processes locked away in legacy systems into a modern ECM solution that has the features required by today’s’ businesses. Additional benefits are including assurance of performance, reduction in total cost of ownership, and functional scalability to meet future requirements.
To achieve these goals, the organization must ensure that the whole process of ECM data migration is well planned and performed.
Wrap-Up
As platforms upgrade, transform and integrate, organizations need to take their data along with them on the ride. Managing an ECM data migration project properly is a major undertaking and crucial factor for the overall ECM upgrade success. With proper planning, executive support, the right tool and tech support, you have every reason to succeed. While the steps above may not be an exhaustive list that’s applicable in every conceivable case, it’s a very good starting point.
As we mentioned at the beginning of this text, a data migration tool is also crucial. Armedia has helped organizations like Office of Personnel Management, Ineos Phenol, Superior Energy, Federal Bureau of Investigation and many others migrate from one ECM to another using Caliente.
Caliente is an automated data migration solution designed specifically for large scale content migrations. Caliente has the ability to migrate to and from many different electronic formats, including but not limited to, ECM Documentum, ODBC compliant databases, Alfresco, eRoom, SharePoint, and even hot folders.
Even if a large scale ECM data migration may seem daunting at first, a proper planning and the right tool can ensure success.
The post 5 Key Steps for Proper ECM Data Migration Planning so You Won’t Trip, Fall, Crash and Burn appeared first on Armedia.
