Top Banner
PLIA Data Migration Plan Page 1 PaaS Technology Expansion Data Migration Plan May 6, 2020 Prepared by Pollution Liability Insurance Agency and EightCloud
12

Data Migration Plan

Mar 19, 2023

Download

Documents

Khang Minh
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Data Migration Plan

PLIA Data Migration Plan Page 1

PaaS Technology Expansion Data Migration Plan

May 6, 2020

Prepared by Pollution Liability Insurance Agency and EightCloud

Page 2: Data Migration Plan

PLIA Data Migration Plan Page 2

Table of Contents

Executive Summary Data Migration Overview Big Picture Data Sources Export Tools Import Tools Staging Testing Source Data Clean-Up Source Data to Clean Data Mapping Relationships Test Data Loads High Level Plan High Level Plan Data Migration High Level Plan Document Migration Roles and Responsibilities Conclusion Appendix A: Signoff

Page 3: Data Migration Plan

PLIA Data Migration Plan Page 3

Executive Summary This migration plan provides an overview of the process that will be used to move existing legacy data and documents from the current database (Access) over to the Pollution Liability Insurance Agency’s (PLIA) instance of Salesforce.com (SFDC). Data migration should not be viewed as primarily a technical task with field mapping exercise, it is actually extremely important for PLIA’s Subject Matter Experts (SMEs) to be heavily involved in this activity in order to ensure that the new system has accurate, reliable data as a starting point. Having accurate and believable data is critical in order to ensure business continuity and end-user adoption. PLIA is faced with some challenges when it comes to data migration. Some are unique to PLIA and some are common to any organization using several disparate systems:

Data cleanup. The need to capture historical data even if it is not being used.

o Make sure picklists are not restricted. Lookup relationships need to be considered in the data mapping.

o Sequence of import matters. o Leave lookup filters optional until after the merge.

Efforts to clean up existing data may not completely resolve all of the known issues prior to migration.

Despite these challenges, successful data migration can and will be achieved through good planning, tight scope control, sophisticated tools, and many rounds of testing. Throughout the following pages, we will explain the overall process including key terms, tools, and methods. Essentially, data migration can be broken down into three phases: Plan, Prepare, and Test. 1) Plan – In the ‘Data Migration Overview’ section we’ll review the overall approach and basic concepts of data migration. 2) Prepare – In the ‘Source Data Clean Up’ and ‘Data Mapping’ sections we’ll discuss the preparation that must take place before any data migration can be performed. 3) Test – In the ‘Test Data Loads’ section we will highlight the process for running several iterative test data loads to check (many times over) that the field maps have truly been defined properly. This document makes several references to an associated Excel document titled PLIA Field Mapping which provides the details that don’t present well in a Word document.

Page 4: Data Migration Plan

PLIA Data Migration Plan Page 4

Data Migration Overview Big Picture This diagram shows the basic components of a data migration. The current internal database is known as the “Source” system, and Salesforce.com is called the “Target” system. The data from the source will be extracted and any necessary preparation can be done to the data in a staging area (typically a spreadsheet and data load tool) prior to being loaded to the target system. Figure 1.1 “Data Migration Diagram” Data Sources

Access Export Tools Access We will rely on PLIA to use the export functionality in the Access database to export files to Excel and provide to eightCloud in a Box folder. Import Tools There are a variety of tools available for importing data into the SFDC platform. Salesforce.com provides this helpful overview of tools, their limits, and their availability. We will use a combination of the Jitterbit Dataloader for Salesforce (near the bottom of this list) and the Salesforce provided Data Loader. Both should have all of the necessary capabilities for this project and are free to use. Table 1.1 “List of Import/ Export Tools supported by the SFDC platform” Source: Salesforce.com (http://goo.gl/iAVUMf)

Tool Editions supported Number of records you can import or export

Internal or external to Salesforce

Additional information

1 Data Import Wizard (unified)

All editions except Database.com

Up to 50,000

Internal An in-browser wizard that imports your organization’s

Page 5: Data Migration Plan

PLIA Data Migration Plan Page 5

accounts, contacts, leads, solutions, and custom objects.

2 Data Loader Enterprise, Unlimited,Performance, Developer, and Database.com Editions

Between 5,000 and 5 million

External Data Loader is a client application for the bulk import or export of data. Use it to insert, update, delete, or export Salesforce records.

3 Import Wizard for Accounts and Contacts

Contact Manager, Group, Professional, Enterprise, Unlimited,Performance, and Developer Editions

Up to 500 records for an individual user Up to 50,000 records for multiple users

Internal An in-browser wizard that imports personal contacts and accounts from ACT! and Outlook or from other sources; or your organization’s business accounts and contacts.

4 Import Wizard for Person Accounts

Enterprise, Performance, Unlimited,Performance and Developer Edition

Up to 50,000

Internal An in-browser wizard that imports an individual user’s person accounts or your organization’s person accounts

5 Import Wizard for Leads

Group, Professional, Enterprise, Unlimited,Performance, and Developer Editions

Up to 50,000

Internal An in-browser wizard that imports your organization’s leads. Read more.

6 Solution Import Wizard

Professional, Enterprise, Unlimited,Performan

Up to 50,000

Internal An in-browser wizard that imports your

Page 6: Data Migration Plan

PLIA Data Migration Plan Page 6

ce, and Developer Editions

organization’s solutions. Read more.

7 Custom Object Import Wizard

Contact Manager, Group, Professional, Enterprise, Unlimited,Performance, and Developer Editions

Up to 50,000

Internal An in-browser wizard that imports your organization’s custom objects. Read more.

8 Data Export Weekly export available in Enterprise, Unlimited, andPerformance, Editions. Monthly export available in all editions, except Database.com

No limit Internal An in-browser wizard that exports data in .zip files that are up to 128 MB each on a monthly or weekly basis. Read more.

9 Third-party tools

Varies Over 5 million

External See the AppExchange for available tools.

10

Jitterbit Data LoaderTM for Salesforce (third-party tool)

All editions Over 5 million

External Free data migration tool offered by a salesforce.com, inc.partner. Read more.

Staging Data will be extracted and staged on a Box.com drive provided by PLIA. We plan to use comma separated files (.csv) to hold the legacy data before it is mapped and imported. Further, since PLIA has access to a full-copy sandbox (which means we have plenty of record storage available for resting) we will populate that instance of SFDC first before attempting a load directly into the SFDC Production environment. Testing The first round will be a simple extract and load for one Object. Then, we will add an Object and continue an extract-map-load cycle. Lastly, we test associations and begin testing tables that relate to each other through external id values. Each step

Page 7: Data Migration Plan

PLIA Data Migration Plan Page 7

along the way ensures that we are mapping to the correct fields and maintaining or establishing proper associations between the various tables.

Source Data Clean-Up A clean, consistent database is the foundation of any successful system. eightCloud will help PLIA avoid the “GIGO (Garbage In Garbage Out)” pitfall when bringing the new SFDC system online by ensuring that we have clean, believable data as a baseline for launch. PLIA’s Access Developer will work closely with PLIA and the various business groups within PLIA to ensure (at a minimum) that 1) Fields are mapped, 2) Future-state field sets are defined, and 3) Data retention policies on a table-by-table basis are defined. As a part of Phase II, the eightCloud team will document issues and concerns with data migration so they can be brought to the SMEs and the Salesforce Implementation Team. As cleanup decisions are made and executed, it is critical to document and share these decisions with all appropriate stakeholders. The cleanup effort includes eliminating duplicate data, making data formats consistent, plugging missing data with valid values, and consolidating data. The SMEs involved in the data cleanup will be asked to decide on the following key points:

Scope of data export – how much data and which records should be migrated?

Accessibility Strategy – it’s best practice to keep the source system available to end users (possibly as read-only) for a period of time after the target system is live.

Archive Strategy – once the source data is extracted and after Salesforce.com is live, what should be done with the internal database? Should it be archived, and after how much time should it be purged?

Source Data to Clean During the test data load process additional data to be cleansed will likely be identified and can be tracked on this list. eightCloud will assist with identifying potential data issues and adding them to the list. PLIA will have ownership of the list and ensuring the data is cleansed.

Data Mapping Data mapping needs to be completed in the PLIA Field Mapping Excel file. Each of the tabs represents a table or a group of related tables that have been mapped from the source to a target Salesforce Object. External record IDs from the several legacy database systems will also be stored in SFDC to facilitate cross-reference checks back to the original systems. These fields are called “Source System ID” and “Source System Name”.

Page 8: Data Migration Plan

PLIA Data Migration Plan Page 8

Some of the source fields are mapped to several target fields (see account and contact tabs) as required by our final data model. Primarily this is to separate individuals out from organizations. In legacy systems the individuals and organizations were often stored on the same record. Other source tables are mapped directly to individual Salesforce Objects, while others are broken up, mapping to multiple Salesforce Objects. Where existing relationships are to be preserved, source tables will be migrated in sequence starting with the base or "parent" objects and proceeding to "child" objects in order to build parent-child "lookup" relationships in Salesforce that preserve relationships from the source database. In order to facilitate this process a tab called Data Load Order has been included in PLIA Field Mapping Excel file to provide a potential load order. The load order accounts for these items:

Dependencies between tables (e.g. a contact related to an account must be loaded after the account).

Like functionality has been grouped together. Track 2 schema that will be built in track 1 but largely not used until track 2.

Relationships It is critical to properly define relationships and associations in the SFDC platform (target system). For example, the system should enforce that a Contact should not be placed without being associated with an Account. Thus, Account and Contact should be related at the schema (table) level. To this end, as a part of the field mapping exercise all of the relationships have been established in the full-copy SFDC sandbox, including the external ID relationships. As part of the build phase a data import will be performed top-down through the relationship hierarchy. The dependent records will be inserted only after parent records have been defined and inserted.

Test Data Loads Testing converted data that has been loaded to Salesforce.com is the last step necessary to confirm that the system is ready for use. When testing is complete it confirms that the historical data is accurate and that the relationships between tables works correctly. The test process will be iterative, a process where more data and more complexity is added with each iteration. We’ll build on success until there is agreement that we’ve achieved our goal. The primary methods of testing will be these:

Record count matches (number of records in scope from source system should match exported records which should match imported record).

Page 9: Data Migration Plan

PLIA Data Migration Plan Page 9

Data Load Tool validation error checking (each of the data load tools work in concert with the Salesforce platform to ensure valid data is being loaded. For example, an email address without an @ symbol will fail to load).

Destination system spot checking (after data is loaded both PLIA and eightCloud will spot check the data).

As has been noted earlier, SMEs are the key to defining and measuring a successful data load. They are the ones best suited to interpret the data and find subtle errors with mapping or take issue with unforeseen implications of table relationships. To that end PLIA must own the final buy-off on the data being migrated. The eightCloud project team will assist and guide the SMEs through the process, proposing and explaining relationship options, executing the plan and resolving issues as they arise.

High Level Plan High Level Plan Data Migration The data migration efforts for the project is focused on Access data. Below is the plan:

1. Schema build will happen. a. This schema build was completed in May and is a required pre-

requisite of doing the pilot data migration. 2. A pilot data migration will be performed in which sample data will be loaded

as a proof of concept. The pilot data load will be minimal in scope and will primarily serve to prove out any major issues with the data load plan. Known concerns are listed below:

a. Test separation of individual/organization records from source system into destination system.

b. Test data loads to ensure we are able to uniquely identify each record. c. This pilot data load is targeted to be complete by the end of June.

3. System build will be ongoing. 4. Test data migration will happen in July.

a. The test data migration will involve migrating all or at least a significant portion of every data set.

b. Time estimates will be obtained from test data loads to more accurately plan final data loads.

5. After test data migration happens, test document migration will happen (see plan below).

6. A step by step production data migration plan will be put together based on test migration.

7. Schema and System build will be migrated to production at the end of August.

8. Production integrations will be deployed. 9. Any necessary final document migration will also happen just before go live

in August (see plan below).

Page 10: Data Migration Plan

PLIA Data Migration Plan Page 10

10. Ideally the data migration will be done during a “dead” time (weekend or otherwise) during which we can make a clean cut-over to the new system.

a. Certain data will be loaded ahead of time to minimize the time needed during the critical hours of the final data load (users, agency, department, etc.). Additionally, some data may be loaded ahead of time to facilitate the migration of documents. In these cases, incremental data loads will be performed at go-live to true up the new production system.

b. If a weekend for example, we will take the final data export at the close of business on Friday and immediately begin loading data.

c. Final Go/No-Go. i. Upon completion of the data load a PLIA representative will be

required to make the final call approving go live in the new system.

ii. In the unlikely event of some show-stopping issue, the PLIA representative would be required to approve a delay and reschedule of go live.

11. After Go-Live the project level plan for support would also support any data issues:

a. Extended eightCloud support remotely. b. Power users in each department will be the “first line of defense” after

the initial onsite period.

Roles and Responsibilities The following roles and responsibilities have been identified:

Organization Name Role / Responsibility for Data Migration

1 Subject Matter Experts (SMEs)

Provide input and review plans

2 Project Team Provide input and review plans

3 PLIA Cassandra Project Manager – Lead effort for data and document migration from PLIA’s side

4 eightCloud, inc. Lisa Engleberg Project Manager, provide oversight of overall project

5 eightCloud, inc. Eric Long Technical Architect – lead development team in providing integrations,

Page 11: Data Migration Plan

PLIA Data Migration Plan Page 11

provide oversight of data migration

Conclusion Data migration is extremely important to the success of the project. Historical data must be available for PLIA to operate successfully and the data must be as accurate as possible. The eightCloud team using the SFDC platform will work with PLIA to achieve this goal, providing tools to facilitate data mapping, data quality, and relationship definitions. We will also be in a position to pinpoint data that may be needed in “future state” but is lacking in the existing systems. Migration is challenging but it can be executed successfully. Using tools to automate manual updates will enhance productivity, and using scripts to enable data mapping will promote a scalable iterative approach. Most importantly, we assume heavy involvement during this phase by PLIA’s Access Developer and the various SMEs who are intimately familiar with the current data and can verify that migrated data accurately represents the future state needs of PLIA.

Page 12: Data Migration Plan

PLIA Data Migration Plan Page 12

Appendix A: Signoff

Name Role Action Date

1 Cassandra Garcia Executive Sponsor

Approved May 7, 2020

2

3

4