Features - Enterprise Data Insights:

DATA MIGRATION: THE SECRETS OF SUCCESS
by Gian Jagai & Larry Coblentz

Data Migration can present a very unique and painful process for most IT environments. Data Migration exposes all of the 'dirty little secrets' that exist within a company's data storage infrastructure. Secrets like non-redundant storage configuration in environments that demand high availability; backups that run past the backup window and fail to complete successfully resulting in 'fuzzy' backups; and poor management of the lifecycle of data, resulting in the migration of large amounts of useless or irrelevant data. Data Migration need not be a public confession of these secrets but with proper understanding of your data needs and resources, the technical options available and the right migration plan; a successful Data Migration can leave your storage infrastructure in a much stronger and more manageable place and remove the large burden those little secrets create. It does not matter how large of a migration you are contemplating, from a small tens of gigabytes DAS environment to a large multi-terabyte Oracle database migration on large SAN arrays; understanding your data profile is critical for a successful migration. Your data profile encompasses the following topics:

• Who is currently consuming the data to be migrated?
• What are the current business groups, processes and customer users affected?
• What applications are being migrated, such as databases, email, web content etc?
• How critical is the data? How long can outages last for the migration?
• What are the driving factors for this migration?
• Who are the stakeholders in the migration? Are they all aware of the migration plans?
• What would be considered a successful data migration project?

For example a recent company had the following data profile:

1. The "Financial" and "Decision Support" business units required a higher performing more reliable storage array technology. In total, there were 6 TB of data to be migrated. 4 TB - "Financial" : 2 TB - "Decision Support".
2. The data migration project would affect the market research, financial analyst, and accounting departments.
3. The databases implemented by these systems were Oracle and Informix.
4. The data involving the "Financial" systems was mission critical and required an on-line migration. The data involving the "Decision Support" systems was business critical and could be migrated off-line. The outage allotted to the "decision support" system was 48 hrs.
5. Faster response times for data access as well as scalability capabilities requiring no downtime were the primary business drivers justifying the effort to migrate to an advanced storage technology.
6. Each business unit assigned a committee dedicated to the planning, design, implementation, and verification for each of their respective data migrations.
7. Upon final testing and data validation for each data migration, the actual removal of the source storage array from the corresponding hosts was considered the milestone for a successful migration.

After you understand the requirements and expectations for the data to be migrated, you can focus on the technical ways and means of implementing the data migration. Data Migration techniques are differentiated based on the level in the data infrastructure the migration takes place in, specifically:

• Host Based - Native operating systems copy methods such as 'cpio', 'dd' or 'xcopy'
• Host Based - Utilizing Volume Management mirroring capabilities. This solution provides for on-line migration capability.
• Host Based - Using a utility such as 'robocopy'
• Storage Array Based - Using vendor supplied data replication facilities such as Business Continuance Mirroring and SnapShots within the storage frame as well as storage frame to storage frame data replication facilities which utilize synchronous / asynchronous data transfer methods. Such facilities provide for on-line data migration and eliminate the need for invasive host based replication solutions.
• Appliance Based - Data Migration appliances have been developed that will perform the data migration at the fabric layer. With this type of solution, there is no need for software (agents) to run on the host or storage array. The appliance sits between the host and the source and target storage arrays. The mapping function as well as the actual data migration is executed by the appliance.
• Software Based - Companies have developed software solutions that allow data to be copied at the block level from one storage array to another storage array. This is accomplished through the implementation of software (agents) which execute on the host. These solutions provide the necessary functionality to allow for on-line data migration capability
• Backup Based - Performing a 'Full' backup of the data, shutting down the application(s) and restoring from tape. For example a recent company used the following methods to migrate their data.
• The company, referred to previously, required 4 TB of "Financial" data to be migrated on-line. The source and target LUN sizes were significantly different in size. As a result, there would have been a substantial amount of wasted disk space if logical volume mirroring was implemented for the data migration.
• The customer investigated different approaches to accomplish their on-line data migration objective.
• The customer's investigation concluded that the host "Software Based" approach for on-line data migration would meet their objectives for uptime and integrity. The third party software solution selected required minimal "planned" downtime (approx. 1hr) to install. The actual data migration was executed on-line. Once the data migration was completed, minimal "planned" downtime was once again required (approx. 45 minutes) in order to rename the target volumes so that they would be recognized by the application.
• This same company also had requirements to migrate 2 TB of "Decision Support" data. As mentioned previously, the "Decision Support" system was not considered to be mission critical. As a result, system downtime was acceptable.
• The customer investigated different utilities offered by the UNIX OS to execute the off-line data migration.
• The customer concluded that the "dd" command would provide for the most effective and efficient approach to achieve the data migration objective. There is "raw volume" data utilized by the "Decision Support" system as well as "file system" data. Although a utility such as "cpio" would be sufficient for "file system" data migration, this utility will not work with "raw volumes". The "dd" command is comprehensive, in that it does a "block by block" copy from the source volume to the target volume, and will work with both "file system" and "raw volume" data.

Once you understand your data profile and the best method to implement the data migration, you will have to create a migration plan to cover all your bases and that no unexpected surprises occur. Like any other plan the steps should be defined, resources mapped and informed of their function and changes to the plan documented and agreed upon.

The migration plan can be divided into two separate documents for very large migrations but should cover the macro level milestones, resource tracking and other traditional project plan content. In addition there should be a low level plan for the sequence and syntax of the commands, scripts and software that will be used during the migration.

Finally, the data profile has be defined and agreed upon, the technical methods to perform the data migration and the migration plan is defined and all of the stakeholders are ready to go on the project. Some final actions that can greatly improve the success of the project are:

• Have a Backup-Out plan, for each command, script, software, array utility used in the migration have an 'undo' capability. Thus if something unforeseen happens you can go back to a known state and cut your losses before you run beyond your migration window. Sometimes your 'undo' plan is restoring from tape.
• Perform a 'dry-run' of the migration. While you may not be able to perform every little step you can try out the migration methods. Perhaps the vendor has some test equipment to which they can give you access to perform the testing. Testing gives confidence that the actual event will go well and can increase the technical ability of your staff performing the migration.
• Stage your migration. 'Don't bite off more than you can chew', if possible perform your data migration in stages. The stages can be hourly or across days and can allow you to get a better judge on the success of the migration. Combined with a back-out plan for each phase this can be an excellent method to ensure success.
• Manage expectations, of the project throughout the life of the project. Prior to beginning the migration ensure that the data profile and stakeholder expectations are still in-synch and that the migration will successfully meet the needs of the migration drivers.
• Communication between your staff, project consultants, technical consultants and vendor personnel is critical throughout the project. To ensure technical feasibility and smooth data migration everyone involved should know and understand their roles and expected times to be involved. Calls at midnight to the vendor support 1-800 number should be the very last option during a data migration.
• Address any short comings in the environment such as failing backups, non-redundant storage connectivity and old data accumulation. Mitigating or fixing these problems before the data migration can increase the odds of success.

Data migration should not be a process of discovery but an act performed with the full understanding of your environment. With knowledge of your data profile, understanding your technical challenges and capabilities and proper planning your data migration can be a process of enlightenment and understanding of your storage environment resulting in a more robust and capable infrastructure and more productive staff.

Gian Jagai, a storage consultant for GlassHouse, has five years' experience providing technical support in multivendor SAN and NAS environments. Reach him at gjagai@yahoo.com.

Larry Coblentz, a senior storage consultant for GlassHouse Technologies Inc, has more than 19 years experience as a technical consultant. His current area of expertise is in developing and deploying High Availability engagements with a focus on designing and implementing Storage Enterprise and Disaster Recovery solutions. Reach him at lcoblentz@glasshouse.com.