In working with many clients across hundreds of projects, we continually hear similar themes around large scale migration efforts. Here are the areas where our clients most commonly struggle:
Scale and Velocity Requirements for Migrating Large Numbers of Applications
Most organizations take months to create future state application deployment architectures, and then migrate a single application. When an enterprise has hundreds of applications to migrate, this challenges the viability of migration.
Lack of Visibility into Complexities and Dependencies
If you don’t have visibility into an application stack, you can’t migrate it. Whether this is due to poor documentation, legacy or acquired assets, or decentralized deployment practices, it is common for organizations to lack visibility into application components, configurations and dependencies. Even though an enterprise may have a list of all assets and their details within a CMDB, how do you validate those assets, their architecture patterns and dependencies, to determine which assets must move, and whether they are compatible with the target cloud platform?
Suitability of a Migration Approach
How do you determine the specific migration approach for an application (Rehost, Replatform, Refactor), and which applications can leverage cloud-native components?
Migration Sequencing and Execution
How do you determine and account for the right sequencing and configuration changes for the various dependencies, and how do you account for the configuration changes required during the execution phase? Making a configuration change can take only a few minutes, but making the change out of sequence, or missing a change, can result in many hours or even days of troubleshooting.
Migrating Mission Critical Applications with Zero Downtime
Most enterprises have mission critical applications that must be up 24×7, with large data sets and databases constantly updated with multiple upstream and downstream integration points. Databases are especially difficult to move, and if moved incorrectly, these assets can have a profound impact on the organization.
Let’s look at the six key enablers that must be in place for any migration project to succeed.
Six Key Enablers
Enabler #1 – High Value Data Collection
It is essential that businesses understand their application estate. We strongly believe in “High Value Data Collection,” which focuses on defining upfront: the context, the data to be gathered, how it will be consumed and the final outcome.
Taking on a migration initiative requires a deep dive into the enterprise’s portfolio. Gear your discovery and analysis effort toward addressing the main business drivers and specific goals. Based on those goals, target an analysis exercise at the estate, application and infrastructure levels, as well as specific to an individual component. We recommend that every organization perform an analysis at each of these levels based on various business drivers.
The following are the typical use cases:
- Defining general strategy for transformation across the landscape, understanding common patterns and identifying first movers (Estate Level Analysis)
- Addressing challenges for a specific application portfolio for a line of business (Application Level Analysis)
- Addressing specific pain points, such as middleware or database transformation (Component Level Analysis)
- Addressing more specific business drivers, such as “exit a data center,” which may require an infrastructure centric analysis
The type of data required for each of these use cases varies, from general application information to asset details, detailed architecture and dependency information.
Even though creating a data model to define exactly what is needed for each of these analyses should be a no-brainer, many organizations struggle with this. Issues range from not finding required information, to being overwhelmed with the amount of data, as well as the time and effort needed to gather it. We recommend that organizations spend up-front time and effort creating a data model that defines the use cases with the following requirements:
- Asset information
- Additional analysis attributes
- Data gathering mechanisms
The data gathering mechanism can range from self-service questionnaires, to discovery/monitoring tools, to CMDB sources. Many discovery tools have additional capabilities for analysis, including cost analysis, architecture recommendations and platform recommendations.
In summary, organizations need to enable high value data collection through the proper definition of use cases, asset details and other functional data that is key to the analysis. They also need a robust discovery mechanism that can gather all the required information, and maintain it in a repository to use in further stages of analysis and eventual migration, if required.
Enabler #2 – Objective Assessment and Analysis with Rules
Once we have the required visibility into the landscape and the various assets, an assessment and analysis determines the fate of an application. As described above, an analysis effort can be either at the estate level, the specific business and applications level or the infrastructure level.
The outcome from an estate level analysis typically provides visibility and direction for the organization, including first movers. It also shows where the organization needs to focus its efforts in the short, medium and long term, to meet business objectives.
In the context of an application level analysis, a typical ask is: which applications are best suited for migration, and what platform and architecture patterns are suitable? The key application metadata used in such an analysis falls into four categories: Business, Technical, Operational, and Security and Governance. As an example, technical metadata categories include: Architecture, Technology Stack, Automation, Performance, Scalability, Dependencies, Data Size and Data Velocity.
A suitability analysis requires defining cloud-ready characteristics, and a scoring mechanism that can be applied to all applications to determine whether they should be moved to the cloud. For example, an organization can define the following characteristics that deem an application not suitable to be moved to the cloud platform:
- Application is large, single instance and/or monolithic, and cannot be broken into services
- Application has an external dependency that cannot be reached from the cloud platform
- Application is not compatible with list of approved, compatible cloud libraries
- Application has contractual, legal or licensing issues due to a third party packaged application or technology needed for running in a cloud environment
Once you deem an application is suitable, you can choose the appropriate cloud platform and migration pattern. We recommend a quantitative approach in which each of the application’s characteristics are scored against an endpoint and summed up to determine the suitable cloud platform. In many instances, the target cloud platform decision is based on factors other than technical features, including licensing, contractual availability of specific services or affinity.
Various factors determine the migration pattern, including business function, objectives, business cycle, criticality and priority, application architecture and effort required. For example, a custom, off-the-shelf application may be more suitable for Rehost (rather than Replatform or Refactor). You can apply a set of rules to the characteristics of an application to determine the migration pattern.
Determining cloud suitability and migration pattern requires creating and defining what suitability means, and then carefully analyzing the characteristics of the application using objective and qualitative means.
Enabler #3 – A Prescriptive Migration Plan with Attention to Details
Once an application analysis determines the application is suitable for migration, and its migration patterns are known, you need to develop a detailed migration plan. Besides the overall goal of the migration, a migration plan should include execution level details, including all migration tasks and owners.
The execution level details will vary based on the chosen migration pattern. For example, a Rehost pattern will mainly have infrastructure tasks with few application configuration changes, as well as a test and validation plan. A Refactor pattern migration effort will need to include details on each of the components to be changed, including its current and future state, functionality, accompanying code, deployment details, and test and validation plans.
At a minimum, the migration plan should include the following:
Business drivers and expectations
- Current and future state architecture
- Patterns and approach
- Asset list, dependencies
- Desired configuration or changes
- Tools (migration, deployment, monitoring, logging)
- Detailed migration tasks, deployment plans with RACI
- Readiness, test, validation, cutover and operations plans
For migration efforts to achieve the desired velocity and scale, the migration plan for execution needs to be very prescriptive. In addition, reusable approaches and artifacts and self-service approaches for different stakeholders will greatly help reduce latency.
In the process of creating a detailed migration plan, one should account for approaches that reduce the amount of downtime required for cutover during a migration effort. You can use various approaches to achieve close to zero downtime for an application migration, including: blue/green deployments, incremental replication, isolated environment for testing and validation before cutover, or temporary DNS records for testing.
For example, you can develop standard migration checklists for each of the patterns, which you can reuse across all applications. As many organizations have a common set of tools for testing and validation, a common set of tests across patterns is useful to the operations team for onboarding applications at the desired velocity.
Enabler #4 – Automation with a Factory Approach
The key to address the scale, velocity and safety challenges in a migration project is to incorporate automation and reusability. As more and more tasks are automated, the migration process becomes easier and can be scaled, increasing the safety and velocity of migrations.
Migrations at scale require a factory approach. This uses a mix of tools and processes to improve quality, accuracy and precision. In terms of tooling, selecting the right mix of tools and investing time and effort to developing the required automation will achieve the required scale and velocity. Besides the tools, you need a process and an orchestration layer that addresses all facets of a migration, including sequencing, configuration, dependencies, testing and validation. You must also have an enablement program in which additional staff are trained and deployed based on the demands of a large scale migration.
In addition, an automation and factory approach will help reduce downtime during a migration, user errors introduced by manual tasks and traceability requirements.
In many instances, an existing CI/CD and governance system may suffice for applications where the deployment is highly automated. However, if automated deployment is not available (e.g., for COTS applications and the Rehost pattern), you need to develop it yourself or use out-of-the-box automation provided by third party tools.
Enabler #5 – Strong Governance for Visibility, Continuous Process and Quality
Executing on the migration plan requires addressing all moving parts during the migration project. Besides detailed migration plans, checklists and automation, you need effective play calling, essential for the smooth execution of migration tasks.
Migration requires a factory and NOC-like approach in which runbooks exist for the majority of the tasks and scenarios, and all relevant staff know exactly what to do in which scenarios. You will also need a solid governance process to oversee the migration execution.
In a factory approach, you can dedicate a workbench (a self-contained group with all the tools and skills) for a specific pattern. The workbench concept is also useful in scaling, where additional workbenches are deployed to increase the velocity of the migration effort when required. Multiple workbenches do however pose additional challenges in terms of governance and tracking: ensuring all parts of the factory are working to their optimal levels.
Enabler #6 – Key Cloud Performance Indicators (KPIs)
Once assets are migrated to the cloud, there needs to be a comprehensive governance and monitoring framework to continuously track various aspects of the target platform and applications. Even though many enterprises have mastered this aspect in their data centers, the cloud platform poses unique challenges where the same set of tools and processes may not work due to various factors, such as multi-tenancy, scale and lack of visibility/granularity to the infrastructure layer.
Besides deploying the typical infrastructure and application monitoring and logging solutions, you need to monitor additional elements when applications are in the cloud. A few examples we consistently hear from clients include cost management, posture and compliance. To keep tabs on the environment, the cloud platform requires a different set of tools and processes. We recommend defining cloud specific KPIs to effectively monitor the target platform.
A large-scale migration project is a transformation effort that requires careful analysis, planning, development of the required automation, and flawless execution. CTP’s Application Migration Framework (AMF) is an extension of the Cloud Adoption Program (CAP) developed to address the multiple challenges and use cases encountered when migrating large portfolios of enterprise applications in a short period of time. It can take years to develop and refine the knowledge, skills, tooling and processes to quickly and efficiently migrate large application portfolios. AMF helps organizations significantly reduce the time needed to migrate applications.
In summary, embracing each of these six enablers addresses all challenges quickly and more easily, encouraging enterprises to realize the benefits of adopting cloud. The six enablers promote fundamental change and deliver results by addressing a broad range of key stakeholder needs.
Contributors to this article included Boris Ekelchik, Andy Clark & Mike Draper