hidekazu-konishi.com

Summary of AWS Application Migration Service (AWS MGN) Architecture and Lifecycle Relationships, Usage Notes - Including Differences from AWS Server Migration Service (AWS SMS)

First Published: 2023-11-30
Last Updated: 2024-01-24

Today, I would like to summarize the architecture and lifecycle relationship of the AWS Application Migration Service (AWS MGN), as well as some important points on how to use it.

AWS Application Migration Service (AWS MGN)

AWS MGN is a service that migrates existing applications running on virtual machines (VMs) in on-premises, private cloud, AWS, and other public clouds to AWS Cloud in their current configuration using lift-and-shift.
In AWS MGN, continuous block-level replication is used to store the latest data from the source VM on AWS, and from this data, Amazon EC2 instances can be launched to perform testing and cutover procedures.

Difference Between AWS Application Migration Service (AWS MGN) and AWS Server Migration Service (AWS SMS)

Before the announcement of AWS Application Migration Service (AWS MGN), there was already a migration service called AWS Server Migration Service (AWS SMS).
AWS MGN and AWS SMS are similar services in terms of migrating VM environments to AWS, but they differ significantly in the following points:

Comparison Item	AWS Application Migration Service (AWS MGN)	AWS Server Migration Service (AWS SMS)
Migration Focus	Focuses on migrating existing applications running on VMs in on-premises, private clouds, AWS, and other public clouds (using continuous block-level data replication).	Focuses on migrating VMs in on-premises, private clouds (like VMware vSphere, Microsoft Hyper-V, etc.).
Target Environments	Targets VMs in on-premises and private clouds (like VMware vSphere, Microsoft Hyper-V, etc.), as well as AWS and other public cloud VMs.	Targets VMs in on-premises and private clouds (like VMware vSphere, Microsoft Hyper-V, etc.) and Amazon EC2.
Available Migration Methods	Supports migration using agents installed in VMs and outbound access. Also supports agentless replication in VMware vCenter environments.	Migration by installing SMS Connector in VMware vCenter environments or Microsoft Hyper-V hosts (migration using agents is not supported).

Thus, AWS MGN has the characteristic of being able to migrate a wider range of VM environments in a more flexible and automated manner compared to AWS SMS.

Basic Architecture of AWS MGN, Relationship with Lifecycle, and Usage Points

Next, I will explain the relationship between the basic architecture of AWS MGN and its lifecycle, along with some important points on how to use it, using a Architecture Diagram with numbered steps (A-1, B-1, etc.).

This architecture assumes the following migration method:
- Migrate by installing the AWS Replication Agent on the source VM server (not agentless replication).
- Encrypt data in transit and migrate via the internet (not using AWS Direct Connect or AWS Site-to-Site VPN).

Relationship between Lifecycle States in AWS MGN and the Architectural Steps Required to Move to the Next State

The lifecycle states in AWS MGN include Not ready, Ready for testing, Test in progress, Ready for cutover, Cutover in progress, Cutover complete. The state transition diagram looks like this:

Lifecycle State Transition Diagram of AWS MGN

The actions in this state transition diagram are linked to the steps in the "Basic Architecture and Steps of AWS MGN" Architecture Diagram as follows:

Lifecycle State Transition Diagram of AWS MGN (With Steps)

I will now explain in detail the relationship between the lifecycle states in AWS MGN and the architectural steps required to move to the next state.

[Not ready]

This state indicates that the server is in the initial synchronization process and is not yet ready for testing.
To move to the next state, first perform the following steps in the Architecture Diagram:

(A-1). Install AWS Replication Agent

First, create an IAM user to run the AWS Replication Agent and attach the following AWS managed policies as needed.

AWSApplicationMigrationAgentInstallationPolicy
AWSApplicationMigrationAgentPolicy
AWSApplicationMigrationAgentPolicy_v2
AWSApplicationMigrationFullAccess
AWSApplicationMigrationReplicationServerPolicy

Next, create an IAM user's access key and secret key.
Then, navigate to Source servers > Add server in the AWS MGN AWS Management Console, enter the required information, and execute the following command on the source VM server to install the AWS Replication Agent.

[ho2k_com@ho2k-com ~]$ sudo wget -O ./aws-replication-installer-init https://aws-application-migration-service-us-east-1.s3.us-east-1.amazonaws.com/latest/linux/aws-replication-installer-init
[ho2k_com@ho2k-com ~]$ sudo chmod +x aws-replication-installer-init; sudo ./aws-replication-installer-init --region us-east-1 --aws-access-key-id XXXXXXXXXXXXXXXXXXXX --aws-secret-access-key XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX --no-prompt

Note: If the AWS Replication Agent installation fails, please check if the requirements in Installation requirements are met.
In particular, I recommend checking the following points:
- Whether the RAM capacity of the source VM server is sufficient
  The above document mentions a minimum requirement of 300MB of RAM, but the actual agent operation may use more than that. Therefore, I personally recommend trying to install and run the AWS Replication Agent with at least 2GB of RAM, if possible.
- For connections over the internet, whether the source VM server can access the AWS MGN service endpoint outbound over HTTPS (TCP 443)
  I recommend checking the network and firewall access restriction settings of the source VM server, and the settings of any web proxy servers, if present.

Additionally, if other issues arise, refer to Troubleshooting for solutions.

Once the issues are resolved, and the AWS Replication Agent installation and access to the AWS MGN service endpoint are successful, the following steps in the Architecture Diagram are automatically executed.

(B-1). API Calls for Authentication, Configuration, Monitoring, and Synchronize Status
(B-2). Retrieve Software Component
(B-3). Launch Replication Server
(B-4). Data Replication
(B-5). Create Launchable Snapshot

Note: If errors occur in these steps, refer to Troubleshooting for solutions.
In particular, I recommend checking the following point:
- Whether the source VM server can access the replication server over TCP port 1500, which is necessary for data replication
  I recommend checking the network and firewall access restriction settings of the source VM server.

Once the initial synchronization of data replication is complete, the source VM server will be ready to launch test instances.

Note: If you want to restrict the IP addresses that can access TCP port 1500 from the network where the source VM server is located, assign an Elastic IP address to the replication server and allow access to TCP port 1500 for the Elastic IP address.
It appears that the AWS Replication Agent automatically identifies and sends traffic to the Elastic IP address assigned to the replication server.
Note: In the "(B-4). Data Replication" step, data is written to the replication server's EBS volume through continuous block-level data replication, keeping the source server up-to-date.
The Snapshot converted from this EBS volume is used in the steps of launching test instances or cutover instances later, so the data replicated from the source VM server to the EBS volume until just before creating this Snapshot will be reflected in the test or cutover instances.

Meanwhile, the following steps in the Architecture Diagram are executed as needed from the Staging Area Subnet where the replication server is located.

(E). API Calls for Authentication, Configuration, Monitoring, etc.
(E). API Calls for Retrieve Software Component, etc.

[Ready for testing]

This state indicates that the initial synchronization of the server is complete and it is ready to launch test instances.
To move to the next state from here, first select the corresponding source server for the VM server from Active source servers in the AWS MGN AWS Management Console, and set the "Launch settings" for the test and cutover instances.

In "Launch settings", you set the instance type, subnet, etc., of the EC2 instance to be actually launched according to the requirements of the source VM server and the target EC2 instance. However, the following point is particularly important to note:

Note: Whether to turn "Instance type right sizing" On or Off in "General launch settings"
If "Instance type right sizing" is On, AWS MGN will automatically specify the most suitable instance type regardless of the instance type set in the "EC2 Launch Template".
Therefore, the test instance may be launched with an instance type unintended by the user.
Especially in cases where the source VM server does not have the ENA module installed, necessary for Enhanced Networking, there is a possibility of failing to launch with an Enhanced Networking-supported instance type.
In such cases, turning "Instance type right sizing" Off and setting the appropriate instance type in the "EC2 Launch Template" will ensure that the EC2 instance is launched with the instance type intended by the user.

After setting the "Launch settings," selecting "Launch test instances" for the target source server in the AWS MGN AWS Management Console will execute the following steps in the Architecture Diagram:

(C-1). Launch Conversion Server
(C-2). Convert Disk to Bootable Snapshot
(C-3). Launch Test EC2 Instance

Note: If errors occur in these steps, refer to Troubleshooting for solutions.
Also, I recommend checking if there are any issues with the previously mentioned "Launch settings."
Note: In the "(C-2). Convert Disk to Bootable Snapshot" step, data from the source VM server written to the replication server's EBS volume through continuous block-level data replication is converted into a Snapshot.
Thus, the data replicated from the source VM server to the EBS volume up until just before creating this Snapshot will be reflected in the test instance.

In addition to the "Launch settings," configuring the "Post-launch settings" to control and automate actions performed after launching test and cutover instances using AWS Systems Manager Agent (SSM Agent) will execute the following steps in the Architecture Diagram:

(F). API Calls for Post-launch settings, etc.

Meanwhile, from the Staging Area Subnet where the replication and conversion servers are located, the following steps in the Architecture Diagram will be executed as needed:

(E). API Calls for Authentication, Configuration, Monitoring, etc.
(E). API Calls for Snapshot creation, etc.
(E). API Calls for Retrieve Software Component, etc.

[Test in progress]

This state indicates that the test instance for the server is currently running.

To move to the next state, first conduct tests such as data migration verification and application operation checks on the test instance, and determine whether it's ready for cutover.

If it's decided that cutover is possible, selecting "Mark as 'Ready for cutover'" for the target source server in the AWS MGN AWS Management Console will move to the next state, "Ready for cutover."

Conversely, if there are issues during testing and you wish to revert to the previous state, "Ready for testing," select "Revert to 'Ready for testing'" for the target source server in the AWS MGN AWS Management Console.

[Ready for cutover]

This state indicates that the server's testing is complete, and it is ready to launch the cutover instance.
The procedure to move from "Ready for cutover" to the next state, "Cutover in progress," is almost the same as moving from "Ready for testing" to "Test in progress," but I include it here for a better understanding of the subtle differences.

To move to the next state from here, first select the corresponding source server for the VM server from Active source servers in the AWS MGN AWS Management Console, and set the "Launch settings" for the test and cutover instances.

In "Launch settings", you set the instance type, subnet, etc., of the EC2 instance to be actually launched according to the requirements of the source VM server and the target EC2 instance. However, the following point is particularly important to note:

Note: Whether to turn "Instance type right sizing" On or Off in "General launch settings"
If "Instance type right sizing" is On, AWS MGN will automatically specify the most suitable instance type regardless of the instance type set in the "EC2 Launch Template".
Therefore, the cutover instance may be launched with an instance type unintended by the user.
Especially in cases where the source VM server does not have the ENA module installed, necessary for Enhanced Networking, there is a possibility of failing to launch with an Enhanced Networking-supported instance type.
In such cases, turning "Instance type right sizing" Off and setting the appropriate instance type in the "EC2 Launch Template" will ensure that the EC2 instance is launched with the instance type intended by the user.

After setting the "Launch settings," selecting "Launch cutover instances" for the target source server in the AWS MGN AWS Management Console will execute the following steps in the Architecture Diagram:

(D-1). Launch Conversion Server
(D-2). Convert Disk to Bootable Snapshot
(D-3). Launch Cutover EC2 Instance

Note: If errors occur in these steps, refer to Troubleshooting for solutions.
Also, I recommend checking if there are any issues with the previously mentioned "Launch settings."
Note: In the "(C-2). Convert Disk to Bootable Snapshot" step, data from the source VM server written to the replication server's EBS volume through continuous block-level data replication is converted into a Snapshot.
Thus, the data replicated from the source VM server to the EBS volume up until just before creating this Snapshot will be reflected in the cutover instance.

[Cutover in progress]

This state indicates that the cutover instance for the server is currently running.
To move to the next state, first conduct tests on the cutover instance such as data migration verification and application functionality checks from the source VM server, and decide whether the cutover can be completed.

If you determine that the cutover can be completed, select "Finalize cutover" for the target source server in the AWS MGN AWS Management Console to proceed to the next state, "Cutover complete."

Conversely, if there are issues during testing and you wish to revert to the previous state, "Ready for cutover," select "Revert to 'Ready for cutover'" for the target source server in the AWS MGN AWS Management Console.

Note: Selecting "Finalize cutover" to move to the next state "Cutover complete" will disconnect the data replication from the source VM server to the replication server's EBS volume, and all replicated data will be discarded.
Also, all AWS resources used for data replication to the source server will be deleted, and it will no longer be possible to return to the previous lifecycle state.

[Cutover complete]

This state indicates that the cutover for the server is complete, data replication is disconnected, and migration is finished.
In this state, data replication is disconnected, all replicated data is discarded, and all AWS resources used for data replication to the source server are deleted.
Selecting "Mark as archived" under "Actions" for the target source server in the AWS MGN AWS Management Console will enable you to filter and view servers that have completed the cutover and those that have not.

References:
Tech Blog with curated related content
AWS Documentation (Application Migration Service User Guide)
How to Use the New AWS Application Migration Service for Lift-and-Shift Migrations | AWS News Blog
AWS Application Migration Service Architecture - YouTube

Summary

In this article, I have summarized the architecture and lifecycle relationship of the AWS Application Migration Service (AWS MGN), including its differences from the AWS Server Migration Service (AWS SMS).

AWS MGN, when used with a compatible agent, allows for the simple lift-and-shift migration of VMs, along with their applications, to AWS without changing hypervisors like VMware vSphere or Microsoft Hyper-V, simply by allowing outbound HTTPS and TCP 1500 port traffic.
Understanding the relationship between lifecycle states and architectural steps in AWS MGN enables the execution of complex migration tasks simply and safely, while being aware of the content of the automated migration flow.

I hope to continue exploring services that support migration to AWS, similar to this one, whenever the opportunity arises.

Written by Hidekazu Konishi