hidekazu-konishi.com

AWS History and Timeline regarding AWS Key Management Service - Overview, Functions, Features, Summary of Updates, and Introduction to KMS

First Published: 2023-06-19
Last Updated: 2023-06-19

This is the sixth installment in the series that I started with the "AWS History and Timeline - Almost All AWS Services List, Announcements, General Availability(GA)", where I extract features from the history and timeline of AWS services (I've previously written about Amazon S3, AWS Systems Manager, Amazon Route 53, Amazon EventBridge).

This time, I have created a historical timeline for AWS Key Management Service (AWS KMS), a service that provides advanced encryption features across AWS.
Just like before, I am summarizing the main features while following the birth of AWS KMS and tracking its feature additions and updates as a "Current Overview, Functions, Features of AWS KMS".
I hope these will provide clues as to what has remained the same and what has changed, in addition to the features and concepts of each AWS service.

Background and Method of Creating AWS KMS Historical Timeline

This time, the reason for creating a historical timeline of AWS KMS is that since AWS KMS was launched in 2014, it has been widely providing encryption functions, which are at the core of security, across all AWS services. Therefore, I decided to organize the information of AWS KMS with the following approaches.
  • Tracking the history of AWS KMS and organizing the transition of updates
  • Summarizing the feature list and characteristics of AWS KMS
This timeline primarily references the following blogs and document history regarding AWS KMS.
There may be slight variations in the dates on the timeline due to differences in the timing of announcements or article postings in the references used.
The content posted is limited to major features related to the current AWS KMS and necessary for the feature list and overview description.
In other words, please note that the items on this timeline are not all updates to AWS KMS features, but are representative updates that I have picked out.

AWS KMS Historical Timeline (Updates from November 12, 2014)

Now, from here is the timeline regarding the features of AWS KMS. The history of AWS KMS is about 7 years at the time of writing this article.

* You can sort the table by clicking on the column name.
Date Summary
2014-11-12 AWS Key Management Service (AWS KMS) is announced as General Availability (GA).
Amazon S3, Amazon EBS, Amazon Redshift support encryption by AWS KMS.
2014-11-24 Amazon Elastic Transcoder supports encryption by AWS KMS.
2015-01-06 Amazon Relational Database Service (Amazon RDS) supports encryption by AWS KMS.
2015-01-28 Amazon WorkMail supports encryption by AWS KMS.
2015-10-01 Amazon Simple Email Service (Amazon SES) supports encryption by AWS KMS.
2015-10-01 AWS CloudTrail supports encryption by AWS KMS.
2015-10-02 Amazon WorkSpaces supports encryption by AWS KMS.
2015-10-15 It becomes possible to schedule the deletion of AWS KMS keys after a waiting period of 7 to 30 days.
2015-12-07 Amazon Aurora supports encryption by AWS KMS.
2015-12-15 Amazon EC2's EBS boot volume supports encryption by AWS KMS.
2016-08-11 It becomes possible to import your own key material into symmetric KMS keys.
2016-08-31 Metrics about AWS KMS are added to Amazon CloudWatch.
2016-12-01 EC2 Systems Manager (later AWS Systems Manager)'s Parameter Store supports encryption by AWS KMS with SecureString.
2017-02-15 Tagging of KMS keys is supported.
2017-04-28 Amazon Simple Queue Service (SQS) supports encryption by AWS KMS.
2017-07-06 Amazon Kinesis Data Streams supports encryption by AWS KMS.
2017-08-14 Amazon Elastic File System (EFS) supports encryption by AWS KMS.
2017-12-07 Amazon Elasticsearch Service (later Amazon OpenSearch Service) supports encryption by AWS KMS.
2017-12-08 Amazon CloudWatch Logs supports encryption by AWS KMS.
2018-01-17 Amazon SageMaker's training and hosting storage volumes now support encryption by AWS KMS.
2018-01-22 AWS KMS can now be privately accessed via the Amazon VPC interface endpoint using AWS PrivateLink.
2018-02-08 Amazon DynamoDB now supports encryption by AWS managed keys.
2018-04-05 AWS Secrets Manager now supports encryption by AWS KMS.
2018-04-25 AWS X-Ray now supports encryption by AWS KMS.
2018-08-01 AWS Storage Gateway now supports encryption by AWS KMS.
2018-08-09 Amazon DynamoDB Accelerator (DAX) now supports encryption by AWS managed keys.
2018-09-04 AWS Glue ETL jobs and development endpoints now support encryption by AWS KMS.
2018-11-07 Amazon SageMaker Batch Transform now supports encryption by AWS KMS.
2018-11-15 Amazon SNS now supports encryption by AWS KMS.
2018-11-15 Amazon DynamoDB now implements encryption for all existing tables using AWS-owned keys.
2018-11-19 AWS Server Migration Service now supports AWS KMS encryption of on-premises server volume AMIs.
2018-11-26 AWS KMS now allows generation, storage, and use of KMS keys in a custom key store using an AWS CloudHSM cluster.
2019-03-28 Amazon Comprehend now supports encryption by AWS KMS.
2019-04-04 AWS Systems Manager Session Manager now supports encryption by AWS KMS.
2019-07-22 Amazon MQ supports encryption by AWS KMS.
2019-08-29 Amazon ElastiCache for Redis supports encryption by AWS KMS.
2019-09-24 Amazon Transcribe supports encryption by AWS KMS.
2019-11-04 AWS KMS supports Hybrid Post-Quantum TLS (hybrid post-quantum TLS) as a hybrid post-quantum key exchange option for the Transport Layer Security (TLS) network encryption protocol used to connect to AWS KMS API endpoints.
2019-11-15 Key policies for AWS-managed keys can now be displayed in the AWS KMS console.
2019-11-25 AWS KMS supports asymmetric keys and asymmetric data key pairs for RSA, Elliptic Curve (ECC).
2019-11-26 Amazon Kinesis Data Firehose supports encryption by AWS KMS.
2019-11-26 Amazon DynamoDB supports encryption by customer-managed keys.
2020-02-24 Amazon FSx for Lustre supports encryption by AWS KMS.
2020-03-05 Amazon Elastic Kubernetes Service (EKS) supports AWS KMS encryption of Kubernetes secrets.
2020-07-07 Amazon EMR supports AWS KMS encryption of log files.
2020-07-14 AWS KMS supports VPC endpoint policies.
2020-07-29 Amazon Elastic Container Registry (ECR) supports encryption by AWS KMS.
2020-11-06 Amazon DynamoDB Global Tables support encryption by customer-managed keys.
2020-11-17 Amazon SageMaker Studio supports encryption by AWS KMS.
2020-11-19 Amazon Textract now supports encryption via AWS KMS.
2020-12-17 Amazon Route 53's Domain Name System Security Extensions (DNSSEC) signing with AWS KMS's asymmetric customer-managed keys is announced.
2020-12-17 AWS KMS supports attribute-based access control (ABAC) in key policies.
2021-01-27 AWS KMS encryption can be applied to existing domains in Amazon Elasticsearch Service (later Amazon OpenSearch Service).
2021-03-01 AWS KMS encryption of Kubernetes secrets can be applied to existing clusters in Amazon Elastic Kubernetes Service (EKS).
2021-04-02 Amazon Comprehend now supports encryption via AWS KMS.
2021-05-05 Amazon CodeGuru Reviewer supports encryption with customer-managed keys.
2021-06-01 Amazon Keyspaces supports encryption with customer-managed keys.
2021-06-10 Amazon Managed Blockchain supports encryption with customer-managed keys.
2021-06-16 Supports multi-region keys that can replicate a KMS key created in one region to another region to perform encryption operations.
2021-06-25 Amazon Rekognition Custom Labels now supports encryption via AWS KMS.
2021-07-23 Amazon QLDB supports encryption with customer-managed keys.
2021-07-28 AWS Control Tower supports AWS KMS encryption for AWS services it deploys and Amazon S3 data.
2021-08-30 The term previously used, "Customer Master Key (CMK)", is replaced with "AWS KMS key" and "KMS key".
* No functional changes.
2021-10-24 Amazon SageMaker Data Wrangler now supports encryption via AWS KMS.
2021-11-05 Amazon Translate now supports encryption via AWS KMS.
2022-01-31 Amazon SageMaker JumpStart now supports encryption via AWS KMS.
2022-03-16 AWS KMS supports the latest hybrid post-quantum TLS.
2022-04-19 AWS KMS supports HMAC KMS keys that can generate and verify hash-based message authentication codes (HMAC).
2022-07-28 Amazon SageMaker Canvas supports encryption with customer-managed keys.
2022-08-25 Amazon CloudFront's Origin Access Control (OAC) now allows the use of SSE-KMS encrypted data at Amazon S3 origins.
2022-11-29 AWS KMS now allows generation, storage, and use of KMS keys in a custom key store using a External Key Store outside of AWS.

At the initial stage when AWS KMS was announced as GA, interfaces related to its core internal mechanisms and encryption operations were already established. Therefore, updates to AWS KMS itself are infrequent, such as the addition of KMS key specifications it can handle, or those related to access permissions. On the other hand, updates integrating AWS KMS with other AWS services continue to this day.
The updates regarding the integration of AWS KMS and AWS services mentioned above are only a part, and many new AWS services are compatible with encryption by AWS KMS from the beginning.

Current Overview, Functions, Features of AWS KMS

From here, I introduce the current list of AWS KMS features and overview.
AWS Key Management Service (AWS KMS) is a fully managed service that uses a FIPS 140-2 validated hardware security module (HSM) to generate, store, audit, and centrally manage keys used in cryptographic operations, providing a solution for integrated data encryption and decryption across AWS services and cryptographic operations such as data encryption and digital signatures in proprietary applications.

The terms used in AWS KMS at the time of writing this article are referred to by different names than when AWS KMS was first introduced.
Specifically, the name "Customer Master Keys (CMK)" has been replaced with "KMS keys (AWS KMS keys)".
Therefore, in this article, I will refer to the previously used name as the old name and also describe the correspondence with the currently used name.

Key Material

Key Material is a bit string used in cryptographic algorithms to encrypt and decrypt data, which is referred to in the following KMS key.
There are multiple origins of key material (sources that identify the generation method of key material), and for symmetric encryption KMS keys, key material generated by AWS KMS (AWS_KMS), key material imported by oneself (EXTERNAL), and key material generated by AWS CloudHSM clusters in custom key stores (AWS_CLOUDHSM). If AWS KMS key material is used for symmetric encryption KMS keys, automatic rotation of key material can be set.
Also, each KMS key has a unique key material by default, but you can create multi-region keys with the same key material.

Import of Key Material

The import of key material can be utilized when there is a security requirement to generate key material independently outside of AWS, but the user needs to bear responsibility for the availability and durability of the key material.
As will be described later, only symmetric KMS keys (including multi-region keys) can import and use your own key material.
The origin of the imported key material will be EXTERNAL, and the key rotation needs to be performed manually by the user.

Custom Key Stores

Custom Key Stores are a method of generating key material in an AWS CloudHSM cluster associated with the origin of KMS keys.
When you use a KMS key with a custom key store set as its origin, the key material generation and cryptographic operations will be performed in the HSM inside the AWS CloudHSM cluster.
Custom Key Stores can be used when there is a security requirement to generate and perform cryptographic operations with non-extractable key material within the AWS CloudHSM cluster, without storing key material in a shared environment.

KMS Keys (AWS KMS keys) | Formerly: Customer Master Key (CMK)

KMS keys are logical representations of encryption keys used for encryption and decryption in AWS KMS, equipped with metadata such as KeySpec(Key Specification), KeyUsage, Origin, Enabled, MultiRegion, KeyManager, EncryptionAlgorithms, SigningAlgorithms, Description, CreationDate, CustomerMasterKeySpec, KeyState, KeyId, ARN, AWSAccountId.
KMS keys perform encryption and decryption operations referring to the key material (by default, the key material generated by AWS KMS).

In the case of the subsequent symmetric KMS keys, you can generate data keys used for general encryption and decryption of data without using API operations such as Encrypt and Decrypt outside of AWS KMS (note 1).
(Note 1) In AWS KMS, you either use API operations like Encrypt, Decrypt, or use data keys generated from the KMS key to actually encrypt and decrypt the data. However, in the case where encryption operations are performed within fully managed AWS services, the encryption and decryption flow using data keys may be omitted in the expression of execution, such as "Encrypting with a customer-managed key (a KMS key created by the user)".

KMS keys can be classified into types such as "Customer Managed Keys (Customer-managed keys)", "AWS Managed Keys", and "AWS Owned Keys" depending on the management style. More details will be discussed later.

The metadata of KMS keys mainly includes the following.

KeySpec, KeyUsage

The KMS keys that can be created in AWS KMS can be broadly divided into symmetric keys that use the same key for encryption and decryption, employing a symmetric encryption algorithm, and asymmetric keys that use a pair of different keys for encryption and decryption, utilizing a public key encryption algorithm.
The symmetric and asymmetric keys are classified on the AWS KMS management console as "key types" (however, there is no field called key type in the metadata of the KMS key).

This key type indicating symmetric or asymmetric keys, and the KeySpec and KeyUsage among the metadata of the KMS keys, are interrelated, and the key type, key spec, and key usage are determined, and you can classify KMS keys into the following types according to KeySpec.


Key Type KMS Key Type Classified by KeySpec KeySpec KeyUsage
Symmetric Symmetric KMS Key SYMMETRIC_DEFAULT ENCRYPT_DECRYPT (Encryption and Decryption)
Symmetric HMAC KMS Key HMAC_224, HMAC_256, HMAC_384, HMAC_512 GENERATE_VERIFY_MAC (Generate and Verify HMAC)
Asymmetric Asymmetric KMS Key (RSA Key Pair) RSA_2048, RSA_3072, RSA_4096 ENCRYPT_DECRYPT (Encryption and Decryption)
SIGN_VERIFY (Signing and Verification)
Asymmetric Asymmetric KMS Key (ECC Key Pair) ECC_NIST_P256, ECC_NIST_P384, ECC_NIST_P521, ECC_SECG_P256K1 SIGN_VERIFY (Signing and Verification)
Asymmetric Asymmetric KMS Key (SM2 Key Pair)
* Only in the China region		 SM2
* Only in the China region		 ENCRYPT_DECRYPT (Encryption and Decryption)
SIGN_VERIFY (Signing and Verification)

Origin

This is a field that indicates the origin of the key material that the KMS key is using. The values of this field are as follows:

AWS_KMS: Key material generated by AWS KMS  
EXTERNAL: Key material imported independently  
AWS_CLOUDHSM: Key material generation by the AWS CloudHSM cluster of the custom key store

On the AWS Management Console, the field is also labeled as "Origin".

Enabled (Status)

This is a field that indicates whether the target KMS key is enabled. The values of this field are as follows:

True: The target KMS key is enabled  
False: The target KMS key is disabled

On the AWS Management Console, the field is labeled as "Status", and the values are expressed as "Enabled" and "Disabled".

MultiRegion (Regionality)

This is a field that indicates whether it is a multi-region key. Multi-region keys are a feature that enables encryption operations by replica keys by replicating customer-managed keys created in one region to other regions.
Multi-region keys are ultimately an inter-region replica function of KMS keys created in one region and are not global resources.
In addition, multi-region keys can only be specified at creation, and existing single-region keys cannot be changed to multi-region keys.
Multi-region keys can be created for symmetric KMS keys other than custom key stores, HMAC KMS keys, and asymmetric KMS keys.

The values of this field are as follows:

True: It is a multi-region key  
False: It is not a multi-region key (it is a single-region key)

On the AWS Management Console, the field is labeled as "Regionality", and the values are expressed as "Multi-Region" and "Single-Region".

KeyManager

This is a field that indicates whether the KMS key is a "Customer Managed Key" or an "AWS Managed Key". The values of this field are as follows:

CUSTOMER: Customer Managed Key  
AWS: AWS Managed Key

On the AWS Management Console, customer managed keys and AWS managed keys are displayed on separate screens and distinguished.

EncryptionAlgorithms

Encryption algorithms used in KeySpec when ENCRYPT_DECRYPT (encryption and decryption) is specified in KeyUsage by the KMS key are displayed.
On the AWS Management Console, the field is labeled as "Encryption Algorithm", and the encryption algorithms used in KeySpec are displayed.

SigningAlgorithms

Signature algorithms used in KeySpec when GENERATE_VERIFY_MAC (generate and verify HMAC) or SIGN_VERIFY (sign and verify) is specified in KeyUsage by the KMS key are displayed.
On the AWS Management Console, the field is labeled as "MAC Algorithm" or "Signature Algorithm", and the signature algorithms used in KeySpec are displayed.

Description

This is a field where you can describe the KMS key. This field can be edited even after the AWS KMS key is created.
On the AWS Management Console, the field is also labeled as "Description".

CreationDate

This is a field where the date and time the KMS key was created is recorded.
On the AWS Management Console, the field is also labeled as "CreationDate".

KeyId

This is a field of a value that uniquely identifies the created KMS key.

ARN

This is a field where the ARN (Amazon Resource Name), which uniquely identifies the created KMS key as an AWS resource, is displayed.
It is expressed as follows using the aforementioned KeyId:

#Format
arn:aws:kms:[AWS Region Identifier]:[AWS Account ID]:key/[KeyId]

#Example
arn:aws:kms:ap-northeast-1:123456789012:key/a2345678-b234-c234-d234-e234567890fg

CustomerMasterKeySpec: *Deprecated Field

As of the time of writing this article, CustomerMasterKeySpec is considered a deprecated field, and it is recommended to use the KeySpec field instead.
At the time of writing this article, it seems that the same value as KeySpec is included.

KeyRotationEnabled (Key Rotation Activation): *Referenced by KeyId

This is a field that indicates whether key rotation is enabled or disabled, managed in connection with the KeyId of the KMS key.
The values of the field are as follows.

True: Key rotation is enabled  
False: Key rotation is disabled

AliasName: *Referenced by KeyId

This is a field for entering an alias of the KMS key, managed in connection with the KeyId of the KMS key.
In the case of a customer-managed key, the actual value of the AliasName field is the string entered and displayed as an alias by the user in the AWS Management Console, prefixed with alias/.
Also, in the case of AWS-managed keys, the actual value of the AliasName field is the string specified and displayed for each AWS service, prefixed with alias/.

#* In the case of customer-managed keys  
#Format  
alias/[String entered by the user]
#Example
alias/MyKmsKey

#* In the case of AWS-managed keys  
#Format  
alias/[String specified for each AWS service]
#Example
alias/aws/s3
alias/aws/lambda

Policy (Key Policy): *Referenced by KeyId

This is a field for specifying a policy to control access to the KMS key, managed in connection with the KeyId of the KMS key.
In the key policy, you specify the AWS accounts, IAM users, IAM roles, AWS resources, conditions, etc. that can use the KMS key, and set the actions that can be performed on the KMS key.
The policy syntax used in the KMS key policy uses the same JSON policy document structure as IAM.

Next, an example of a key policy for a KMS key is provided.

{
    "Id": "key-consolepolicy-sample",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/admin"
            },
            "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/admin"
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:DescribeKey",
                "kms:GetPublicKey"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow attachment of persistent resources",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/admin"
            },
            "Action": [
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:RevokeGrant"
            ],
            "Resource": "*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}

Customer Managed Keys | Formerly: Customer Managed CMK

Customer Managed Keys are a type of KMS key management form that users can create, own, and manage within their AWS account.
While you can create a Customer Managed Key according to your needs using AWS KMS, some AWS services allow you to choose between using a Customer Managed Key or an AWS Managed Key when using AWS KMS. If you choose to use a Customer Managed Key, you will have to create a Customer Managed Key.
You can choose whether or not to perform key rotation for a Customer Managed Key, or to do so once a year.
* The rotation schedule period is once a year (approximately 365 days), and this cannot be changed.

The fields of the metadata that a KMS key has that you can create, edit, and refer to with a Customer Managed Key include the following:

  • Main metadata that can be specified at the time of creation with a Customer Managed Key
KeyUsage, KeySpec, Origin, MultiRegion, Description, KeyRotationEnabled, AliasName, Policy (Key Policy)
  • Main metadata that can be added/edited after creation with a Customer Managed Key
Description, KeyRotationEnabled, AliasName, Policy (Key Policy)
  • Main metadata that can be referred to with a Customer Managed Key
KeyUsage, KeySpec, Origin, Enabled (Status), MultiRegion, KeyManager, EncryptionAlgorithms, SigningAlgorithms, Description, CreationDate, ARN, KeyId, KeyRotationEnabled, AliasName, Policy (Key Policy)

AWS Managed Keys | Formerly: AWS Managed CMK

AWS Managed Keys are a type of KMS key management form that AWS services create on behalf of users within their AWS account when using AWS KMS.
AWS services may automatically create an AWS Managed Key. However, some AWS services allow you to choose between using a Customer Managed Key or an AWS Managed Key when using AWS KMS. If you choose to use an AWS Managed Key, an AWS Managed Key will be created.
The key rotation of AWS Managed Keys is executed once a year.
* As of May 2022, the rotation schedule for AWS Managed Keys has been changed from every three years (approximately 1,095 days) to every year (approximately 365 days).

Although you cannot specify or edit the metadata that a KMS key has in an AWS Managed Key, you can primarily refer to the following:

  • Main metadata that can be referred to with an AWS Managed Key (creation-time specification, post-creation editing not possible)
KeyUsage, KeySpec, Origin, Enabled (Status), MultiRegion, Description, CreationDate, ARN, KeyId, Policy (Key Policy)

AWS Owned Keys | Formerly: AWS Managed CMK

AWS Owned Keys are a type of KMS key that AWS owns and manages and does not create within an individual AWS account for use across multiple AWS accounts.
AWS Owned Keys are not created within an AWS account, so they are not displayed in the AWS KMS console. Some AWS services allow you to choose whether to use a customer-managed key or an AWS owned key when specifying encryption.
The key rotation of AWS owned keys varies depending on the AWS service.

Encryption Operations

Encryption operations are actions taken on AWS KMS via APIs such as the AWS CLI, AWS SDK.
The relationship between the main encryption operations and their KeyUsage is as follows:

Operation KeyUsage Overview
Decrypt ENCRYPT_DECRYPT Decrypt using symmetric keys or asymmetric keys
Encrypt ENCRYPT_DECRYPT Encrypt using symmetric keys or asymmetric keys
ReEncrypt ENCRYPT_DECRYPT Re-encrypt using symmetric keys or asymmetric keys
GenerateDataKey ENCRYPT_DECRYPT Generate a data key with a symmetric key
GenerateDataKeyWithoutPlaintext ENCRYPT_DECRYPT Generate a data key with a symmetric key
* The plaintext data key is not included in the response.
GenerateDataKeyPair ENCRYPT_DECRYPT Generate an asymmetric key pair protected with symmetric key encryption
GenerateDataKeyPairWithoutPlaintext ENCRYPT_DECRYPT Generate an asymmetric key pair protected with symmetric key encryption
* The plaintext private key is not included in the response.
GenerateMac GENERATE_VERIFY_MAC Generate a HMAC key
VerifyMac GENERATE_VERIFY_MAC Verify a HMAC key
Sign SIGN_VERIFY Sign using an asymmetric key
Verify SIGN_VERIFY Verify a signature using an asymmetric key
GenerateRandom N/A Return a cryptographically secure random byte string

Data Key

The data key is an encryption key generated from a symmetric KMS key outside AWS KMS, which is used for general encryption and decryption of data without using the Encrypt and Decrypt API operations.
However, as will be mentioned later, the data key itself is protected by encrypting and decrypting using API operations with the KMS key.

The data key can be generated from the specified symmetric KMS key by calling the GenerateDataKey or GenerateDataKeyWithoutPlaintext operation with APIs such as AWS CLI or AWS SDK, and the following parameters are returned at creation.

KeyId: The KeyId of the symmetric KMS key used to generate the data key  
Plaintext: The plaintext data key used for data encryption, not returned in GenerateDataKeyWithoutPlaintext  
CiphertextBlob: Encrypted data key that can be used for storage with the data to be encrypted

An example of using a data key with the GenerateDataKey operation would be as follows.

Example of Encryption Flow Using a Data Key

  1. Generate a data key by specifying the symmetric KMS key's KeyId and calling the GenerateDataKey operation with an API such as AWS CLI or AWS SDK.
  2. Encrypt the data to be encrypted without using the Encrypt API operation with the data key specified in the Plaintext returned in the response from GenerateDataKey.
  3. Do not save the data key specified in the Plaintext anywhere and discard it from memory.
  4. Store the encrypted data together with the CiphertextBlob returned in the response from GenerateDataKey.

Example of Decryption Flow Using a Data Key

  1. Obtain the data key by decrypting the CiphertextBlob stored with the encrypted data by specifying the symmetric KMS key's KeyId and calling the Decrypt operation with an API such as AWS CLI or AWS SDK.
  2. Decrypt the encrypted data without using the Decrypt API operation using the decrypted data key.

* In AWS KMS, you either use API operations such as Encrypt, Decrypt, or encrypt and decrypt the actual data using a data key generated from a KMS key. However, in cases where encryption operations are being executed internally in fully managed AWS services, the encryption and decryption flow using a data key might be omitted in the expression of the execution statement, such as "Encrypt using a customer-managed key (a KMS key created by the user)".

Data Key Pair

The data key pair is an encryption key pair generated from an asymmetric KMS key outside AWS KMS, which is used for general encryption and decryption of data without using the Encrypt and Decrypt API operations.

The data key pair can be generated from the specified asymmetric KMS key by calling the.Today in a blog article I explained about Data Key and Data Key Pair, and how to use them for general purpose encryption and decryption of data without the use of Encrypt or Decrypt API operations.

Data Key

A Data Key is an encryption key generated from a symmetric KMS key to be used for generic encryption and decryption of data, without using the Encrypt or Decrypt API operations in AWS KMS. However, as we'll discuss later, the Data Key itself is protected using the KMS key and API operations for encryption and decryption, known as Envelope Encryption.

Data Keys can be generated from a specified symmetric KMS key by calling the GenerateDataKey or GenerateDataKeyWithoutPlaintext operation through the AWS CLI, AWS SDK, or other APIs. Upon creation, the following parameters are returned:

KeyId: The KeyId of the symmetric KMS key used to generate the Data Key
Plaintext: The plaintext Data Key to be used for data encryption. It is not returned in GenerateDataKeyWithoutPlaintext.
CiphertextBlob: The encrypted Data Key, which can be stored along with the data to be encrypted

Below is an example of how to use a Data Key with the GenerateDataKey operation:

Example of Encryption Flow Using Data Key

  1. Generate a Data Key by specifying the KeyId of a symmetric KMS key and invoking the GenerateDataKey operation through the AWS CLI, AWS SDK, or other APIs.
  2. Use the Data Key specified in the Plaintext returned by the GenerateDataKey operation to encrypt the data without using the Encrypt API operation.
  3. Do not save the Data Key specified in Plaintext anywhere, and discard it from memory.
  4. Store the encrypted data along with the CiphertextBlob returned in the response of the GenerateDataKey operation.

Example of Decryption Flow Using Data Key

  1. Retrieve the Data Key by decrypting the CiphertextBlob that was stored with the encrypted data, specifying the KeyId of a symmetric KMS key, and invoking the Decrypt operation through the AWS CLI, AWS SDK, or other APIs.
  2. Decrypt the encrypted data using the decrypted Data Key, without using the Decrypt API operation.

* With AWS KMS, you can either use API operations like Encrypt and Decrypt, or use a Data Key generated from a KMS key to actually encrypt and decrypt data. However, in cases where encryption operations are performed internally in fully managed AWS services, the expression in the text, such as "Encrypt using a customer-managed key (a KMS key created by the user)", may omit the encryption and decryption flow using a Data Key.

Data Key Pair

A data key pair is a cryptographic key pair generated from an asymmetric KMS key, used for generic data encryption and decryption without using the Encrypt and Decrypt API operations outside of AWS KMS.

A data key pair can be generated from a specified asymmetric KMS key by invoking the GenerateDataKeyPair or GenerateDataKeyPairWithoutPlaintext operations via AWS CLI, AWS SDK, or other APIs. When created, the following parameters are returned:

KeyId: The KeyId of the asymmetric KMS key used to generate the data key pair
KeyPairSpec: The KeySpec of the asymmetric KMS key used to generate the data key pair
PrivateKeyCiphertextBlob: Encrypted secret key that can be used for storage
PrivateKeyPlaintext: Plaintext secret key used for data decryption * Not returned in GenerateDataKeyPairWithoutPlaintext
PublicKey: Plaintext public key used for data encryption

Example usage of a data key pair using the GenerateDataKeyPair or GenerateDataKeyPairWithoutPlaintext operations is as follows:

Preparation of Secret Key and Public Key

  1. Generate a data key pair with the GenerateDataKeyPair or GenerateDataKeyPairWithoutPlaintext operation by specifying the KeyId and KeySpec of an asymmetric KMS key via the AWS CLI, AWS SDK, or other APIs.
  2. Pass the PublicKey from the response returned by GenerateDataKeyPair or GenerateDataKeyPairWithoutPlaintext to the entity (function, node, user, etc.) that encrypts the data.
  3. Store the PrivateKeyCiphertextBlob from the response returned by GenerateDataKeyPair or GenerateDataKeyPairWithoutPlaintext in the entity (function, node, user, etc.) that decrypts the encrypted data.

Example of Encryption Flow using Data Key Pair

  1. Encrypt the target data without using the Encrypt operation, using the public key specified in the PublicKey at the entity (function, node, user, etc.) that encrypts the data.
  2. Pass the encrypted data to the entity (function, node, user, etc.) that decrypts it.

Example of Decryption Flow using Data Key Pair

  1. Receive the encrypted data at the entity (function, node, user, etc.) that decrypts it.
  2. Obtain the secret key by decrypting the PrivateKeyCiphertextBlob at the entity (function, node, user, etc.) that decrypts the encrypted data, using the Decrypt operation by specifying the KeyId of an asymmetric KMS key via the AWS CLI, AWS SDK, or other APIs.
  3. Decrypt the encrypted data without using the Decrypt API operation using the decrypted secret key.

Envelope Encryption

Envelope encryption is a method where plaintext data is encrypted with a data key, and then the data key used for encryption is encrypted with another key.
Specifically, the usage is similar to the "Example of encryption flow using a data key" explained in the previous section, where the Plaintext data key is discarded from memory after use for encryption, and the encrypted data key, CiphertextBlob, which is encrypted with a KMS key, is stored together with the target data for encryption.

The benefits of envelope encryption include the following:

  • Protects the data key itself through encryption and simplifies key management
  • By combining symmetric keys, which allow fast and smaller encrypted data but are difficult to control access for key exchange, with asymmetric keys, which have slower and larger encrypted data but easier access control for key exchange, it enables effective key management that takes advantage of the benefits of both

Hybrid Post-Quantum TLS

Currently, the encryption communication protocol used to connect to the AWS KMS API endpoint, Transport Layer Security (TLS), uses classical key exchange algorithms such as Finite Field Diffie-Hellman Ephemeral (FFDHE) and Elliptic Curve Diffie-Hellman Ephemeral (ECDHE).

There is a potential risk that the encryption of TLS by these traditional algorithms could eventually be deciphered by large-scale quantum computers. Hybrid post-quantum TLS is a measure to avoid this risk.

Hybrid post-quantum TLS is a hybrid cryptographic suite of TLS that combines classical ECDHE with post-quantum cryptographic key encapsulation mechanisms (KEM) such as Kyber, BIKE, and SIKE, which NIST has begun to standardize.

AWS KMS allows the use of hybrid post-quantum TLS through combinations like ECDHE with Kyber, utilizing the open source s2n-tls.

AWS KMS Key Management and Data Key Generation Internals

The internal management of KMS keys and data key generation in AWS KMS mainly consists of the following elements.

  • Domain Key
    A key that is held in HSM's memory for version control and encryption of KMS keys within a region.
    Rotated daily.

  • HSM Backing Key(HBK)
    The actual key of the KMS key created from key material generated in HSM or imported key material, and held in HSM's memory. (KMS key deployed on HSM's memory)
    The HSM Backing Key(HBK) is encrypted with the Domain Key for permanent retention and is saved in the high durability storage within AWS KMS as the Exported Key Token(EKT) described below.
    Rotated annually.

  • Exported Key Token(EKT)
    The encrypted HSM Backing Key(HBK) with the Domain Key deployed on the memory of HSM and stored in the high durability storage within AWS KMS. (KMS key encrypted and stored in high durability storage)
    When using the KMS key, it is decrypted and the HSM Backing Key(HBK) is placed in the memory on HSM.
    When the key material expires, the Exported Key Token(EKT) is deleted.

  • Customer Data Key(CDK)
    A symmetric or asymmetric key that can be used outside AWS KMS that is generated from the HSM Backing Key(HBK) and exported from HSM.
    It corresponds to what is referred to as a "data key" in plaintext.

  • Derived Encryption Key
    A key that is used to encrypt data or Customer Data Key and is held in HSM's memory.
    Derived for each encryption from the HSM Backing Key(HBK).
    Used once per encryption and regenerated at the time of decryption.

  • Ciphertext(CT)
    The encrypted text of the Customer Data Key(CDK) with the Derived Encryption Key.
    It corresponds to the encrypted data key mentioned above.

Overview Diagram of AWS KMS Key Management and Data Key Usage

The following is a summary of the overview diagram regarding the management of KMS keys in AWS KMS and the use of data keys in symmetric KMS keys.
In AWS KMS, the diagram mainly illustrates the concepts of internal KMS key management and data key generation, while in the Client, it illustrates the flow of concepts of encryption and decryption using a data key in a symmetric KMS key.

Overview Diagram of AWS KMS Key Management and Data Key Usage
Overview Diagram of AWS KMS Key Management and Data Key Usage

[Reference Materials]
AWS Documentation(AWS Key Management Service)
AWS Documentation(AWS KMS Cryptographic Details)
Round 2 post-quantum TLS is now supported in AWS KMS

Summary

This time, I have created a timeline of AWS KMS and looked at the feature list and overview of AWS KMS.

AWS KMS enables encryption and decryption of data with KMS keys integrated across AWS services now and in the future, while also providing secure encryption and decryption operations with data key generation and envelope encryption outside AWS KMS.
In particular, in recent years, the number of KeySpec and KeyUsage that can be handled by KMS keys has been increasing, including HMAC generation and verification with HMAC KMS keys, encryption/decryption/signature and verification with asymmetric KMS keys (RSA), and signature and verification with asymmetric KMS keys (ECC), expanding its applications.
In addition, there have been updates such as multi-region keys and access control through key policies, enabling more flexible usage while maintaining security.

I would like to continue watching the trends of what kind of features AWS KMS will continue to provide in the future.

In addition, there is also a timeline of the entire AWS services including AWS KMS, so please have a look if you are interested.

AWS History and Timeline - Almost All AWS Services List, Announcements, General Availability(GA)

Written by Hidekazu Konishi


Copyright © Hidekazu Konishi ( hidekazu-konishi.com ) All Rights Reserved.