hidekazu-konishi.com
How to Add an Approval Flow to AWS Step Functions Workflow (AWS Systems Manager Automation and Amazon EventBridge Edition)
First Published:
Last Updated:
In the previous article, I returned the approval result to the AWS Step Functions state machine via an AWS Lambda function from AWS Systems Manager Automation executed by an AWS Lambda function.
This time, I'd like to try a method of detecting the approval result of AWS Systems Manager Automation with an Amazon EventBridge rule and adding an approval flow using AWS Systems Manager Automation to AWS Step Functions.
This article, like the aforementioned one, was created with the following motivations and intentions:
In recent years, the rapid advancement of AI technology has made it possible to replace or strongly support traditionally manual approval processes performed by humans with generative AI. However, final judgment by humans with specialized knowledge and authority remains important.
Therefore, with a view to incorporating generative AI into approval flows in the future, I have prototyped an approval flow system using AWS Step Functions with AWS services. The main objectives of this prototype are as follows:
Note that this demo is built using an AWS CloudFormation template. The reasons for not using more advanced IaC tools like AWS CDK or AWS SAM include the following:
- By systematizing the approval flow through APIs, decision-making processes can be flexibly switched between humans and generative AI
- Initially, approvals are performed by humans, and can be gradually transitioned to AI when the capabilities of generative AI are deemed sufficient
- If there are concerns about the judgment of generative AI or if final confirmation is needed, humans can intervene in the approval process
- Multi-stage approval flows combining humans and generative AI enable decision-making with higher accuracy
For these reasons, I believe that using a CloudFormation template provides a demo environment that more readers can easily understand and implement.
- Reproducibility and portability: By making it self-contained with a single CloudFormation template, it ensures that anyone can obtain the same results regardless of their environment. Using CDK or SAM could potentially reduce reproducibility due to version differences or dependency issues.
- Lower learning barrier: CloudFormation is a familiar technology for many AWS users. Using CDK or SAM might require knowledge of additional tools or programming languages, which could be a barrier for readers.
- Direct understanding of AWS resources: CloudFormation templates directly define AWS resources. This makes it easier to understand the detailed settings and behavior of AWS services, increasing educational value.
- Ease of debugging: Being a single template file, identifying and fixing errors is relatively easy. This facilitates troubleshooting for readers when implementing in their own environments.
- Simplification of version control and maintenance: Management with a single file simplifies version control and improves long-term maintainability. The basic syntax of CloudFormation has been stable for many years, reducing the risk of future changes or deprecation.
- Rapid deployment and testing: Without additional build steps, changes to the template can be applied and tested immediately. This allows readers to quickly try it out in their own environments.
- Compatibility with AWS console: CloudFormation templates can be directly edited and applied in the AWS console. This enables quick changes and confirmation through the GUI, making it more accessible to a wider audience.
* The source code published in this article and other articles by this author was developed as part of independent research and is provided 'as is' without any warranty of operability or fitness for a particular purpose. Please use it at your own risk. The code may be modified without prior notice.
Architecture Diagram to be tested in this article
The configuration for adding an approval flow to AWS Step Functions using AWS Lambda, AWS Systems Manager Automation, and Amazon EventBridge that I will try this time is as follows:The flow is as follows:
First, in the AWS Step Functions state machine, an AWS Systems Manager Document (SSM Document) is executed as AWS Systems Manager Automation (SSM Automation) from an AWS Lambda function with
waitForTaskToken
specified.The execution step of SSM Automation confirms approval or denial with the approver via an Amazon SNS topic email notification using the approval action (
aws:approve
).An Amazon EventBridge rule detects the event of the SSM Automation execution step associated with the approval or denial decision in the approval flow, and executes an AWS Lambda function for returning results.
In the AWS Lambda function for returning results, the token of the AWS Step Functions state machine and the approval result are obtained from the contents of the SSM Automation execution step, and the result is returned to the AWS Step Functions state machine.
The advantage of using the approval action of AWS Systems Manager Automation as a component in this way is that it allows specifying the authentication of approvers and the permissions for the approval action.
When the Amazon SNS topic email notification sent by the approval action (
aws:approve
) of the SSM Document is received, the approver can log in to the AWS Management Console via the link and decide whether to approve or deny only if they have an IAM role or IAM user allowed to perform the approval action.Compared to the method of linking the execution results of parent-child related SSM Automation via an AWS Lambda function introduced in the following article, this method of detecting SSM Automation execution step events with an Amazon EventBridge rule does not require making SSM Automation in a parent-child relationship, and obtains approval results based on events, making the process simpler.
On the other hand, similar to the above article, a consideration when updating the AWS CloudFormation template is that when updating the definitions of the parent SSM Document and child SSM Document, it is necessary to change the SSM Document name and recreate it due to the specifications of AWS CloudFormation.
Example of AWS CloudFormation template and parameters
AWS CloudFormation template (Adding an approval flow to AWS Step Functions using AWS Lambda and AWS Systems Manager Automation)
Example of input parameters
EmailForNotification: sample@ho2k.com #Email address to send approval requests EventRuleForAutomationResultState: ENABLED #Setting for enabling (ENABLED) or disabling (DISABLED) Amazon EventBridge SsmApprovers: arn:aws:iam::XXXXXXXXXXXX:role/ho2k.com #IAM role or IAM user to decide approval or denial for approval requests SsmMinRequiredApprovals: 1 #Number of people required for approval. The process is approved only when the number of people specified here approve. SsmAutomationAssumeRoleName: SsmAutomationAssumeRole #Name of the AutomationAssumeRole to be created (This role will be used for executing SSM Automation) SsmDocumentForApprovalActionName: SsmParentDocumentApprovalAction #Name of the SSM Document SsmDocumentForApprovalActionVersionName: 1 #Version name of the SSM Document
Template body
File name:SfnApprovalCFnSfnWithSsmApprovalAndEventBridge.yml
AWSTemplateFormatVersion: '2010-09-09' Description: 'Add AWS Systems Manager Automation Approval Action to AWS Step Functions.' Parameters: SsmAutomationAssumeRoleName: Type: String Default: "SsmAutomationAssumeRole" SsmDocumentForApprovalActionName: Type: String Default: "SsmDocumentForApprovalAction" SsmDocumentForApprovalActionVersionName: Type: String Default: "1" SsmApprovers: Type: String Default: "arn:aws:iam::XXXXXXXXXXXX:role/ho2k.com" SsmMinRequiredApprovals: Type: String Default: "1" EmailForNotification: Type: String Default: "sample@ho2k.com" EventRuleForAutomationResultState: Type: String Default: ENABLED AllowedValues: - ENABLED - DISABLED Resources: SsmAutomationAssumeRole: Type: AWS::IAM::Role Properties: RoleName: !Sub '${SsmAutomationAssumeRoleName}-${AWS::Region}' Path: / MaxSessionDuration: 43200 AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - ssm.amazonaws.com - lambda.amazonaws.com - edgelambda.amazonaws.com - events.amazonaws.com - scheduler.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: !Sub 'IAMPolicy-AdditionalPolicyForAutomationRole-${AWS::Region}' PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - iam:PassRole Resource: - !Sub 'arn:aws:iam::${AWS::AccountId}:role/*' - Effect: Allow Action: - logs:CreateLogGroup Resource: - 'arn:aws:logs:*:*:*' - Effect: Allow Action: - logs:CreateLogStream - logs:PutLogEvents Resource: - !Sub 'arn:aws:logs:*:*:log-group:/aws/*/*:*' ManagedPolicyArns: - 'arn:aws:iam::aws:policy/service-role/AmazonSSMAutomationRole' LambdaForSsmStartAutomationExecution: Type: AWS::Lambda::Function DependsOn: - SsmAutomationAssumeRole Properties: FunctionName: LambdaForSsmStartAutomationExecution Description : 'LambdaForSsmStartAutomationExecution' Runtime: python3.9 MemorySize: 10240 Timeout: 900 Role: !GetAtt SsmAutomationAssumeRole.Arn Handler: index.lambda_handler Code: ZipFile: | import botocore import boto3 import json import os import sys region = os.environ.get('AWS_REGION') sts_client = boto3.client("sts", region_name=region) account_id = sts_client.get_caller_identity()["Account"] ssm_client = boto3.client('ssm', region_name=region) def lambda_handler(event, context): print(("Received event: " + json.dumps(event, indent=2))) try: ssm_auto_resp = ssm_client.start_automation_execution( DocumentName=event['ssm_doc_name'], Parameters={ 'AutomationAssumeRole': [event['ssm_automation_assume_role']], 'Description': [event['ssm_description']], 'Message': [event['ssm_message']], 'NotificationArn': [event['ssm_notification_arn']], 'Approvers': [event['ssm_approvers']], 'MinRequiredApprovals': [event['ssm_min_required_approvals']], 'SfnToken': [event['token']] }, Mode='Auto' ) except Exception as ex: print(f'Exception:{ex}') tb = sys.exc_info()[2] print(f'ssm_client start_automation_execution FAIL. Exception:{str(ex.with_traceback(tb))}') raise result = {} result['params'] = event.copy() return result LambdaForReceivingAutomationResult: Type: AWS::Lambda::Function DependsOn: - AutomationResultReceivedByLambdaRole Properties: FunctionName: AutomationResultReceivedByLambda Description : 'LambdaForReceivingAutomationResult' Runtime: python3.9 MemorySize: 10240 Timeout: 900 Role: !GetAtt AutomationResultReceivedByLambdaRole.Arn Handler: index.lambda_handler Code: ZipFile: | import botocore import boto3 import json import os import sys region = os.environ.get('AWS_REGION') sts_client = boto3.client("sts", region_name=region) account_id = sts_client.get_caller_identity()["Account"] sns_client = boto3.client('sns', region_name=region) ssm_client = boto3.client('ssm', region_name=region) sfn_client = boto3.client('stepfunctions', region_name=region) def lambda_handler(event, context): print(("Received event: " + json.dumps(event, indent=2))) sfn_token = '' is_approved = False try: #Get the Step Functions token from the SSM Document parameters using the ExecutionId from the Event. ssm_exe_res = ssm_client.get_automation_execution( AutomationExecutionId=event['detail']['ExecutionId'] ) print('ssm_client.get_automation_execution: ') print(ssm_exe_res) sfn_token = ssm_exe_res['AutomationExecution']['Parameters']['SfnToken'][0] print(f'sfn_token: {sfn_token}') #Get the approval result from the ExecutionId and Action of the Event. ssm_step_res = ssm_client.describe_automation_step_executions( AutomationExecutionId=event['detail']['ExecutionId'], Filters=[ { 'Key': 'Action', 'Values': ['aws:approve'] }, ] ) print('ssm_client.describe_automation_step_executions: ') print(ssm_step_res) approval_result = ssm_step_res['StepExecutions'][0]['Outputs']['ApprovalStatus'][0] print(f'approval_result:{approval_result}') #Get the approval result from the ExecutionId and step name of the Event. if approval_result == 'Approved': is_approved = True except Exception as ex: print(f'Exception:{ex}') tb = sys.exc_info()[2] print(f'ssm_client get_automation_execution, describe_automation_step_executions FAIL. Exception:{str(ex.with_traceback(tb))}') is_approved = False try: #Send task success to SFN side with the callback token. sfn_res = sfn_client.send_task_success( taskToken=sfn_token, output=json.dumps({'is_approved':is_approved}) ) except Exception as ex: print(f'Exception:{ex}') tb = sys.exc_info()[2] print(f'sfn_client send_task_success FAIL. Exception:{str(ex.with_traceback(tb))}') raise return {'is_approved':is_approved} AutomationResultReceivedByLambdaRole: Type: AWS::IAM::Role Properties: RoleName: !Sub 'IAMRole-LambdaForReceivingAutomationResult-${AWS::Region}' Path: / MaxSessionDuration: 43200 AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - edgelambda.amazonaws.com - lambda.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: !Sub 'IAMPolicy-LambdaForReceivingAutomationResult-${AWS::Region}' PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - logs:CreateLogGroup Resource: - 'arn:aws:logs:*:*:*' - Effect: Allow Action: - logs:CreateLogStream - logs:PutLogEvents Resource: - !Sub 'arn:aws:logs:*:*:log-group:/aws/lambda/AutomationResultReceivedByLambda:*' - Effect: Allow Action: - ssm:GetAutomationExecution - ssm:DescribeAutomationStepExecutions Resource: - '*' - Effect: Allow Action: - states:ListActivities - states:ListExecutions - states:ListStateMachines - states:DescribeActivity - states:DescribeExecution - states:DescribeStateMachine - states:DescribeStateMachineForExecution - states:GetExecutionHistory - states:SendTaskSuccess Resource: - '*' LambdaForReceivingAutomationResultPermission: Type: AWS::Lambda::Permission DependsOn: - LambdaForReceivingAutomationResult - EventRuleForAutomationResult Properties: Action: lambda:InvokeFunction FunctionName: !GetAtt LambdaForReceivingAutomationResult.Arn Principal: events.amazonaws.com SourceArn: !GetAtt EventRuleForAutomationResult.Arn EventRuleForAutomationResult: Type: AWS::Events::Rule DependsOn: - LambdaForReceivingAutomationResult - EventRuleForAutomationResultRole Properties: Name: EventRuleForAutomationResult EventBusName: default Description: 'EventRuleForAutomationResult' State: !Ref EventRuleForAutomationResultState EventPattern: source: - aws.ssm detail-type: - 'EC2 Automation Step Status-change Notification' detail: Definition: - !Ref SsmDocumentForApprovalActionName Status: - 'Success' - 'Failed' Action: - 'aws:approve' Targets: - Id: 'EventRuleForAutomationResultTarget' Arn: !GetAtt LambdaForReceivingAutomationResult.Arn EventRuleForAutomationResultRole: Type: AWS::IAM::Role DependsOn: - LambdaForReceivingAutomationResult Properties: RoleName: !Sub 'EventRuleForAutomationResultRole-${AWS::Region}' Path: / AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com - lambda.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: !Sub 'EventRuleForAutomationResultRole-${AWS::Region}' PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - lambda:InvokeFunction Resource: - !Sub '${LambdaForReceivingAutomationResult.Arn}:*' SsmDocumentForApprovalAction: Type: AWS::SSM::Document Properties: Name: !Ref SsmDocumentForApprovalActionName DocumentType: Automation VersionName: !Ref SsmDocumentForApprovalActionVersionName DocumentFormat: YAML Content: description: 'SsmDocumentForApprovalAction' schemaVersion: '0.3' assumeRole: "{{ AutomationAssumeRole }}" parameters: AutomationAssumeRole: type: String description: "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf." default: '' Description: description: 'Operation Description' type: String default: 'SsmDocumentForApprovalAction' Message: description: Message type: String default: 'Please Approve after Confirmation.' NotificationArn: description: 'Amazon SNS Topic ARN for Approval Notification.' type: String default: 'arn:aws:sns:ap-northeast-1:000000000000:AutomationApprovalNotification' Approvers: description: 'The IAM User or IAM Role of the Approver.' type: StringList SfnToken: description: 'AWS Step Functions State Machine Token' type: String default: '' MinRequiredApprovals: description: MinRequiredApprovals type: Integer default: 1 mainSteps: - name: ApprovalAction action: 'aws:approve' timeoutSeconds: 604800 inputs: Message: '{{Message}}' NotificationArn: '{{NotificationArn}}' Approvers: '{{Approvers}}' MinRequiredApprovals: '{{MinRequiredApprovals}}' isEnd: true SnsAutomationApprovalNotification: Type: AWS::SNS::Topic Properties: TopicName: AutomationApprovalNotification DisplayName: AutomationApprovalNotification FifoTopic: False Subscription: - Endpoint: !Ref EmailForNotification Protocol: email StepFunctionsWithSsmAutomationApproval: Type: AWS::StepFunctions::StateMachine DependsOn: - LambdaForSsmStartAutomationExecution - LambdaForReceivingAutomationResult - StepFunctionsWithSsmAutomationApprovalRole - StepFunctionsWithSsmAutomationApprovalLogGroup Properties: StateMachineName: StepFunctionsWithSsmAutomationApproval StateMachineType: STANDARD RoleArn: !GetAtt StepFunctionsWithSsmAutomationApprovalRole.Arn LoggingConfiguration: Level: ALL IncludeExecutionData: true Destinations: - CloudWatchLogsLogGroup: LogGroupArn: !GetAtt StepFunctionsWithSsmAutomationApprovalLogGroup.Arn DefinitionString: !Sub |- { "Comment": "Sample of adding an Approval flow to AWS Step Functions.", "TimeoutSeconds": 604800, "StartAt": "InvokeLambdaForSsmStartAutomationExecution", "States": { "InvokeLambdaForSsmStartAutomationExecution": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken", "Parameters": { "FunctionName": "${LambdaForSsmStartAutomationExecution.Arn}:$LATEST", "Payload": { "step.$": "$$.State.Name", "token.$": "$$.Task.Token", "ssm_automation_assume_role.$": "$$.Execution.Input.ssm_automation_assume_role", "ssm_doc_name.$": "$$.Execution.Input.ssm_doc_name", "ssm_description.$": "$$.Execution.Input.ssm_description", "ssm_notification_arn.$": "$$.Execution.Input.ssm_notification_arn", "ssm_approvers.$": "$$.Execution.Input.ssm_approvers", "ssm_min_required_approvals.$": "$$.Execution.Input.ssm_min_required_approvals", "ssm_message.$": "States.Format('Approval request has been received. Please review file {} at the following URL to decide whether to approve or deny. URL: {}', $$.Execution.Input.confirmation_file, $$.Execution.Input.confirmation_url)" } }, "Retry": [ { "ErrorEquals": [ "Lambda.ServiceException", "Lambda.AWSLambdaException", "Lambda.SdkClientException", "Lambda.TooManyRequestsException" ], "IntervalSeconds": 2, "MaxAttempts": 6, "BackoffRate": 2 } ], "Catch": [ { "ErrorEquals": [ "States.ALL" ], "Next": "Fail" } ], "Next": "ApprovalResult" }, "ApprovalResult": { "Type": "Choice", "Choices": [ { "Variable": "$.is_approved", "BooleanEquals": true, "Next": "Approved" }, { "Variable": "$.is_approved", "BooleanEquals": false, "Next": "Rejected" } ], "Default": "Rejected" }, "Approved": { "Type": "Succeed" }, "Rejected": { "Type": "Succeed" }, "Fail": { "Type": "Fail" } } } StepFunctionsWithSsmAutomationApprovalRole: Type: AWS::IAM::Role DependsOn: - LambdaForSsmStartAutomationExecution - LambdaForReceivingAutomationResult Properties: RoleName: !Sub 'IAMRole-StepFunctionsWithSsmAutomationApproval-${AWS::Region}' Path: / MaxSessionDuration: 43200 AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - states.amazonaws.com - lambda.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: !Sub 'IAMPolicy-StepFunctionsWithSsmAutomationApproval-${AWS::Region}' PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - lambda:InvokeFunction Resource: - !Sub '${LambdaForSsmStartAutomationExecution.Arn}:*' - !Sub '${LambdaForReceivingAutomationResult.Arn}:*' - Effect: Allow Action: - lambda:InvokeFunction Resource: - !Sub '${LambdaForSsmStartAutomationExecution.Arn}' - !Sub '${LambdaForReceivingAutomationResult.Arn}' - PolicyName: CloudWatchLogsDeliveryFullAccessPolicy PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - logs:DescribeResourcePolicies - logs:DescribeLogGroups - logs:GetLogDelivery - logs:CreateLogDelivery - logs:DeleteLogDelivery - logs:UpdateLogDelivery - logs:ListLogDeliveries - logs:PutResourcePolicy Resource: - '*' - PolicyName: XRayAccessPolicy PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - xray:PutTraceSegments - xray:PutTelemetryRecords - xray:GetSamplingRules - xray:GetSamplingTargets Resource: - '*' StepFunctionsWithSsmAutomationApprovalLogGroup: Type: AWS::Logs::LogGroup Properties: LogGroupName: /aws/vendedlogs/states/Logs-StepFunctionsWithSsmAutomationApproval Outputs: Region: Value: !Ref AWS::Region StepFunctionsInputExample: Description: "AWS Step Functions Input Example" Value: !Sub |- { "region": "${AWS::Region}", "ssm_automation_assume_role": "${SsmAutomationAssumeRole.Arn}", "ssm_doc_name": "${SsmDocumentForApprovalAction}", "ssm_description": "Automation Approval Action For AWS Step Functions.", "ssm_notification_arn": "${SnsAutomationApprovalNotification}", "ssm_approvers": "${SsmApprovers}", "ssm_min_required_approvals": "${SsmMinRequiredApprovals}", "confirmation_url": "https://hidekazu-konishi.com/", "confirmation_file": "index.html" }
Build procedure
- Deploy with AWS CloudFormation by entering the necessary values for the template parameters in a region that supports AWS Step Functions and AWS Systems Manager Automation.
After creating the AWS CloudFormation stack, an example of input parameters (in JSON format) for AWS Step Functions execution will be output asStepFunctionsInputExample
in theOutput
field, so make a note of it. - Approve the SNS topic subscription request that will be sent to the email address you entered.
Executing the demo
- Modify the
confirmation_url
andconfirmation_file
in the JSON parameters ofStepFunctionsInputExample
noted in the "Build procedure" above, and use it as the input value to execute the AWS Step Functions state machineStepFunctionsWithSsmAutomationApproval
.
Theconfirmation_url
andconfirmation_file
will be included in the email from the AWS Systems Manager Automation approval action.confirmation_url
is expected to be the URL to refer to for approval, andconfirmation_file
is expected to be the file name within the URL to refer to for approval. For example, you could use the URL of the Amazon S3 console forconfirmation_url
and the file name of an Amazon S3 object forconfirmation_file
. - When you receive the email for the AWS Systems Manager Automation approval action at the email address specified during setup, choose to approve (
Approve
) or reject (Reject
) from the AWS Management Console. - Confirm that the steps of the AWS Step Functions state machine
StepFunctionsWithSsmAutomationApproval
transition according to the chosen approval (Approve
) or rejection (Reject
).
Deletion procedure
- Delete the AWS CloudFormation stack created in the "Build procedure".
References:
Tech Blog with curated related content
What is AWS Step Functions? - AWS Step Functions
AWS Systems Manager Automation - AWS Systems Manager
What is AWS CodePipeline? - AWS CodePipeline
Summary
In this article, I tried adding an approval flow to an AWS Step Functions workflow using the approval action of AWS Systems Manager Automation and Amazon EventBridge.By using this method, I confirmed that a flexible approval process can be incorporated into AWS Step Functions workflows.
By utilizing the approval action of AWS Systems Manager Automation and IAM roles, it's also possible to control the authentication of approvers and the permissions for approval actions.
Additionally, it's possible to perform appropriate continuous processing not only when approval is granted but also when it's denied, enabling it to handle complex approval flows that require intricate processes.
As next steps, we could consider integrating this approval flow with other AWS services or expanding it into a multi-stage approval flow with multiple approval steps.
I learned that by combining AWS serverless services, we can achieve flexible and applicable approval workflows while minimizing maintenance efforts.
I look forward to continuing to explore approval workflow management approaches using AWS services like this in the future.
Written by Hidekazu Konishi
Copyright © Hidekazu Konishi ( hidekazu-konishi.com ) All Rights Reserved.