hidekazu-konishi.com
AWS History and Timeline regarding AWS Lambda - Overview, Functions, Features, Summary of Updates, and Introduction
First Published:
Last Updated:
This time, I have created a historical timeline for AWS Lambda, a serverless and fully managed code execution service announced in November 2014.
As AWS Lambda will celebrate its 10th anniversary since the announcement in November 2024, I wrote this article as an early celebration, although it is quite premature.
Just like before, I am summarizing the main features while following the birth of AWS Lambda and tracking its feature additions and updates as a Current Overview, Functions, Features of AWS Lambda.
I hope these will provide clues as to what has remained the same and what has changed, in addition to the features and concepts of each AWS service.
Background and Method of Creating AWS Lambda Historical Timeline
The reason for creating a historical timeline of AWS Lambda this time is none other than that AWS Lambda will celebrate its 10th anniversary since the announcement in 2024.Another reason is that since AWS Lambda was announced in November 2014, it has been integrated with various AWS services and the supported runtimes (supported languages) have been expanded. Therefore, I wanted to organize the information of AWS Lambda with the following approaches.
- Tracking the history of AWS Lambda and organizing the transition of updates
- Summarizing the feature list and characteristics of AWS Lambda
There may be slight variations in the dates on the timeline due to differences in the timing of announcements or article postings in the references used.
The content posted is limited to major features related to the current AWS Lambda and necessary for the feature list and overview description.
In other words, please note that the items on this timeline are not all updates to AWS Lambda features, but are representative updates that I have picked out.
AWS Lambda Historical Timeline (Updates from November 13, 2014)
Now, here is a timeline related to the functions of AWS Lambda. As of the time of writing this article, the history of AWS Lambda spans about 9 years and 7 months, with its 10th anniversary coming in November 2024.* The table can be sorted by clicking on the column names.
| Date | Summary |
|---|---|
| 2014-11-13 | AWS Lambda is announced. It supports Node.js runtime and deployment packages uploaded as zip files. It provides automatic code execution with a maximum execution timeout of 1 minute triggered by Amazon S3 object change event notifications, messages arriving in Amazon Kinesis streams, and Amazon DynamoDB table updates. |
| 2015-04-08 | AWS Lambda can now be used as a custom resource in AWS CloudFormation. |
| 2015-04-09 | AWS Lambda becomes Generally Available (GA). It supports both real-time (synchronous) and asynchronous events, and resource policies are introduced. |
| 2015-04-09 | Amazon SNS notifications can now trigger Lambda functions. |
| 2015-04-09 | AWS Mobile SDK begins supporting AWS Lambda. |
| 2015-05-28 | Lambda function deployment packages (.zip files) can now be uploaded to an Amazon S3 bucket in the same region, and Lambda functions can be created or updated by specifying the bucket name and object key name. |
| 2015-06-15 | Lambda code can now be created in Java. |
| 2015-07-09 | Previously, the AWS SDK was required to invoke Lambda functions from web, mobile, and IoT applications, but Lambda functions can now be invoked from REST-compatible clients using Amazon API Gateway. |
| 2015-07-09 | The Lambda console begins providing "blueprints" that offer sample event source configurations and sample code. |
| 2015-07-16 | DynamoDB Streams becomes generally available, and Lambda functions can be used as triggers for DynamoDB tables. |
| 2015-10-08 | Lambda function code can now be developed using Python. |
| 2015-10-08 | Versioning functionality is added, allowing different versions of Lambda functions to be executed in different environments. |
| 2015-10-08 | Scheduled events can now be set to periodically execute code. |
| 2015-10-08 | The maximum execution timeout limit is extended from 1 minute to 5 minutes. |
| 2016-02-11 | Lambda functions can now be configured to access resources within a VPC. |
| 2016-04-07 | Node.js runtime v4.3 is supported. |
| 2016-06-23 | The role creation process in the Lambda console is simplified. |
| 2016-11-18 | Serverless applications can be defined using AWS SAM. |
| 2016-11-18 | Environment variables can now be used to specify Lambda function configuration values outside of code. |
| 2016-11-30 | Chatbots can now be created by integrating with Amazon Lex. |
| 2016-12-01 | C# is supported. |
| 2016-12-01 | Lambda functions can now be executed in response to Amazon CloudFront events with Lambda@Edge. |
| 2017-03-22 | Node.js runtime v6.10 is supported. |
| 2017-04-19 | X-Ray is introduced, allowing the detection, analysis, and optimization of performance issues in Lambda applications. |
| 2017-08-11 | SAM Local (now SAM CLI) is introduced, allowing serverless applications to be developed, tested, and analyzed locally. |
| 2017-11-28 | Aliases can be used to shift traffic, and safe deployment of new Lambda function versions is supported. |
| 2017-11-30 | The maximum memory allocation limit for AWS Lambda functions is increased from 1536MB to 3008MB. |
| 2017-11-30 | A new Lambda console is introduced, making it easier to debug and fix function code. Concurrency limits can be set for individual functions. |
| 2018-01-15 | Go and .NET 2.0 runtimes are supported. |
| 2018-04-02 | Node.js runtime version 8.10 is supported. |
| 2018-06-28 | Amazon Simple Queue Service (Amazon SQS) is supported as an event source. |
| 2018-07-09 | .NET Core 2.1.0 runtime is supported. |
| 2018-09-11 | PowerShell Core language is supported. |
| 2018-10-10 | The maximum execution timeout limit is extended from 5 minutes to 15 minutes. |
| 2018-10-11 | Serverless applications can be managed and monitored in the AWS Lambda console. |
| 2018-10-16 | The AWS Lambda Service Level Agreement (SLA) is announced. |
| 2018-11-16 | The maximum asynchronous invocation payload size is increased from 128 KB to 256 KB. |
| 2018-11-19 | Kinesis event sources begin supporting Kinesis Data Streams (KDS) Enhanced Fan-out and HTTP/2 data retrieval. |
| 2018-11-19 | Python 3.7 runtime is supported. |
| 2018-11-29 | Ruby 2.5 runtime is supported. |
| 2018-11-29 | Lambda Layers are introduced, allowing libraries, custom runtimes, and other dependencies to be packaged and deployed separately from function code. |
| 2018-11-29 | Custom runtimes can be built to execute Lambda functions in any programming language. |
| 2018-11-29 | Lambda functions can be specified as targets for Application Load Balancers. |
| 2019-04-25 | Layer version information can be downloaded using the GetLayerVersionByArn API, with the version ARN as input. |
| 2019-05-14 | The announcement is made that the Lambda execution environment will use Amazon Linux 2018.03. |
| 2019-05-15 | Node.js 10 runtime is supported (using Amazon Linux 2). |
| 2019-06-20 | The Lambda console's monitoring page begins including reports from Amazon CloudWatch Logs Insights. |
| 2019-09-03 | New Elastic Network Interfaces (ENIs) begin being used in VPCs. Additional network interfaces no longer need to be created as functions scale up, significantly improving startup times. |
| 2019-09-23 | Custom batch windows can now be set for Amazon DynamoDB and Amazon Kinesis event source mappings. |
| 2019-10-31 | Serverless applications with continuous delivery pipelines integrated with AWS CodeCommit, GitHub, etc. can now be created in the Lambda console. |
| 2019-11-18 | Node.js 12, Python 3.8, and Java 11 runtimes are supported. |
| 2019-11-19 | Event source mappings can now be created to support reading from Amazon SQS FIFO queues. |
| 2019-11-25 | Kinesis streams and DynamoDB streams support the Parallelization Factor, allowing multiple batches to be processed at a time in event source mappings. |
| 2019-11-25 | Failure handling for Kinesis stream and DynamoDB stream event sources is supported. |
| 2019-11-25 | When functions are created or updated while connected to a VPC, shared Elastic Network Interfaces (ENIs) can be instantly created. |
| 2019-11-25 | Configuration options for asynchronous invocations are added. Maximum Retry Attempts and Maximum Event Age can be set. |
| 2019-11-25 | Execution results of asynchronous invocations can now be sent to destinations such as SQS queues, SNS topics, Lambda functions, and EventBridge event buses. |
| 2019-11-26 | Amazon CloudWatch Metrics percentiles are supported. |
| 2019-12-03 | The option to allocate Provisioned Concurrency to function versions or aliases is added. |
| 2019-12-03 | Amazon RDS Proxy can now be created in the Lambda console. |
| 2019-12-16 | Updates to the lifecycle of Lambda states using VPCs are announced. |
| 2020-01-20 | StateReasonCode and LastUpdateStatusReasonCode are added to Reason Codes for when functions are connected to VPCs. |
| 2020-01-20 | Updates to the lifecycle of Lambda states using VPCs become applied by default. |
| 2020-02-18 | ConcurrentExecutions metrics are emitted for all functions, aliases, and versions, allowing the number of Lambda function concurrent executions to be displayed and monitored by version and alias. |
| 2020-02-19 | Ruby 2.7 runtime is supported (using Amazon Linux 2). |
| 2020-02-20 | AWS Lambda is added to Compute Savings Plans. |
| 2020-03-02 | Updates to the lifecycle of Lambda states using VPCs are applied to all Lambda functions. |
| 2020-03-31 | .NET Core 3.1.0 runtime is supported. |
| 2020-06-16 | Amazon EFS file systems can now be attached to Lambda functions. |
| 2020-06-18 | The batch window for HTTP/2 streams can now be configured. |
| 2020-08-10 | IAM condition keys (lambda:VpcIds, lambda:SubnetIds, lambda:SecurityGroupIds) for Lambda VPC settings become usable. |
| 2020-08-12 | Java 8 and custom runtimes are supported (using Amazon Linux 2). |
| 2020-08-13 | Lambda begins supporting Amazon Managed Streaming for Apache Kafka (Amazon MSK) as an event source. |
| 2020-09-17 | AWS Lambda adds console support for visualizing AWS Step Functions workflows. |
| 2020-10-08 | New AWS Lambda extensions for integrating Lambda with operational tools are preview released. |
| 2020-10-20 | AWS Lambda begins supporting AWS PrivateLink, allowing access to the Lambda API from VPCs via interface VPC endpoints. |
| 2020-11-05 | Lambda begins supporting Amazon MQ as an event source. |
| 2020-11-12 | The Runtime Logs API is supported, allowing Lambda extensions to subscribe to the log streams of execution environments. |
| 2020-11-23 | Code signing, allowing administrators to accept only signed code during deployment, becomes usable. |
| 2020-12-01 | Container images can now be used for Lambda functions. |
| 2020-12-01 | The maximum memory allocation limit for AWS Lambda functions is increased from 3008MB to 10240MB. |
| 2020-12-15 | Lambda begins supporting self-managed Apache Kafka as an event source. |
| 2021-02-03 | Node.js 14 runtime is supported (using Amazon Linux 2). |
| 2021-03-02 | The design of the Lambda console is completely revamped with the aim of improving performance and consistency. |
| 2021-05-24 | Lambda extensions become generally available. Extensions provided by Lambda partners can now be used. |
| 2021-06-29 | Lambda begins supporting the SASL/PLAIN authentication mechanism for self-managed Apache Kafka event sources. |
| 2021-07-07 | Lambda begins supporting Amazon MQ for RabbitMQ as an event source. |
| 2021-08-16 | Python 3.9 runtime is supported. |
| 2021-09-29 | Lambda begins supporting the 64-bit ARM architecture (arm64) for AWS Graviton2 processors. |
| 2021-11-19 | mTLS authentication is supported for Amazon MSK and self-managed Apache Kafka event sources. |
| 2021-11-26 | Event filtering is supported for Kinesis, DynamoDB, and Amazon SQS event sources. |
| 2022-02-24 | .NET 6 runtime is supported. |
| 2022-03-11 | Lambda begins supporting granting permissions to AWS Organizations' organizations in resource-based policies. |
| 2022-03-16 | Test events can now be shared with other users in the same AWS account. |
| 2022-03-24 | Ephemeral storage for Lambda functions, which was 512MB, can now be extended up to 10GB. |
| 2022-04-06 | AWS Lambda Function URLs, dedicated HTTPS endpoints for Lambda functions, are supported. |
| 2022-05-12 | Node.js 16 runtime is supported (using Amazon Linux 2). |
| 2022-07-15 | AWS Lambda Powertools for TypeScript becomes generally available. |
| 2022-07-19 | AWS Lambda begins supporting attribute-based access control (ABAC). |
| 2022-07-22 | An IAM condition key (lambda:SourceFunctionArn) for specifying the ARN of the source function in IAM policy conditions is added. |
| 2022-11-18 | Node.js 18 runtime is supported (using Amazon Linux 2). |
| 2022-11-28 | Lambda SnapStart can be used to reduce Java function startup times. |
| 2023-01-24 | AWS Lambda supports runtime management controls, allowing control over when functions are updated to new runtime versions. |
| 2023-02-09 | CloudWatch metrics for asynchronous invocations (AsyncEventsReceived, AsyncEventAge, AsyncEventsDropped) are supported. |
| 2023-04-07 | Response payload streaming is supported, allowing response payloads including over 6MB to be incrementally streamed to clients. |
| 2023-06-07 | Ruby 3.2 runtime is supported. |
| 2023-07-27 | Python 3.11 runtime is supported. |
| 2023-10-12 | Internet Protocol version 6 (IPv6) is supported for VPC outbound connections. Resources within dual-stack VPCs (outbound connections) can now be accessed via IPv6. |
| 2023-11-09 | Lambda supports Amazon Linux 2023 as both managed runtimes and container-based images. |
| 2023-11-15 | Node.js 20 runtime is supported. |
| 2023-11-17 | Java 21 runtime is supported. |
| 2023-12-14 | Python 3.12 runtime is supported. |
| 2024-02-22 | .NET 8 runtime is supported. |
| 2024-04-04 | Ruby 3.3 runtime is supported. |
Current Overview, Functions, Features of AWS Lambda
From here, we will explain in detail the main features of the current AWS Lambda.AWS Lambda is a serverless computing service that allows you to run code without managing servers or infrastructure.
AWS Lambda streamlines application development and operation by providing event-driven processing and automatic resource scaling.
It also automatically allocates the resources needed to execute your code and bills you only for what you use, enabling cost-effective solutions.
Additionally, it achieves high security through access management with AWS Identity and Access Management (IAM) and encryption of data in transit and at rest.
Along with these features, it can easily integrate with other AWS services, allowing for rapid development and flexible operation of applications.
AWS Lambda Use Cases
AWS Lambda is utilized in a variety of scenarios and is designed to build scalable, event-driven applications.The main use cases of the serverless computing service provided by AWS Lambda include the following:
- Data Processing
Lambda can process or transform data triggered by change events from data stores such as S3 and DynamoDB. - Web Application Backend
By combining API Gateway and Lambda, you can build a serverless backend for web applications. - Event-Driven Workflows
By combining Step Functions and Lambda, you can implement workflows with complex business logic.
Specific Examples of Use Cases
For instance, there are the following specific examples of use cases.- Generating Image Thumbnails
When an image is uploaded to an S3 bucket, Lambda automatically generates a thumbnail image and saves it to another S3 bucket.
This allows you to generate thumbnail images required by your application in real-time. - Web Application Backend
Build a backend for a web application using API Gateway and Lambda.
By implementing business logic in Lambda and invoking it from API Gateway, you can separate the frontend and backend and build a scalable application. - IoT Data Processing
Collect data from IoT devices with AWS IoT and process it in real-time with Lambda.
For example, you can perform anomaly detection based on sensor data and generate alerts.
AWS Lambda Conceptual Diagram
From here, I will explain the main features and characteristics of AWS Lambda, but before that, I will show the following conceptual diagram of AWS Lambda to make it easier to imagine the overall picture of AWS Lambda.
This diagram illustrates how events from event sources are processed by AWS Lambda functions in either a synchronous or asynchronous manner.
The invocation method, synchronous or asynchronous, depends on the event source. Some services support both synchronous and asynchronous invocations, while others support only one method.
In the case of asynchronous invocation, the processing is done through AWS Lambda event queues.
Lambda functions support multiple programming runtimes, including Node.js, Python, Java, .NET, Ruby, or OS-only Runtime, allowing code written in different languages to be executed.
Within the Lambda function, you can call AWS services or other services using the AWS SDK or API when needed.
The results of asynchronous invocations can be sent to specified destinations such as Amazon SQS, Amazon SNS, AWS Lambda itself, or Amazon EventBridge.
In the following sections, we will provide more details about these features and configurations of AWS Lambda.
AWS Lambda Function Runtimes
AWS Lambda provides support for a wide variety of programming languages through "runtimes".Runtimes provide language-specific environments and mediate event invocation, context information, and responses between Lambda and functions.
AWS Lambda users can use the managed runtimes provided by AWS Lambda or build their own custom runtimes.
The choice of runtime is made when creating a function, and transitioning to a new major version requires a change in the runtime identifier. AWS Lambda does not guarantee backward compatibility between major versions, so this update must be user-driven.
Currently Supported Main Runtimes
As of the writing of this article, the following runtimes are supported in AWS Lambda.- Node.js:
nodejs20.x,nodejs18.x,nodejs16.x - Python:
python3.12,python3.11,python3.10,python3.9,python3.8 - Java:
java21,java17,java11,java8.al2 - .NET:
dotnet8,dotnet6 - Ruby:
ruby3.3,ruby3.2 - OS-only Runtime:
provided.al2023,provided.al2(Other languages such as Go and Rust can be used with custom runtimes on OS-only runtimes)
Lambda regularly manages runtime updates, providing security patches and support for minor version releases.
However, when a runtime is deprecated, new security patches are no longer applied, and technical support ends, so it is recommended to migrate to the latest or supported runtime.
With this information in mind, when developing and operating Lambda functions, it is important to constantly check the support status of the runtime in use and plan runtime upgrades as necessary.
AWS Lambda Function Configuration Options
When configuring an AWS Lambda function, you can customize the core functionality and options of the function using the Lambda API or console.Here are the main configuration options and the range of values that can be set for them.
Memory
Set the amount of memory required to execute the function. The amount of memory directly impacts performance, and CPU and network bandwidth are scaled according to the allocated memory amount.Settable values: Minimum 128MB, Maximum 10,240MB
Ephemeral Storage
Learn how and when to increase the ephemeral storage capacity of your function. Ephemeral storage is used to store temporary files generated during the execution of your function.Settable values: Minimum 512MB, Maximum 10,240MB
Timeout
Set the maximum execution time for your function. Setting this value appropriately can prevent unexpected long-running executions and control costs.Settable values: Minimum 1 second, Maximum 900 seconds (15 minutes)
Environment Variables
Use environment variables to keep your function code portable while isolating secrets from your code.Environment variables store configuration values in your function's configuration and can be referenced in your code without directly hardcoding them.
VPC
By applying VPC settings to Lambda functions, they can securely interact with resources such as RDS and EC2 instances within private subnets in the VPC.Using interface VPC endpoints, it's also possible to invoke Lambda functions without going through the public internet.
While this configuration enhances security and potentially improves performance, it requires proper network setup.
File System
You can mount an Amazon EFS file system to your Lambda function and use it in your local directory.Using a file system allows your function code to safely access and modify shared resources with high concurrency.
Aliases
Set up aliases to allow clients to invoke a specific version of your Lambda function.This allows you to specify different versions of your function without updating your clients.
Versions
By publishing a version of your function, you can save your code and configuration as a separate, immutable resource.This facilitates stable deployments and rollbacks.
Response Streaming
Set up a function URL to stream response payloads to your clients.Response streaming can improve the performance of latency-sensitive applications by reducing the time to first byte. It can also be used to build functions that return large payloads.
AWS Lambda Function Deployment Methods
There are mainly two methods for deploying an AWS Lambda function.The first method is to upload a .zip file archive, and the second is to create and upload a container image.
Here is a brief explanation of each method.
Deploying with a .zip File Archive
A .zip file archive contains your application code and its dependencies.When you create a function using the Lambda console or toolkit, Lambda automatically creates a .zip file archive of your code.
If you create a function using the Lambda API, command line tools, or AWS SDKs, you need to create the deployment package yourself.
Upload the deployment package from Amazon S3 or your local machine to deploy your function code.
File permissions are required for the Lambda runtime to read the files in your deployment package.
Non-executable files require 644 permissions, and directories and executable files require 755 permissions.
Deploying with a Container Image
You can also package your code and dependencies using a container image.Create a container image using tools like Docker CLI and upload it to Amazon Elastic Container Registry (Amazon ECR).
When you invoke your function, Lambda deploys the container image to the execution environment.
After running your function's initialization code, Lambda invokes the configured entry point to run your function.
AWS provides open-source base images that are compatible with the Lambda service, and you can also use a different base image from other container registries.
It also provides a runtime interface emulator for testing your functions locally.
There are no additional charges for packaging or deploying functions as container images, but there are associated charges for storing container images in Amazon ECR.
Also, when Lambda downloads your container image for the first time, it optimizes the image and encrypts and stores it.
All encryption keys are protected by AWS KMS, and you can audit Lambda's key usage through AWS CloudTrail logs.
Deployment methods for Lambda functions vary widely, but it is important to choose the appropriate method according to your project needs and ease of management.
AWS Lambda Function Invocation Methods and Testing
After deploying an AWS Lambda function, you can invoke and test the function in various ways.Here, I will briefly explain the main invocation methods and testing methods and summarize them as sections of a blog post.
Main Function Invocation Methods
- Lambda Console
You can directly test your function using the Lambda console. This is very convenient during development or debugging. - AWS SDK
Invoke your Lambda function programmatically using the AWS SDK. This is commonly used when your application integrates with a Lambda function. - Invoke API
You can directly invoke your function using the Lambda Invoke API. This allows for fine-grained configuration and synchronous/asynchronous control. - AWS Command Line Interface (CLI)
You can directly invoke your function from the command line using the AWS CLI. This is useful for scripts and automated tasks. - AWS Lambda Function URLs HTTPS Endpoint
You can also invoke your function through a dedicated HTTP(S) endpoint by configuring AWS Lambda Function URLs. This allows you to use your Lambda function as a web service that receives HTTP requests. However, since AWS Lambda Function URLs are directly accessible from the internet, it is important to set up proper access control. You need to manage security risks by using resource-based policies to allow access only from specific IP addresses and properly configuring Cross-Origin Resource Sharing (CORS). - Event Source Mapping
You can automatically trigger your Lambda function in response to events from AWS services such as Amazon DynamoDB, Amazon Kinesis, and Amazon SQS. This enables efficient processing of stream data or messages from queues.
Synchronous and Asynchronous Invocation
- Synchronous Invocation
The invoker waits for the Lambda function to complete and receives the result. This is suitable when you need a response from the function in real-time. - Asynchronous Invocation
Events are passed asynchronously to Lambda via the AWS Lambda event queue, and Lambda performs the processing in the background. The caller receives an immediate response, and the processing result is checked later.
This is suitable for event-driven processing or when processing delays are acceptable.
The result records of asynchronous invocations can be sent to specified destinations such as Amazon SQS, Amazon SNS, AWS Lambda, or Amazon EventBridge.
For these asynchronous invocation result records, you can specify separate destinations for successfully processed events and events that failed all processing attempts.
At these destinations for asynchronous invocation result records, it is also possible to perform event-driven processing triggered by the result records.
Testing and Debugging
- Testing in the Console
In the Lambda console, you can create custom test events and test your function. This allows you to immediately check if your function under development behaves as expected. - Local Testing
You can also test your Lambda function locally using AWS SAM (Serverless Application Model) or a local development environment. This is useful for detailed debugging before deployment.
Depending on the development phase or operational scenario, you can effectively use synchronous invocation, asynchronous invocation, and event source mapping, and leverage the rich AWS tools and services to enhance the performance and reliability of your functions.
AWS Lambda Function Management Methods
The following methods and tools can be used to efficiently manage AWS Lambda functions.Using AWS CLI
The AWS Command Line Interface (CLI) is a tool for directly managing Lambda functions and other AWS resources from the command line.The AWS CLI is written in Python and utilizes the AWS SDK (Boto3) to communicate with AWS services. With this tool, you can script and automate management tasks such as creating, updating, and deleting functions.
Function Scaling
Lambda provides two concurrency control features:- Reserved Concurrency
You can reserve concurrency for a specific function in advance. This allows you to maintain the necessary performance without being affected by other functions. - Provisioned Concurrency
To improve responsiveness, you can provision pre-initialized execution environments. This allows your function to respond immediately, reducing startup latency (cold starts).
Code Signing
Using the code signing feature, you can ensure that the code deployed to your Lambda function is signed by an approved developer and has not been tampered with.This is important for increasing the reliability and integrity of your function code. The signature is verified during deployment, and if verification fails, the deployment is blocked.
Organization with Tagging
Tagging AWS resources allows you to categorize and manage them more easily.For example, by tagging resources with project names, department names, or environment names (e.g., development, testing, production), it becomes easier to manage cost centers and set up access controls.
Using Layers
Lambda layers are a feature for managing and reusing common dependencies and libraries separately from your function code.With layers, you can manage these common components in one place and share them across multiple Lambda functions.
This allows you to keep your deployment packages small while streamlining the development process and reducing code duplication.
By appropriately using these features, you can improve the performance, security, and management efficiency of your Lambda functions.
AWS Lambda Security Measures
Security is also a top priority when using AWS Lambda.AWS cloud security is designed to meet the requirements of the most security-conscious organizations.
Like other AWS services, AWS Lambda operates under the AWS shared responsibility model, where AWS is responsible for the security of the cloud infrastructure, and customers are responsible for the security configuration and management of the AWS services they use.
Data Protection and Compliance When Using AWS Lambda
The following recommendations are suggested for achieving data protection and compliance when using AWS Lambda:- Protect AWS account credentials: Configure AWS IAM Identity Center and AWS Identity and Access Management (IAM) to grant only necessary permissions to each user.
- Use Multi-Factor Authentication (MFA): Enhance account security.
- Use SSL/TLS for communication: Use TLS 1.2 or higher for communication with AWS resources (TLS 1.3 is recommended).
- Log API and user activity: Set up AWS CloudTrail to enhance security monitoring and activity tracking.
- Utilize AWS encryption solutions: Use in conjunction with default security controls.
- Use managed security services like Amazon Macie: Helps protect sensitive data stored in Amazon S3.
- Leverage security monitoring tools: Use AWS Security Hub and Amazon GuardDuty to continuously monitor security status and quickly detect threats.
AWS Lambda IAM and Access Management
IAM plays a central role in access management for AWS Lambda.Using IAM, administrators can securely control access to AWS resources. Pay attention to the following points:
- Proper authentication and authorization management: Use IAM to control who can access Lambda resources.
- Policy-based access control: Use IAM policies to define access to specific actions and resources.
Encryption in Transit and at Rest
Like other AWS services, data encryption in transit and at rest is recommended for data protection in AWS Lambda.- Encryption in Transit: Lambda API endpoints support only secure connections over HTTPS, and communication is encrypted using TLS when managing Lambda resources through the AWS Management Console, SDKs, or APIs.
- Encryption at Rest: Lambda always encrypts environment variables for storage. By default, it uses AWS-managed keys, but you can also encrypt them using customer-managed keys.
Security Governance and Compliance
Businesses using AWS Lambda need to build appropriate governance strategies based on data sensitivity and regulatory requirements.Use AWS CloudFormation Guard or AWS Config to automatically assess security and maintain compliance for Lambda function deployments.
This also includes tagging Lambda functions and connecting them to VPCs.
Resilience and Scaling
- Versioning and Aliases: Lambda supports versioning of code and configuration, enabling advanced deployment strategies such as blue/green deployments.
- High Availability: Lambda executes functions across multiple Availability Zones, allowing event processing to continue even if there are service disruptions in some zones.
AWS Lambda Monitoring
AWS Lambda integrates with monitoring services such as Amazon CloudWatch to monitor and troubleshoot Lambda functions.Lambda automatically monitors functions and reports metrics to CloudWatch, such as the number of requests, execution time per request, and the number of requests that resulted in errors.
Function Monitoring in the Lambda Console
In the Lambda console, you can visually check metrics such as the number of requests, execution time, and error count using the monitoring tab of your function.These graphs allow you to quickly understand your function's performance and reduce response time when issues occur.
Integration with CloudWatch Logs
Logs related to function execution are automatically sent to CloudWatch Logs.This allows you to retrieve detailed information about function execution from the logs, which is useful for analyzing the cause of errors and performance issues.
Visualization with AWS X-Ray
Using AWS X-Ray, you can visually inspect trace data related to function invocations, which helps identify performance bottlenecks and error causes.X-Ray is suitable for understanding the behavior of your entire application by tracing requests across services.
Amazon CloudWatch Lambda Insights
Using CloudWatch Lambda Insights, you can collect and aggregate system-level metrics and logs related to function execution, which helps diagnose issues with Lambda functions.Lambda Insights provides metrics such as CPU usage, memory, disk, and network usage, helping you gain a deeper understanding of your function's performance.
By using these tools and services, you can maintain the health of your AWS Lambda functions and be prepared to quickly respond to unexpected downtime or performance degradation.
The various monitoring tools help you efficiently operate your Lambda functions and ultimately contribute to improving the reliability and customer experience of your application.
Managing Common Components with AWS Lambda Layers
AWS Lambda Layers is a feature for managing and reusing common components separately from your Lambda function code.Using layers, you can manage libraries, custom runtimes, and other dependencies separately from your Lambda function and share them across multiple Lambda functions.
This allows you to keep your function's deployment package small while increasing code reusability.
Lambda layers can be associated with specific Lambda functions, and the contents of the layer are added to the runtime environment when the function is executed.
Developers can create their own layers or use layers published by other developers.
Using AWS Lambda Layers provides the following benefits.
- Increased Code Reusability
By sharing the same libraries and custom runtimes across multiple Lambda functions, code reusability is improved. - Reduced Lambda Function Size
By moving common dependencies to layers, the size of your Lambda functions can be reduced. This reduces function deployment time. - Simplified Management
Updates to libraries and runtimes can be performed in one place, simplifying management.
By leveraging AWS Lambda Layers, you can improve code manageability and reusability, enabling efficient development of serverless applications.
AWS Services That Can Invoke AWS Lambda
As a serverless computing service, AWS Lambda plays a central role in realizing event-driven processing.One of the powerful features of Lambda is its ability to seamlessly integrate with other AWS services, allowing Lambda functions to be triggered by events occurring in various AWS services.
Here, I have summarized in a table the main AWS services that can invoke AWS Lambda, explaining the methods they use (event-driven, asynchronous, synchronous, event source mapping, etc.) to invoke AWS Lambda functions and the use cases in which they are used.
| Service | Method of invocation | Description |
|---|---|---|
| Amazon Alexa | Event-Driven Synchronous Invocation |
Use Lambda functions as the backend for Alexa skills to generate custom responses based on user requests. |
| Amazon Managed Streaming for Apache Kafka | Event source mapping | Stream messages from Kafka topics to Lambda functions for real-time data processing. |
| Self-managed Apache Kafka | Event source mapping | Execute Lambda functions triggered by messages from self-managed Apache Kafka clusters for real-time data processing. |
| Amazon API Gateway | Event-Driven Synchronous Invocation |
Invoke Lambda functions through REST APIs or WebSocket APIs. API Gateway converts HTTP requests to Lambda functions and acts as a backend. |
| AWS CloudFormation | Event-Driven Asynchronous Invocation |
Define Lambda functions as custom resources to automate specific operations during stack creation or update processes. |
| Amazon CloudFront (Lambda@Edge) | Event-Driven Synchronous Invocation |
Execute Lambda functions during CloudFront request or response processing, enabling custom code execution within a global content delivery network. |
| Amazon CloudWatch Logs | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by log data from CloudWatch Logs log groups, allowing you to set up automated actions based on log data. |
| AWS CodeCommit | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by pushes to a repository for automated tasks based on code changes. |
| AWS CodePipeline | Event-Driven Asynchronous Invocation |
Use Lambda functions in each stage of the pipeline for custom actions and validations. |
| Amazon Cognito | Event-Driven Synchronous Invocation |
Trigger Lambda functions based on user pool events, such as user registration or authentication, to execute custom logic. |
| AWS Config | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by configuration changes or compliance rule evaluation results. |
| Amazon Connect | Event-Driven Synchronous Invocation |
Invoke Lambda functions within Amazon Connect contact flows to enable dynamic customer interactions. |
| Amazon DynamoDB | Event source mapping | Send table change data as a stream to Lambda functions to automate processing based on data changes. |
| Amazon Elastic File System | Special integration | Trigger Lambda functions based on EFS file system events to execute custom actions based on file operations. |
| Elastic Load Balancing (Application Load Balancer) | Event-Driven Synchronous Invocation |
Directly trigger Lambda functions in response to HTTP(S) requests, serving as a backend server. |
| AWS IoT | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by messages from IoT devices for device data processing and analysis. |
| Amazon Kinesis | Event source mapping | Send stream data in real-time to Lambda functions for data processing. |
| Amazon Data Firehose | Event-Driven Synchronous Invocation |
Send collected data to Lambda functions for data transformation or preprocessing before loading into data stores or analytics tools. |
| Amazon Lex | Event-Driven Synchronous Invocation |
Execute chatbot intent processing in Lambda to dynamically control user interactions. |
| Amazon MQ | Event source mapping | Execute Lambda functions triggered by messages from message brokers. |
| Amazon Simple Email Service (SES) | Event-Driven Asynchronous Invocation |
Execute Lambda functions based on incoming emails for automated email processing or responses. |
| Amazon Simple Notification Service (SNS) | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by message publications to topics. |
| Amazon Simple Queue Service (SQS) | Event source mapping | Execute Lambda functions triggered by messages from message queues. |
| Amazon Simple Storage Service (S3) | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by object operations (creation, update, deletion) in a bucket. |
| Amazon Simple Storage Service Batch | Event-Driven Synchronous Invocation |
Synchronously execute Lambda functions on a large number of objects in an S3 bucket for batch processing. |
| AWS Secrets Manager | Event-Driven Synchronous Invocation |
Process events such as secret rotation in Lambda functions to automate the management of database credentials and other secrets. |
| Amazon VPC Lattice | Event-Driven Synchronous Invocation |
Forward received requests to Lambda functions based on VPC Lattice routing rules for dynamic application processing. |
| AWS X-Ray | Special integration | Send Lambda function execution traces to X-Ray for performance visualization and analysis. |
| Amazon EventBridge | Event-Driven Asynchronous Invocation |
Execute Lambda functions triggered by events from AWS services or custom applications to implement event-driven architectures. |
| AWS Step Functions | Event-Driven Synchronous Invocation or Asynchronous Invocation |
Invoke Lambda functions at each step of a state machine to control workflows and implement complex business logic. |
You can combine Lambda with services in various domains, including storage, databases, messaging, log management, API management, authentication, IoT, and deployment.
This enables you to extend application functionality, automate operations, and build innovative solutions.
AWS Lambda Best Practices
Effectively leveraging best practices when using AWS Lambda optimizes security, performance, and cost.Below is an overview of key best practices to implement across major areas including function code, configuration, scalability, and security.
Function Code
- Separate handler and core logic: Separate business logic from the Lambda handler to improve unit testability.
- Reuse execution environment: Initialize SDK clients and database connections outside the function handler, and cache static assets in the /tmp directory.
- Use environment variables: Utilize environment variables when passing operation parameters to functions.
- Minimize deployment package size: Strictly manage dependencies and keep deployment packages to a minimum.
- Ensure idempotency: Write idempotent code so that functions return the same result for the same event.
- Avoid private APIs: Don't rely on APIs that AWS provides privately.
Function Configuration
- Optimize memory and timeout: Conduct performance tests to set appropriate memory size and timeout values.
- Principle of least privilege: Properly configure IAM policies to grant only the minimum necessary permissions to functions.
- Remove unused functions: Delete unused Lambda functions to avoid wasting resources.
Function Scalability
- Manage throttling: Apply timeouts, retries, and jittered backoff, and set provisioned concurrency to increase resilience against throttling.
Metrics and Alarms
- Utilize CloudWatch alarms: Monitor Lambda function metrics to detect unexpected behavior or performance degradation early.
- AWS Cost Anomaly Detection: Use Cost Anomaly Detection to identify unusual cost increases.
Processing Streams from Event Sources
- Optimize batch size: Adjust event source polling frequency and batch size for efficient data processing.
- Ensure idempotency: Keep function code idempotent to prevent issues from duplicate event processing.
Security
- Enhance security monitoring: Use AWS Security Hub and Amazon GuardDuty to continuously monitor the security status of Lambda functions and detect threats early.
It's also important to regularly review your code and setup to find opportunities for improvement.
References:
Tech Blog with curated related content
AWS Documentation(AWS Lambda)
Summary
In this article, I created a historical timeline of AWS Lambda and looked at the list of features and overview of AWS Lambda.AWS Lambda, a serverless and fully managed code execution service, was announced in 2014 as the first AWS infrastructure service and became generally available (GA) in 2015.
Even after nearly 10 years since then, AWS Lambda continues to provide the extremely important functionality of serverless code execution triggered by events or schedules in modern computing such as microservices architecture, distributed systems, and serverless computing, while continuously updating its features.
I would like to continue monitoring the trends of what kind of features AWS Lambda will provide in the future.
In addition, there is also a historical timeline of all AWS services including services other than AWS Lambda, so please have a look if you are interested.
AWS History and Timeline - Almost All AWS Services List, Announcements, General Availability(GA)
Written by Hidekazu Konishi
Copyright © Hidekazu Konishi ( hidekazu-konishi.com ) All Rights Reserved.