Lambda Event Pollers in Provisioned Mode for Kafka and SQS - MinimumPollers, MaximumPollers, Poller Groups, and Setting Only One Side
First Published:
Last Updated:
It is also possible to specify only one side – either the minimum or the maximum. What value is used for the side that is not explicitly specified? How far do the pollers grow, and what limits them? Which configurations are incompatible? This article examines these questions based on the AWS Lambda developer guide and API reference. It arranges them in tables by type (Amazon MSK and self-managed Apache Kafka, Amazon SQS) and combinations of settings (nothing set, minimum only, maximum only, both, poller groups).
This article does not cover topics such as pricing, performance comparisons, Kafka authentication, or ordering design. It also does not recommend whether on-demand mode or Provisioned mode is preferable. The information presented in this article was verified on October 19, 2026. The results presented in Section 9 were obtained on October 19, 2026, with a small SQS configuration built for this article in the US East (N. Virginia) Region.
Related articles on this site:
- AWS Lambda Concurrency and Scaling Guide - Reserved and Provisioned Concurrency, Throttling, and Event Source Scaling
- How AWS Lambda Authenticates to Apache Kafka - Where Each Event Source Mapping Method Keeps the Credential, Who Refreshes the Token, and What the Brokers Must Trust
- What Breaks Ordering, Service by Service on AWS - SQS FIFO, SNS FIFO, Kinesis, Amazon MSK, DynamoDB Streams, and the EventBridge Custom Event Bus
- AWS History and Timeline regarding AWS Lambda - Overview, Functions, Features, Summary of Updates, and Introduction
- AWS Lambda Master Index - A Hub for Lambda Articles
- AWS Lambda Web Functions as a Contract - Endpoint Types, Authentication, Scaling Limits, 429s and 503s, and What Is Fixed at Creation
Table of Contents
- 1. Scope and Assumptions
- 2. How to Read the Tables
- 3. Event Pollers and the Two Modes
- 4. MinimumPollers and MaximumPollers
- 5. Setting Only One Side
- 6. What Caps the Number of Event Pollers, and What Cannot Be Combined
- 7. Poller Groups with PollerGroupName
- 8. Where It Shows
- 9. What This Configuration Returned
- 10. Failure Modes
- 11. Frequently Asked Questions About Lambda Event Pollers in Provisioned Mode
- 12. Summary
- 13. References
1. Scope and Assumptions
This section first defines the intended audience, the scope of this article, the sources read, and availability and naming. The key conclusions are as follows:- Ranges and defaults differ by event source type. For Kafka,
MinimumPollersranges from 1 to 200 with a default of 1, andMaximumPollersranges from 1 to 2,000 with a default of 200. For SQS,MinimumPollersranges from 2 to 200 with a default of 2, andMaximumPollersranges from 2 to 10,000 with a default of 200 (as per the API reference). - If only one side is specified, the unspecified side will use the default value. The developer guide's console procedure states that for both Kafka and SQS, leaving a field blank will cause Lambda to select the default value. The Compute Blog, dated November 26, 2024, describes how far the pollers grow when only one side is set for Kafka. However, in the SQS configuration described in this article, passing only the minimum to an ESM that already had a maximum left the maximum at its previous value (the reverse direction was not tried; Section 9.2).
- In Kafka, Lambda caps
MaximumPollersat the number of partitions in the topic to maintain order within those partitions. How far the pollers can scale also depends on the function's concurrency settings. - In SQS, Provisioned mode and maximum concurrency (specified in
ScalingConfig'sMaximumConcurrency) cannot be used together. PollerGroupNameis a setting for Kafka ESMs only. A single group can contain up to 100 ESMs, and the sum ofMaximumPollersacross the ESMs in a group cannot exceed 2,000.
1.1 Intended Audience
This article is intended for the following readers:- Developers and operations personnel who are using Lambda to process Kafka topics or SQS queues and are considering implementing Provisioned mode as a way to handle latency and burst traffic
- Those who believe: "If you only write the minimum, the maximum becomes unlimited," "By using Provisioned mode, function concurrency settings become irrelevant," "SQS can also use
MaximumConcurrencytogether with Provisioned mode," "Poller groups can also be used with SQS," and "With Kafka, you can achieve virtually unlimited parallelism by increasing the number of pollers."
None of these beliefs matches the sources this article read. The API reference states that the default value for
MaximumPollers is 200. The Kafka page states the maximum event pollers your event source mapping can scale to depends on the function's concurrency settings, and states that MaximumPollers is capped by the number of partitions. The SQS page states that you cannot use the maximum concurrency setting in conjunction with Provisioned mode. Finally, the API reference description for PollerGroupName begins with (Amazon MSK and self-managed Apache Kafka).1.2 What This Article Covers and What It Does Not
This article covers the event poller modes of Kafka and SQS ESMs and the combinations of their settings. Specifically, it covers the differences between on-demand mode and Provisioned mode, the ranges and default values forMinimumPollers and MaximumPollers, behavior when only one side is specified, what caps the number of pollers, incompatible configurations, PollerGroupName, and metrics and API responses.This article does not cover:
- Pricing and billing details.
- Performance comparisons, and measured values from documentation or blog posts.
- Authentication of Kafka ESMs. This site's How AWS Lambda Authenticates to Apache Kafka covers it.
- The detailed workings of standard SQS scaling and
MaximumConcurrency. Section 6 of this site's AWS Lambda Concurrency and Scaling Guide covers this. - Ordering design. This site's What Breaks Ordering, Service by Service on AWS covers it.
- Stream start positions, or ESM for Kinesis and DynamoDB Streams.
- Detailed error handling (failed destinations, partial batch failures).
1.3 Sources Read and What Was Run
On October 19, 2026, this article read the following AWS sources: from the Lambda Developer Guide, the ESM overview page (the Provisioned mode section), the Kafka poller scaling mode page, the SQS page (two sections on Provisioned mode), the SQS scaling page, the SQS ESM parameters page, the SQS ESM creation and configuration page, and the metrics types page.It also read the Lambda API Reference pages
ProvisionedPollerConfig, CreateEventSourceMapping, GetEventSourceMapping, UpdateEventSourceMapping, ScalingConfig, the CloudFormation pages for AWS::Lambda::EventSourceMapping and ProvisionedPollerConfig, the AWS SAM page for SQS events, and the Provisioned mode page of the Lambda console's help panel.It read eight What's New posts: the announcement of Kafka's Provisioned mode (November 22, 2024), Kafka's low-latency processing (July 17, 2025), the announcement of SQS's Provisioned mode (November 14, 2025), Kafka poller groups (November 21, 2025), integration with the Amazon MSK console (November 21, 2025), the addition of Regions for Kafka (April 24, 2026), SQS's 10,000 poller support (August 3, 2026), and the announcement of Lambda Web Functions (October 12, 2026).
It also read three blog posts: a Compute Blog post introducing Kafka's Provisioned mode (November 26, 2024), a Compute Blog post introducing SQS's Provisioned mode (August 10, 2026), and a News Blog post introducing SQS's Provisioned mode (November 14, 2025).
The results in Section 9 come from a configuration built for this article. In the US East (N. Virginia) Region (
us-east-1), CloudFormation created one standard SQS queue and one Event Function. The AWS CLI then created an ESM, and the responses were recorded while ProvisionedPollerConfig was changed step by step. No Kafka (Amazon MSK) configuration was created. No performance was measured.1.4 Availability and Naming
The What's New of November 22, 2024 describes Kafka's Provisioned mode asgenerally available in all AWS Commercial Regions where AWS Lambda is available, except …. That What's New lists the excluded Regions. The What's New of April 24, 2026 announces additional Regions. For SQS's Provisioned mode, the What's New of November 14, 2025 states This feature is generally available in all AWS Commercial Regions. The What's New of August 3, 2026 states that the limit for SQS's MaximumPollers was raised from 2,000 to 10,000.Since October 12, 2026, existing Lambda functions have been called Lambda Event Functions. The What's New of October 12, 2026 describes this as
a naming change only that requires no code changes or redeployments for your existing functions. After this first mention, this article calls them Event Functions. The Kafka page already uses the term target Lambda Event Function. The API names and parameter names such as FunctionName have not changed. The term "Lambda function" used in previously published articles on this site is the name used before October 12, 2026 and is not incorrect.The term "provisioned" is used for other services within AWS as well. In this article, "Provisioned mode" refers to the configuration of event pollers for ESM. "Provisioned concurrency," which pre-initializes the function execution environment, is a different setting on the function itself. The frequently asked questions section of this site's AWS Lambda Concurrency and Scaling Guide covers the distinction between the two. The "provisioned capacity" for Amazon DynamoDB and the "provisioned" clusters for Amazon MSK are not related to the "Provisioned mode" discussed in this article.
This article uses Provisioned mode and on-demand mode as the names of the modes. The Kafka page refers to the default mode as
On-demand mode (default), while the SQS scaling page uses Standard mode (Default). When this article quotes a source verbatim, it keeps the source's capitalization.2. How to Read the Tables
This section defines the columns and terms used in the tables in this article.2.1 Common Columns
The tables in this article use six columns. The first five columns are common to two related articles on this site, while the sixth column is specific to this article. The row label column isSetting and holds the combination of fields that is set.| Column | What the cell holds |
|---|---|
AWS Runs | What AWS operates for this setting |
You Set | What the user configures, and on which resource |
If You Set Nothing | The value or behavior when the user leaves a field unset, with the source of that statement |
Where It Shows | Where the poller count and the configuration become visible: metrics and API responses |
What Stays With You | What AWS does not take over and remains the user's responsibility |
Capped By | What limits how far the number of event pollers can go |
2.2 Where Defaults Come From, and How an Unset Field Is Written
TheIf You Set Nothing column gives the source of each value in parentheses. Four sources document the default values for Provisioned mode: the API reference (ProvisionedPollerConfig), the ESM overview page in the developer guide, the console procedure in the developer guide, and the Compute Blog from November 26, 2024.The first three sources directly state the default values. The Compute Blog describes how far the pollers grow when only one side is set for Kafka. This information regarding the scaling behavior is unique to the Compute Blog among the sources read for this article. Therefore, this article presents the scaling behavior as a description from the blog, differentiating it from the values listed in the reference documentation.
When the default value is used for the side that is not set, this article writes
Default with the value and its source. When no statement is found in the sources, this article writes No statement found with the date checked.2.3 Values from the Sources and Values Returned in This Configuration
The tables in Sections 3 through 8 hold only values from the sources. In the tables in Section 9, the values returned in this article's configuration are placed in a column separate from the statements in the sources. The two are not mixed in one column. The values in Section 9 are a record of that configuration on that day, not values that the sources guarantee. They cannot be used for performance comparisons.When a statement is not found, this article says that no statement was found, with the sources searched and the date checked. It does not say that the feature does not exist.
3. Event Pollers and the Two Modes
This section defines what an event poller is and the two modes.3.1 Event Pollers
The ESM overview page defines event pollers as follows:Within an event source mapping, resources called event pollers actively poll for new messages and invoke functions.
The same page specifies, for each type, the maximum volume that a single poller can handle.
For Amazon MSK and self-managed Apache Kafka: up to 5 MB/sec of throughput or up to 5 concurrent invokes
For Amazon SQS: up to 1 MB/sec of throughput or up to 10 concurrent invokes or up to 10 SQS polling API calls per second.
For SQS, the wording of the third limit differs between sources. The ESM overview page in the developer guide states
10 SQS polling API calls per second, while the Lambda console's help panel states 10 concurrent invokes or 10 invokes per second. The sources disagree. This article uses the developer guide's wording.Provisioned mode applies to the three event sources in the ESM overview page's sentence
Provisioned mode is available for Amazon MSK, self-managed Apache Kafka, and Amazon SQS event sources. ESM for Kinesis, DynamoDB Streams, Amazon MQ, and Amazon DocumentDB are not covered in this article.3.2 On-Demand Mode for Kafka
The Kafka page states that on-demand mode is the default. When a Kafka ESM is created, Lambda allocates a default number of pollers to process all partitions in the topic. The Kafka page describes the later scaling as follows:In one-minute intervals, Lambda evaluates the offset lag of all the partitions in the topic.
This autoscaling process of adding or removing event pollers occurs within three minutes of evaluation.
The same section also describes the behavior when the Lambda Event Function is throttled.
If your target Lambda Event Function is throttled, Lambda reduces the number of event pollers.
This action reduces the number of records that the ESM sends to the function. The Kafka page describes no setting with which users determine the number of pollers in on-demand mode.
3.3 Standard Scaling for SQS
The scaling page for SQS refers to the default mode asStandard mode (Default). In standard queues, when messages are present, Lambda begins processing them using five batches with five concurrent invocations. If messages remain, it adds up to 300 concurrent invocations every minute. A single ESM can process up to 1,250 invocations at the same time.This mode allows users to limit the number of invocations through the
MaximumConcurrency setting. The API reference for ScalingConfig specifies a range of 2–1,000 for MaximumConcurrency. Section 6 of the AWS Lambda Concurrency and Scaling Guide details how standard scaling and MaximumConcurrency work.3.4 Provisioned Mode
In Provisioned mode, users specify the minimum and maximum number of pollers, and Lambda adjusts the number of pollers within that range. The ESM overview page states, for both Kafka and SQS,Lambda scales the number of event pollers between your configured minimum and maximum. The Kafka and SQS pages state that pollers in Provisioned mode are dedicated to your event source mapping.The ESM overview page explains how the roles of the function's concurrency settings and Provisioned mode differ:
While concurrency settings give you control over the scaling of your function, provisioned mode gives you control over the throughput of your event source mapping. To ensure maximum performance, you might need to adjust both settings independently.
Function concurrency settings and the number of ESM pollers are separate configurations. However, the Kafka page states that the maximum number of event pollers an ESM can scale to depends on the function's concurrency settings (Section 6.2).
3.5 The Mode You Get When You Set Nothing Depends on Where You Create the ESM
The API reference documentation describesProvisionedPollerConfig as an optional element. The Kafka page and the SQS scaling page both give on-demand mode (standard mode for SQS) as the default mode. If an ESM is created without passing a ProvisionedPollerConfig, it will operate in the default mode.However, the What's New of November 21, 2025, about the integration for creating Kafka ESMs in Lambda from the Amazon MSK console, states:
To optimize latency and throughput, and to remove the need for networking setup, the integration uses Provisioned Mode for ESM as the recommended default.
Therefore, the integration in the Amazon MSK console creates ESMs with Provisioned mode as the recommended default. The mode used when no configuration is specified depends on where the ESM is created. The mode of an existing ESM can be verified in the response from
GetEventSourceMapping (Section 8) by checking ProvisionedPollerConfig.4. MinimumPollers and MaximumPollers
This section lists the valid ranges and default values for theProvisionedPollerConfig parameters, categorized by type and source.4.1 Ranges and Defaults
TheProvisionedPollerConfig in the API reference specifies two values as follows:The maximum number of event pollers this event source can scale up to. For Amazon SQS event source mappings, the accepted range is between 2 and 10,000, with a default of 200. For Amazon MSK and self-managed Apache Kafka event source mappings, the accepted range is between 1 and 2,000, with a default of 200.
The minimum number of event pollers this event source can scale down to. For Amazon SQS events source mappings, default is 2, and minimum 2 required. For Amazon MSK and self-managed Apache Kafka event source mappings, default is 1.
The second quotation keeps the original spelling
events source mappings. The Valid Range fields of the API reference give 1 to 200 for MinimumPollers and 1 to 10000 for MaximumPollers, without separating event sources.| Event source | Field | Accepted range | Default | Source |
|---|---|---|---|---|
| Kafka (Amazon MSK and self-managed Apache Kafka) | MinimumPollers | 1–200 | 1 | API reference; developer guide (overview page, Kafka page, console procedure) |
| Kafka (Amazon MSK and self-managed Apache Kafka) | MaximumPollers | 1–2,000 | 200 | API reference; developer guide (overview page, Kafka page, console procedure) |
| Amazon SQS | MinimumPollers | 2–200 | 2 | API reference; developer guide (overview page, SQS page, console procedure) |
| Amazon SQS | MaximumPollers | 2–10,000 | 200 | API reference; developer guide (overview page, SQS page, console procedure); What's New, August 3, 2026 |
For SQS,
MinimumPollers is 2 both as the default and as the minimum. For Kafka, MinimumPollers is 1 both as the default and as the minimum. For both types, the default of MaximumPollers is 200.4.2 The Maximum Must Be at Least the Minimum
The Kafka page statesMaximumPollers must be greater than or equal to MinimumPollers. The SQS page has the same sentence. For the maximum field, the console procedure states This value must be greater than or equal to your value for Minimum event pollers.4.3 The Sources Disagree on the SQS Maximum
The upper limit of the SQSMaximumPollers changed on August 3, 2026. The What's New of that date states:AWS Lambda now supports a maximum of 10,000 event pollers for Provisioned Mode for Amazon SQS event source mappings (ESMs), a 5x increase from the previous limit of 2,000.
As of October 19, 2026, some sources state the limit is 10,000, while others state it is 2,000. The sources disagree.
| Page | What it says for the SQS maximum | Checked |
|---|---|---|
Lambda API reference, ProvisionedPollerConfig | 2–10,000 | October 19, 2026 |
CloudFormation, ProvisionedPollerConfig | 2–10,000 (same text as the API reference) | October 19, 2026 |
| Developer guide, ESM overview page and the SQS page | 2–10,000 | October 19, 2026 |
| Developer guide, "Lambda parameters for Amazon SQS event source mappings" | maximum (2-2000) | October 19, 2026 |
| Developer guide, "Creating and configuring an Amazon SQS event source mapping" | maximum (2-2000) | October 19, 2026 |
| AWS SAM, SQS event source | a maximum of 2000 pollers | October 19, 2026 |
AWS CDK API reference (Python, ProvisionedPollerConfig) | Maximum value of 2000 | October 19, 2026 |
The limit of 2,000 is an upper bound prior to August 3, 2026. This article, following the What's New post and the API reference, gives the current limit as 10,000. This article does not state that pages indicating a limit of 2,000 are incorrect. In this article's configuration, the API accepted 2,500 for the SQS
MaximumPollers (Section 9.3).4.4 Turning Provisioned Mode On and Off
As the example of setting Provisioned mode, the Kafka page and the SQS page showupdate-event-source-mapping with MinimumPollers and MaximumPollers. To return to on-demand mode, they show an example that passes an empty ProvisionedPollerConfig.aws lambda update-event-source-mapping \
--uuid a1b2c3d4-5678-90ab-cdef-EXAMPLE11111 \
--provisioned-poller-config '{}'
The Compute Blog from November 26, 2024, states regarding Kafka:
Note that you must configure at least one of minimum or maximum event pollers to activate Provisioned Mode. For SQS, no statement in the same direction was found in the SQS page, the ESM overview page, or the Compute Blog from August 10, 2026 (as of October 19, 2026). In this article's configuration, after '{}' was passed to the SQS ESM, the response had no ProvisionedPollerConfig element (Section 9.2).The three properties of
ProvisionedPollerConfig in CloudFormation all indicate Update requires: No interruption. This refers to the classification of updates performed on the stack. The developer guide's examples for update-event-source-mapping show that values can be changed on existing ESMs.5. Setting Only One Side
This section details what happens when only one ofMinimumPollers or MaximumPollers is specified – specifically, what value is used for the side that is not set, and how far the number of pollers can increase, categorized by type.5.1 Kafka
The console procedure on the Kafka page states the following for the minimum and maximum fields:For Minimum event pollers, enter a value between 1 and 200. If you don't specify a value, Lambda chooses a default value of 1.
For Maximum event pollers, enter a value between 1 and 2,000. This value must be greater than or equal to your value for Minimum event pollers. If you don't specify a value, Lambda chooses a default value of 200.
The Compute Blog of November 26, 2024 describes how far the pollers grow:
When you configure only the minimum number of event pollers (‘Min-only’), your ESM allocates this minimum quantity and can dynamically scale up to a maximum. This maximum is determined by the OffsetLag and is limited by either the number of partitions or the default maximum event pollers, whichever is lower. When you configure only the maximum number of event pollers (“Max-only”), your ESM starts with one minimum poller by default, and can scale up to the maximum event pollers or number of partitions, whichever is lower.
| Setting | AWS Runs | You Set | If You Set Nothing | Where It Shows | What Stays With You | Capped By |
|---|---|---|---|---|---|---|
On-demand mode (no ProvisionedPollerConfig) | Event pollers that Lambda adds or removes after evaluating offset lag every minute | Nothing about pollers | The default mode (developer guide); the Amazon MSK console integration uses Provisioned mode as the recommended default (What's New, November 21, 2025) | OffsetLag | Function concurrency settings; the number of partitions | Decided by Lambda; no poller limit to set |
MinimumPollers only | Dedicated pollers, at least the minimum | MinimumPollers (1–200) | MaximumPollers default 200 (API reference; console procedure) | ProvisionedPollers; OffsetLag | Function concurrency settings; the number of partitions | The number of partitions or the default maximum, whichever is lower (Compute Blog, November 26, 2024); function concurrency settings (developer guide) |
MaximumPollers only | Dedicated pollers, starting from the default minimum | MaximumPollers (1–2,000) | MinimumPollers default 1 (API reference; console procedure) | ProvisionedPollers; OffsetLag | Function concurrency settings; the number of partitions | Your maximum or the number of partitions, whichever is lower (Compute Blog, November 26, 2024); function concurrency settings (developer guide) |
| Both | Dedicated pollers between the two values | MinimumPollers and MaximumPollers | Not applicable | ProvisionedPollers; OffsetLag | Function concurrency settings; the number of partitions | Your maximum; the number of partitions; function concurrency settings (developer guide) |
Empty object '{}' | Returns the ESM to on-demand mode (developer guide) | — | Not applicable | GetEventSourceMapping | — | Same as on-demand mode |
When only
MinimumPollers is set, the pollers start at the designated minimum value. According to the Compute Blog, they then grow, based on the OffsetLag, to the number of partitions or the default maximum (200), whichever is smaller. When only MaximumPollers is set, the pollers start at the default minimum (1) and grow to the designated maximum or the number of partitions, whichever is smaller.5.2 SQS
The console procedure on the SQS page states the following for the minimum and maximum fields:For Minimum event pollers, enter a value between 2 and 200. If you don't specify a value, Lambda chooses a default value of 2.
For Maximum event pollers, enter a value between 2 and 10,000. This value must be greater than or equal to your value for Minimum event pollers. If you don't specify a value, Lambda chooses a default value of 200.
The value of the side that is not set comes from this procedure and the defaults in the API reference. If only a minimum value is specified, the maximum defaults to 200; if only a maximum value is specified, the minimum defaults to 2.
For how far the pollers grow, no statement corresponding to the Kafka Compute Blog was found for SQS. The sources searched were the SQS page, the SQS scaling page, the ESM overview page, the Compute Blog of August 10, 2026, and the News Blog of November 14, 2025 (as of October 19, 2026). For both types, the ESM overview page only states that Lambda changes the number of pollers between the configured minimum and maximum.
When passing only one side of a value via the API, does the
ProvisionedPollerConfig in the responses from CreateEventSourceMapping or GetEventSourceMapping contain the value for the side that was not specified? No statement on this was found either. The sources searched were four pages of the API reference (ProvisionedPollerConfig, CreateEventSourceMapping, GetEventSourceMapping, UpdateEventSourceMapping), five pages of the developer guide, two pages related to CloudFormation, and two Compute Blog posts (as of October 19, 2026). The console procedure's "Lambda chooses a default value" refers to the value chosen when a field is left blank in the console and does not describe the shape of the API response. In this article's configuration, when only one side was sent, the response held the default value for the side that was not sent. However, when only the minimum was sent after only the maximum had been set, the maximum did not revert to the default of 200; the value set earlier remained (Section 9.2).| Setting | AWS Runs | You Set | If You Set Nothing | Where It Shows | What Stays With You | Capped By |
|---|---|---|---|---|---|---|
Standard mode (no ProvisionedPollerConfig) | Pollers that Lambda scales: five batches with five concurrent invokes at first, then up to 300 more concurrent invokes per minute | Optionally MaximumConcurrency (2–1,000) in ScalingConfig | The default mode (developer guide) | ConcurrentExecutions (function) | Function concurrency settings | 1,250 concurrent invokes per ESM, or MaximumConcurrency if set (developer guide) |
MinimumPollers only | Dedicated pollers, at least the minimum | MinimumPollers (2–200) | MaximumPollers default 200 (API reference; console procedure); how far the pollers grow: no statement found (as of October 19, 2026) | ProvisionedPollers; GetEventSourceMapping (Section 9) | Function concurrency settings | The default maximum of 200 (derived from the API reference default; Section 5.2) |
MaximumPollers only | Dedicated pollers, starting from the default minimum | MaximumPollers (2–10,000) | MinimumPollers default 2 (API reference; console procedure) | ProvisionedPollers; GetEventSourceMapping (Section 9) | Function concurrency settings | Your maximum |
| Both | Dedicated pollers between the two values | MinimumPollers and MaximumPollers | Not applicable | ProvisionedPollers | Function concurrency settings | Your maximum |
Empty object '{}' | Returns the ESM to standard mode (developer guide) | — | Not applicable | GetEventSourceMapping | — | Same as standard mode |
The
Capped By column for SQS is based on the ESM overview page's statement that Lambda changes the number of pollers between the minimum and maximum, and on the defaults in the API reference. On the SQS page, no statement corresponding to the Kafka page's depends on the function's concurrency settings was found. However, a Compute Blog post from August 10, 2026, recommends verifying whether the number of concurrent invocations a poller can handle fits in the account's overall concurrency quota (Section 6.2).5.3 Leaving a Poller Group
A Kafka ESM can leave its group whenPollerGroupName is an empty string. The Kafka page states remove an ESM from a group by passing an empty string ("") for PollerGroupName and provides the following example:aws lambda update-event-source-mapping \
--uuid a1b2c3d4-5678-90ab-cdef-EXAMPLE11111 \
--provisioned-poller-config '{
"MinimumPollers": 1,
"MaximumPollers": 10,
"PollerGroupName": ""
}'
This example passes
MinimumPollers and MaximumPollers together with the empty PollerGroupName. Section 7 covers the group itself.6. What Caps the Number of Event Pollers, and What Cannot Be Combined
This section lists what stops the number of event pollers before the maximum you set, and the settings that cannot be combined with Provisioned mode.
6.1 The Number of Kafka Partitions
The Kafka page places this sentence at the end of the paragraph on the ranges:
In addition, to maintain ordered processing within partitions, Lambda caps the MaximumPollers to the number of partitions in the topic.
The reason for this limit is to preserve the order within each partition. Even if you increase the value of
MaximumPollers, the number of pollers will not exceed the number of partitions. This site's What Breaks Ordering, Service by Service on AWS discusses the relationship between partitions and order in detail.6.2 Function Concurrency Settings
The best practices section of the Kafka page states:To maintain ordered processing within a partition, Lambda limits the maximum event pollers to the number of partitions in the topic. Additionally, the maximum event pollers your event source mapping can scale to depends on the function's concurrency settings.
Since pollers invoke functions, how far the function can run concurrently also affects the number of pollers. As Section 3.2 describes, in on-demand mode Lambda reduces the number of pollers when the function is throttled.
Regarding SQS, a Compute Blog post from August 10, 2026, recommends verifying your account's concurrent execution quota.
Provisioned pollers invoke your Lambda function concurrently. Verify that your account’s concurrent execution quota accommodates the maximum concurrency your pollers can drive:
The blog post includes the formula
MaxConcurrency = MaximumPollers × 10 following this statement. Because one poller handles up to 10 concurrent invocations, ten times MaximumPollers is a rough upper bound on the concurrency the pollers can drive; that is what the formula says. The formula is from the blog post, not a statement in the developer guide.6.3 SQS MaximumConcurrency Cannot Be Combined with Provisioned Mode
For SQS, Provisioned mode and maximum concurrency cannot be used together. Five sources state this.| Source | Statement |
|---|---|
| Developer guide, ESM overview page | Note that for Amazon SQS event sources, the maximum concurrency setting cannot be used with provisioned mode. |
| Developer guide, SQS page (Note) | Provisioned mode cannot be used in conjunction with the maximum concurrency setting. |
| Developer guide, SQS scaling page | You can control the scaling behavior of your Amazon SQS event source mappings either through maximum concurrency settings or by enabling provisioned mode. These are mutually exclusive options. |
| Developer guide, SQS ESM creation and configuration page (Note) | You cannot use Provisioned Mode and Maximum concurrency together. |
CloudFormation, ScalingConfig | You cannot use ProvisionedPollerConfig while using ScalingConfig. |
For concurrency in Provisioned mode, the ESM overview page states
When using provisioned mode, you control concurrency through the maximum event pollers setting. The SQS page's Note states the same thing as you control maximum concurrency through the maximum number of event pollers.The API reference for
ScalingConfig states To remove the configuration, pass an empty value. In this article's configuration, adding MaximumConcurrency to an ESM in Provisioned mode and passing both settings in a single call both returned an InvalidParameterValueException with the same message (Section 9.3).6.4 What Else Changes in Provisioned Mode for Kafka
Kafka's Provisioned mode also brings changes to other aspects. The Kafka page notes:When using provisioned mode, you don't need to create AWS PrivateLink VPC endpoints or grant the associated permissions as part of your network configuration.
The same page also lists additional configuration options available for Kafka ESMs in Provisioned mode, including retry settings and the option to designate Kafka topics as failure destinations. These are outside the scope of this article. This site's What Breaks Ordering, Service by Service on AWS discusses the relationship between retries and ordering.
7. Poller Groups with PollerGroupName
This section covers the setting that groups Kafka ESMs withPollerGroupName.7.1 What a Poller Group Combines
The API reference describesPollerGroupName as follows. The second sentence, on pricing, is outside the scope of this article and is not quoted. This article quotes the first and third sentences.(Amazon MSK and self-managed Apache Kafka) The name of the provisioned poller group. Use this option to group multiple ESMs within the event source's VPC to share Event Poller Unit (EPU) capacity.
You can group up to 100 ESMs per poller group and aggregate maximum pollers across all ESMs in a group cannot exceed 2000.
According to the API reference, the length of
PollerGroupName must be between 0 and 128 characters, and the pattern is [a-zA-Z0-9-_]*. According to the Kafka page, assigning the same PollerGroupName to multiple ESMs in the same Amazon VPC allows those ESMs to share resources.
7.2 Constraints on a Group
| Item | Value | Source |
|---|---|---|
| Event sources | Amazon MSK and self-managed Apache Kafka only | API reference |
| Scope | ESMs within the event source's VPC | API reference |
| ESMs per group | Up to 100 | API reference; developer guide (Kafka page) |
Sum of MaximumPollers in a group | Cannot exceed 2,000 | API reference; developer guide (Kafka page) |
| Leaving a group | Pass an empty string for PollerGroupName | Developer guide (Kafka page) |
The sum of the
MaximumPollers values for all ESMs within a group cannot exceed 2,000. For example, you can include 10 ESMs, each with a MaximumPollers value of 200, for a total of 2,000. Adding an 11th ESM with the same maximum makes the sum exceed 2,000. This article did not check what is returned when such a setting is passed.In this article's configuration, the API rejected a
PollerGroupName passed to an SQS ESM with InvalidParameterValueException (Section 9.3).7.3 Three Considerations in the Developer Guide
The Kafka page lists three considerations for grouping. The first, application boundary, is to group ESMs that belong to the same application or service. The second and third considerations are as follows:Traffic pattern – Avoid grouping ESMs with high throughput and spiky traffic pattern, as this might lead to resource contention.
Blast radius – Consider the impact if the shared infrastructure experiences issues. All ESMs in the same group are affected by shared resource limitations. For mission-critical workloads, you might want to use separate groups or dedicated ESMs.
Grouping means that ESMs share resources in the same group. The Kafka page notes that grouping ESMs together can result in those ESMs in the group impacting one another.
8. Where It Shows
This section lists where the number of event pollers and the configuration can be checked.8.1 ProvisionedPollers
The Kafka page and the SQS page say to monitor the use of event pollers in Provisioned mode with theProvisionedPollers metric. Four sources define this metric in four different ways. The sources disagree.| Source | Definition of ProvisionedPollers |
|---|---|
| Developer guide, "Types of metrics for Lambda Event Functions" | the number of event pollers that are actively running (view with MAX) |
| Compute Blog, November 26, 2024 (Kafka) | the number of event pollers that actively processed at least one event in the last 5-minute window |
| Compute Blog, August 10, 2026 (SQS) | Current number of provisioned event pollers allocated |
| News Blog, November 14, 2025 (SQS) | the number of active event pollers processing events in one-minute windows |
This article does not state that any of these definitions are incorrect. The developer guide says to view this metric with
MAX.8.2 Other Metrics
The metrics types page lists two metrics for ESMs in Provisioned mode, each marked in parentheses for Kafka event sources.EventPollerThroughputInBytes is the total size of the records that the pollers read from the event source, and is viewed using SUM. EventPollerUnit is the name of a metric for Kafka ESMs, indicated as (Amazon MSK and self-managed Apache Kafka event sources only), and is also viewed using SUM.OffsetLag is the difference between the last record written to a Kafka topic and the last record processed by the function's consumer group. The metrics types page states that this metric measures the offset difference per topic. Both Kafka on-demand mode and Provisioned mode adjust the number of pollers based on the offset difference.Many of the ESM metrics become available when you select and enable a metric group. The metrics types page lists three metric groups:
EventCount, ErrorCount, and KafkaMetrics, and states You must opt-in the metric group to receive metrics related metrics. On the same page, ProvisionedPollers is introduced with if your event source mapping is in provisioned mode, Lambda provides the following metric. It is unclear from this statement whether enabling a metric group is required for ProvisionedPollers. In the SQS configuration described in this article, ProvisionedPollers appeared without enabling a metric group (Section 9.4).8.3 GetEventSourceMapping
The ESM configuration can be checked in theProvisionedPollerConfig of the GetEventSourceMapping response. The API reference describes this element of the response as (Amazon SQS, Amazon MSK, and self-managed Apache Kafka only) The provisioned mode configuration for the event source. The MaximumConcurrency setting for SQS is in the ScalingConfig element of the same response.9. What This Configuration Returned
This section documents the responses obtained using the SQS configuration created specifically for this article, changing theProvisionedPollerConfig step by step. The statements in the sources and the values returned in this configuration are placed in separate columns. Every value is a record of this configuration on that day, not a value the sources guarantee. No Kafka (Amazon MSK) configuration was created.9.1 Configuration and Steps
The configuration was built in the US East (N. Virginia) Region (us-east-1) on October 19, 2026. CloudFormation created one standard SQS queue and one Event Function that simply returns the received batch (python3.13, arm64, 128 MB). The ESM was created with the AWS CLI (version 2.37.12), with a batch size of 10. No metric group was enabled for the ESM.The procedure followed these steps. After each step that succeeded, the response from
get-event-source-mapping was saved, and the next step began after the ESM's state returned to Enabled.- C1: The ESM was created with only
{"MinimumPollers": 3}passed, and was left for 5 minutes. - C2: The ESM was returned to on-demand mode (SQS standard mode) by passing
'{}'. - C3: Only
{"MaximumPollers": 6}was passed, and the ESM was left for 5 minutes. - C4: From the C3 state, only
{"MinimumPollers": 3}was passed. - C5:
{"MinimumPollers": 3, "MaximumPollers": 6}was passed, 100 messages were sent, and the ESM was left for 5 minutes. - E1–E6: Values that the sources describe as out of range or not combinable were passed (Section 9.3).
- F1, F2: After passing
'{}', the ESM was deleted.
The UUID, ARN, account ID, and queue URL for the ESM have been redacted.
9.2 Responses When Only One Side Was Sent
| Step | Sent in ProvisionedPollerConfig | What the documentation states | ProvisionedPollerConfig returned in this configuration |
|---|---|---|---|
C1 (create-event-source-mapping) | {"MinimumPollers": 3} | MaximumPollers default 200 (API reference; console procedure); the API response is not described | {"MinimumPollers": 3, "MaximumPollers": 200} |
| C2 | {} | Returns the ESM to the default mode (developer guide) | No ProvisionedPollerConfig element in the response |
| C3 | {"MaximumPollers": 6} | MinimumPollers default 2 (API reference; console procedure); the API response is not described | {"MinimumPollers": 2, "MaximumPollers": 6} |
| C4 (after C3) | {"MinimumPollers": 3} | MaximumPollers default 200 (API reference; console procedure); no statement found on what happens to a value set earlier | {"MinimumPollers": 3, "MaximumPollers": 6} |
| C5 | {"MinimumPollers": 3, "MaximumPollers": 6} | — | {"MinimumPollers": 3, "MaximumPollers": 6} |
In C1 and C3, the response held, for the side that was not sent, the default value stated in the API reference and the console procedure. The
create-event-source-mapping response in C1 and the update-event-source-mapping response in C3 were each the same as the subsequent get-event-source-mapping response.In C4, even though only the
MinimumPollers value was provided, MaximumPollers did not revert to the default value of 200, but instead returned the value of 6 that was set in C3. In this configuration, when the ESM already had Provisioned mode values, sending only the minimum left the maximum at its earlier value. The reverse direction, sending only the maximum, was not tried. No statement describing this behavior was found in the sources this article read (as of October 19, 2026).In C2, the response after providing
'{}' lacked the ProvisionedPollerConfig element entirely. Whether an ESM that was never set to Provisioned mode returns the same shape was not checked.9.3 Values That Were Not Accepted
| Step | Sent | What the documentation states | Returned in this configuration |
|---|---|---|---|
| E1 | {"MinimumPollers": 1, "MaximumPollers": 6} | SQS MinimumPollers 2–200 (API reference) | InvalidParameterValueException: MinimumPollers should be between [2, 200] and MaximumPollers should be between [MinimumPollers, 10000] |
| E2 | {"MinimumPollers": 5, "MaximumPollers": 3} | MaximumPollers must be greater than or equal to MinimumPollers. (developer guide) | InvalidParameterValueException: MinimumPollers must be less than or equal to MaximumPollers. |
| E3 | {"MinimumPollers": 3, "MaximumPollers": 2500} | 2–10,000 (API reference); maximum (2-2000) (two developer guide pages) | Accepted; the response held {"MinimumPollers": 3, "MaximumPollers": 2500} |
| E4 | {"MinimumPollers": 3, "MaximumPollers": 6, "PollerGroupName": "b62-03-group"} | (Amazon MSK and self-managed Apache Kafka) (API reference) | InvalidParameterValueException: Invalid parameter: pollerGroupName |
| E5 | --scaling-config MaximumConcurrency=10 on an ESM in Provisioned mode | Cannot be used together (developer guide; CloudFormation) | InvalidParameterValueException: Provisioned mode for SQS ESM does not support maximum concurrency feature. Use MaximumPollers as the control mechanism and remove maximum concurrency. |
| E6 | MaximumConcurrency=10 and {"MinimumPollers": 3} in one call, from the default mode | Cannot be used together (developer guide; CloudFormation) | The same message as E5 |
The E1 error message gives
10000 as the upper end of the SQS MaximumPollers range. In E3, the API accepted 2,500, which is above 2,000. The API in this configuration returned values that match the sources in the table in Section 4.3 that state 10,000. Right after E3, MaximumPollers was set back to 6.The E5 and E6 messages ask the user to control concurrency with
MaximumPollers in Provisioned mode and to remove the maximum concurrency setting. This points the same way as the developer guide's sentences in Section 6.3.9.4 ProvisionedPollers
ProvisionedPollers appeared in the AWS/Lambda namespace although no metric group was enabled. The dimension was EventSourceMappingUUID. Among the metrics with the EventSourceMappingUUID dimension for this ESM, ProvisionedPollers was the only one. The Maximum for each 1-minute period was as follows:| Period (UTC, October 19, 2026) | Configuration at the time | ProvisionedPollers (Maximum, 1 minute) |
|---|---|---|
| 01:15–01:19 | C1: MinimumPollers 3, MaximumPollers 200 | 3 |
| 01:20–01:22 | C2: default mode | No data points |
| 01:23–01:27 | C3: MinimumPollers 2, MaximumPollers 6 | 2 |
| 01:28–01:35 | C4 and C5: MinimumPollers 3, MaximumPollers 6; 100 messages sent at the start of C5 | 3 |
In each interval, the observed value matched the configured minimum. Even after sending 100 messages, the maximum value remained at 3. This value does not show which of the four definitions in Section 8.1 applies. Because it is a short record from a small configuration, it cannot be used to judge how the pollers scale or how fast processing is.
9.5 What This Record Supports, and What It Does Not
What the record supports is that this configuration inus-east-1 returned the following on October 19, 2026. When only one side was sent, the side that was not sent held the default value. However, when only the minimum was sent to an ESM that already had values, the earlier maximum remained. The API accepted a value of 2,500 for the SQS MaximumPollers. It rejected PollerGroupName on the SQS ESM, and MaximumConcurrency together with Provisioned mode.What it does not support is that this shape of response is guaranteed in the documentation, or that it is the same in other Regions or in the future, or that a Kafka ESM behaves the same. This article did not check the response when only one side is sent to a Kafka ESM.
10. Failure Modes
This section lists the failure modes that follow from the sources in this article.- Specifying only the minimum, assuming the maximum is unlimited: If the maximum is not specified, the default value of 200 is used (API reference, console procedure). In Kafka, this is further limited by the number of partitions (Sections 4.1 and 5.1).
- Setting the minimum for SQS to 1: SQS's
MinimumPollershas a default and minimum value of 2. Directly applying a Kafka value results in an out-of-range setting (Section 4.1). - Assuming that raising the Kafka maximum above the number of partitions adds parallelism: Lambda caps
MaximumPollersat the number of partitions to maintain order within each partition (Section 6.1). - Assuming that using Provisioned mode means function concurrency settings are irrelevant: The Kafka page states that how far the pollers can scale depends on the function's concurrency settings. The ESM overview page states that the two settings might need to be adjusted independently (Sections 3.4 and 6.2).
- Turning on Provisioned mode for SQS while leaving
MaximumConcurrencyin place: These two settings are incompatible. When using Provisioned mode, concurrency is limited byMaximumPollers(Section 6.3). - Putting an SQS ESM in a poller group:
PollerGroupNameis a setting for Kafka ESMs only (Section 7.1). - Reading the SQS maximum from a page that states 2,000: Since August 3, 2026, the limit is 10,000. Some developer guide pages, as well as pages for AWS SAM and AWS CDK, still state a limit of 2,000 as of October 19, 2026 (Section 4.3).
- Assuming that if nothing is specified, it will always be in on-demand mode: The Amazon MSK console integration creates the ESM with Provisioned mode as the recommended default (Section 3.5).
- Assuming that specifying only the minimum will revert the maximum to the default value of 200: In the SQS configuration described in this article, setting only the minimum after specifying the maximum resulted in the maximum retaining its previous value. When providing only one of these values, verify both values in the response's
ProvisionedPollerConfig(Section 9.2). - Assuming that Provisioned mode and provisioned concurrency are the same thing: The former refers to the poller configuration for ESM, while the latter refers to the function execution environment settings (Section 1.4).
11. Frequently Asked Questions About Lambda Event Pollers in Provisioned Mode
Q1. If you set only MinimumPollers, what does MaximumPollers become?
The default value of 200 is used. The API reference states that the default value for MaximumPollers is 200 for both Kafka and SQS. The console procedure states that if the field is left blank, Lambda chooses the default value of 200. According to the Compute Blog from November 26, 2024, for Kafka the pollers grow up to the number of partitions or the default maximum, whichever is smaller. No statement on the shape of the API response was found in the documentation. In the SQS configuration described in this article, when creating an ESM and providing only the MinimumPollers value, the MaximumPollers value in the response was 200. When only the MaximumPollers value was initially set, and then only the MinimumPollers value was provided, the previously set maximum value remained (as described in Section 9.2).Q2. If you set only MaximumPollers, what does MinimumPollers become?
In Kafka, the value is 1, while in SQS, it is 2 (as indicated in the API reference and console procedure). According to a Compute Blog post from November 26, 2024, in Kafka, the number of pollers starts at 1 and grows up to the specified maximum or the number of partitions, whichever is smaller. In the SQS configuration described in this article, when only the maximum is provided, the MinimumPollers value in the response was 2 (Section 9.2).Q3. Is the SQS MaximumPollers limit 2,000 or 10,000?
As of October 19, 2026, the limit is 10,000. The What's New of August 3, 2026 states that the limit was raised from 2,000 to 10,000, and the API reference also states 2–10,000. However, some pages in the developer guide, as well as pages related to AWS SAM and AWS CDK, state the limit is 2,000. The sources disagree (Section 4.3).Q4. Can a Kafka ESM have more event pollers than partitions?
No. The Kafka page states that Lambda capsMaximumPollers at the number of partitions in the topic to maintain order within partitions (Section 6.1).Q5. Can SQS use MaximumConcurrency together with Provisioned mode?
No, it cannot. The developer guide (in four separate pages) and CloudFormation documentation state that these two settings are not compatible. When using Provisioned mode, concurrency is controlled using the MaximumPollers setting (Section 6.3).Q6. Can PollerGroupName be used on an SQS ESM?
The API reference's description begins with (Amazon MSK and self-managed Apache Kafka), and SQS is not among the sources it names. In this article's configuration, the API rejected it on an SQS ESM with Invalid parameter: pollerGroupName (Sections 7.1 and 9.3).Q7. How do you turn off Provisioned mode?
Pass an emptyProvisionedPollerConfig ('{}') with update-event-source-mapping. The Kafka page and the SQS page both show this as the example for returning to on-demand mode (Section 4.4).Q8. When were the statements in this article checked?
They were checked on October 19, 2026. The limit forMaximumPollers in SQS changed on August 3, 2026, and if the pages that state 2,000 are revised, the table in Section 4.3 will change. The results in Section 9 are also recorded as of October 19, 2026.12. Summary
- Ranges and defaults differ by type: Kafka's
MinimumPollersranges from 1 to 200, with a default of 1.MaximumPollersranges from 1 to 2,000, with a default of 200. SQS'sMinimumPollersranges from 2 to 200, with a default of 2.MaximumPollersranges from 2 to 10,000, with a default of 200 (Section 4.1). - If you set only one side, the other side uses the default: The API reference and the console procedure document the default values. The Compute Blog of November 26, 2024 describes how far the Kafka pollers grow; no statement was found on how far the SQS pollers grow or on the shape of the API response. In this article's SQS configuration, the default was returned, but sending only the minimum to an ESM that already had values kept the earlier maximum (Sections 5 and 9.2).
- The maximum is not the only cap on the pollers: In Kafka,
MaximumPollersis capped at the number of partitions, and how far the pollers can scale also depends on the function's concurrency settings (Sections 6.1 and 6.2). - SQS Provisioned mode cannot be used with
MaximumConcurrency: When using Provisioned mode, useMaximumPollersto limit concurrent execution (Section 6.3). PollerGroupNameis for Kafka only: A single group can have up to 100 ESMs, and the totalMaximumPollersfor that group is limited to 2,000. An empty string removes an ESM from its group (Section 7).- The sources disagree on the SQS maximum, 2,000 or 10,000: As of October 19, 2026, the limit is 10,000, in effect since August 3, 2026 (Section 4.3).
13. References
- How Lambda processes records from stream and queue-based event sources - AWS Lambda Developer Guide
- Apache Kafka event poller scaling modes in Lambda - AWS Lambda Developer Guide
- Using Lambda with Amazon SQS - AWS Lambda Developer Guide
- Configuring scaling behavior for SQS event source mappings - AWS Lambda Developer Guide
- Lambda parameters for Amazon SQS event source mappings - AWS Lambda Developer Guide
- Creating and configuring an Amazon SQS event source mapping - AWS Lambda Developer Guide
- Types of metrics for Lambda Event Functions - AWS Lambda Developer Guide
- ProvisionedPollerConfig - AWS Lambda API Reference
- CreateEventSourceMapping - AWS Lambda API Reference
- GetEventSourceMapping - AWS Lambda API Reference
- UpdateEventSourceMapping - AWS Lambda API Reference
- ScalingConfig - AWS Lambda API Reference
- AWS::Lambda::EventSourceMapping - AWS CloudFormation Template Reference
- AWS::Lambda::EventSourceMapping ProvisionedPollerConfig - AWS CloudFormation Template Reference
- SQS - AWS Serverless Application Model Developer Guide
- ProvisionedPollerConfig - AWS CDK Python API Reference
- Provisioned mode - AWS Lambda console help panel
- AWS Lambda announces Provisioned Mode for Kafka event source mappings (ESMs) - AWS What's New
- AWS Lambda announces low latency processing for Kafka events - AWS What's New
- AWS Lambda announces Provisioned Mode for SQS event source mapping (ESM) - AWS What's New
- Announcing AWS Lambda Kafka event source mapping integration in Amazon MSK Console - AWS What's New
- AWS Lambda Provisioned Mode for Kafka event source mappings (ESMs) now available in AWS Asia Pacific (Taipei) and AWS GovCloud (US) Regions - AWS What's New
- AWS Lambda Provisioned Mode for Amazon SQS event source mappings now supports up to 10,000 event pollers - AWS What's New
- AWS Lambda announces Lambda Web Functions - AWS What's New
- Introducing Provisioned Mode for Kafka Event Source Mappings with AWS Lambda - AWS Compute Blog
- Low-latency, high-throughput SQS event processing with AWS Lambda provisioned mode - AWS Compute Blog
- AWS Lambda enhances event processing with provisioned mode for SQS event-source mapping - AWS News Blog
References:
Tech Blog with curated related content
Written by Hidekazu Konishi