How AWS Lambda Authenticates to Apache Kafka - Where Each Event Source Mapping Method Keeps the Credential, Who Refreshes the Token, and What the Brokers Must Trust

First Published: 2026-10-14
Last Updated: 2026-10-14

AWS Lambda's event source mapping (ESM) reads records from Apache Kafka topics and invokes Lambda functions. To read these records, Lambda must authenticate with the Kafka broker. On October 9, 2026, the "What's New" announcement detailed that ESM for self-managed Apache Kafka now supports OAuth authentication. The authentication chapter of the Lambda developer guide states:

Lambda requests an access token from your OAuth 2.0 identity provider and presents that token to your Kafka brokers. Lambda refreshes the token before it expires.

The same chapter also describes two methods that do not require storing secrets. One is IAM_AUTH, and the other is IAM_OAUTHBEARER_AUTH. Regarding IAM_OAUTHBEARER_AUTH, the chapter states:

Lambda requests a short-lived token for your function's execution role with the sts:GetWebIdentityToken action, so you don't store any credentials in Secrets Manager.

As the number of methods increases, the configuration required from users, as well as what the broker must provide, varies depending on the method. Some methods require storing credentials in AWS Secrets Manager, while others do not. In some methods Lambda refreshes a token, and in others there is no token to refresh. This article provides a line-by-line overview of Lambda's Kafka ESM authentication methods, outlining where credentials are stored, who refreshes the token, what must be presented to the broker, what users must configure, and what the broker must trust. This is based on information from the Lambda developer guide, API reference, AWS CloudFormation and AWS SAM reference, AWS STS API reference, "What's New," and Apache Kafka documentation. Where the sources disagree, and where a statement could not be found, the article shows both statements with their sources and verification dates. This article does not recommend a specific method. This article does not create any AWS resources.

Related articles on this site:

1. Authentication in Lambda's Kafka Event Source Mappings — Scope and Verification Date
2. Two Chapters, and How to Read the Tables
3. Methods That Keep a Secret in Secrets Manager — SASL/SCRAM, SASL/PLAIN, and mTLS
4. OAuth 2.0 — OAUTHBEARER_AUTH
5. Methods That Store No Secret — IAM_AUTH and IAM_OAUTHBEARER_AUTH
6. The Amazon MSK Event Source Mapping — Four Methods and How Lambda Chooses a Broker
7. Execution Role Permissions and the Combination Rules
8. Where the Sources Disagree and Where No Statement Was Found
9. Frequently Asked Questions about Lambda's Kafka Authentication
10. Summary
11. References

1. Authentication in Lambda's Kafka Event Source Mappings — Scope and Verification Date

This section first shows what was added. It then defines the scope of this article, the date of verification, the consulted materials, and the terminology used within this article.

1.1 What Was Added

The "What's New" announcement, "AWS Lambda supports OAuth authentication for self-managed Apache Kafka event sources," has a postDateTime of 2026-10-09T16:30:00Z. This article will refer to that date as October 9, 2026. The announcement details the support for OAuth authentication for self-managed Apache Kafka event sources. It provides self-managed examples, including Kafka clusters managed by users themselves, as well as Confluent Cloud, Aiven, and Redpanda. The body of the announcement describes the earlier state as follows:

Previously, Kafka ESM supported only SASL/PLAIN, SASL/SCRAM, and mutual TLS (mTLS) as authentication methods

The self-managed authentication chapter, as read on October 13, 2026, describes two more methods besides OAuth: IAM_AUTH and IAM_OAUTHBEARER_AUTH. These two methods are not mentioned in the announcement from October 9, 2026. No statement was found on when these two methods were added (Section 8.3). This article treats these two methods as methods that the chapter describes as of October 13, 2026.

1.2 Scope of This Article, and What It Does Not Cover

This article focuses on the following:

For the self-managed Apache Kafka ESM and the Amazon MSK ESM: for each authentication method, where the credentials are kept, who refreshes the token, how it is presented to the brokers, what the user configures, and what the broker side requires (Sections 3 through 6).
The rules for validating combinations of SourceAccessConfigurations entries, and the permissions the execution role needs for each method (Section 7).
Places where the sources disagree, and statements that were not found (Section 8).

This article does not cover the following:

The design of the caller for GetWebIdentityToken and the implementation for validating tokens issued by AWS STS (including issuer URLs, JWKS, claims, and revocation). This is covered in the existing article AWS IAM Outbound Identity Federation.
IAM authentication from applications to Amazon MSK. This is covered in Section 3.4 of the existing article IAM Authentication to Databases, Caches, and Streams on AWS.
The use of extended key usage (EKU) for mTLS client certificates, and public certificate authorities. This is covered in Section 4.4 of the existing article When Public Certificates Stop Carrying clientAuth.
Scaling and Provisioned mode for ESM. This is covered in the existing article AWS Lambda Concurrency and Scaling Guide.
How Lambda passes the execution role's credentials to the function code, and the Lambda metadata endpoint. This is covered in the companion article Metadata Endpoints Across AWS Compute.
Kafka sources within Amazon EventBridge Pipes, and the steps for building Kafka brokers.

While MSK Replicator can connect to external Kafka clusters using OAuth, this article addresses it with a single paragraph for comparison purposes (Section 5.5).

1.3 Verification Date and Sources Read

The information presented in this article was verified on October 13, 2026, using the following materials:

Lambda Developer Guide: Chapters on Self-Managed Apache Kafka (entry page, authentication methods page, execution role permissions page, networking page, ESM creation page, parameter settings overview page), and chapters on Amazon MSK (entry page, authentication methods page, permissions page, ESM creation page, parameter settings overview page, cross-account configurations page), the page on using Kafka topics as on-failure destinations, the page on troubleshooting Kafka ESM, the page on Kafka ESM logs, and the document history.
Lambda API Reference: SourceAccessConfiguration, CreateEventSourceMapping, UpdateEventSourceMapping, GetEventSourceMapping.
AWS CloudFormation: AWS::Lambda::EventSourceMapping and its SourceAccessConfiguration properties.
AWS SAM: Event properties for SelfManagedKafka and MSK.
AWS STS API Reference: GetWebIdentityToken.
IAM User Guide: Page on outbound identity federation.
Amazon MSK Developer Guide: Page on prerequisites for self-managed Kafka clusters using MSK Replicator.
What's New: Announcements from October 9, 2026, regarding OAuth, and announcements from October 7, 2021, regarding Amazon MSK ESM IAM authentication (the postDateTime was 2021-10-07T07:00:00Z).
Apache Kafka: Broker Configs page from version 4.3 documentation. The latest release on the verification date was 4.3.1.

Dates listed under "What's New" refer to the postDateTime of the announcements. For user guides and API reference pages, the date of verification is used, as those pages do not have specific publication dates.

1.4 Words That Share a Spelling but Refer to Different Things

In this article, the same word is used to refer to different things. To avoid misreading, this article distinguishes them up front.

Tokens (the four types distinguished in this article): (1) OAuth access tokens. The user's OAuth 2.0 identity provider (IdP) issues them, and Lambda refreshes them. (2) AWS web identity tokens. These are signed JSON Web Tokens (JWTs) returned by AWS STS as a response to GetWebIdentityToken. (3) JWT bearer assertions. In the OAuth JWT bearer grant type, these are JWTs that Lambda signs with the private key in the secret and sends to the IdP's token endpoint. (4) Tokens presented to the broker. Lambda presents these to the broker over SASL/OAUTHBEARER. Specifically, OAUTHBEARER_AUTH uses (1) OAuth access tokens, while IAM_OAUTHBEARER_AUTH uses (2) AWS web identity tokens. Throughout this article, when the term "token" is used, the specific type being referenced will be explicitly stated.
IAM (the four types distinguished in this article): (1) Amazon MSK ESM IAM authentication. When no secrets are provided, Lambda connects to the MSK cluster using the execution role. (2) Self-managed ESM IAM_AUTH. This is the method for registering, as self-managed Kafka, an Amazon MSK cluster that the user reaches through their own networking setup. (3) Self-managed ESM IAM_OAUTHBEARER_AUTH. This involves presenting the AWS web identity token to the broker using SASL/OAUTHBEARER. (4) SASL mechanism for applications authenticating with Amazon MSK using IAM (specifically, SASL_OAUTHBEARER and AWS_MSK_IAM). (4) is outside the scope of this article and is covered in Section 3.4 of the existing IAM authentication article.
JWT bearer (the two types distinguished in this article): (1) Lambda's OAuth JWT bearer grant type. Lambda signs the assertion using the secret's oauthPrivateKey. (2) MSK Replicator's IAM JWT bearer. MSK Replicator obtains a signed JWT using the AWS identity of the service's execution role and sends it as an assertion to the IdP's token endpoint (see Section 5.5).
Secrets: When the term "secrets" is used in this article, it refers to secrets stored in AWS Secrets Manager. The term "client secret" specifically refers to the client secret used in OAuth.
Self-managed: This term is used in the Lambda developer guide. The entry page for self-managed Apache Kafka states that self-managed clusters include Kafka clusters hosted outside AWS, and gives Confluent Cloud and Redpanda as examples. As described in Section 5.1, it is also possible to register an Amazon MSK cluster as self-managed. This article calls Lambda's self-managed Apache Kafka chapter and its ESM the self-managed chapter and the self-managed ESM.
Refresh: In this article, refresh means obtaining a new token before a short-lived token expires. Replacing the value of a Secrets Manager secret is called replacing.

2. Two Chapters, and How to Read the Tables

This section highlights how the Lambda developer guide divides Kafka authentication into two chapters and defines the columns and values presented in the tables within this article.

2.1 Self-Managed Chapter and Amazon MSK Chapter

The Lambda developer guide describes Kafka's ESM authentication in two separate chapters: one for self-managed clusters (Configuring cluster authentication methods in Lambda) and another for Amazon MSK (Configuring Amazon MSK cluster authentication methods in Lambda). The sets of methods listed in the two chapters are different.

The self-managed chapter includes five sections detailing authentication methods, along with three sections describing secret formats.

SASL/SCRAM authentication (This section includes a paragraph on SASL/PLAIN.)
Mutual TLS authentication
OAuth 2.0 authentication
IAM authentication (IAM_AUTH)
IAM authentication with SASL/OAUTHBEARER (IAM_OAUTHBEARER_AUTH)

The Amazon MSK chapter includes five sections. The first four detail authentication methods, while the final section describes how Lambda selects a bootstrap broker.

Unauthenticated access
SASL/SCRAM authentication
Mutual TLS authentication
IAM authentication
How Lambda chooses a bootstrap broker

This article treats the methods in the self-managed chapter as seven rows, with SASL/SCRAM and SASL/PLAIN in separate rows and OAuth split into one row per grant type. It treats the methods in the Amazon MSK chapter as four rows, including unauthenticated access. The number of methods is not consolidated into a single count, as the count varies depending on how they are counted.

Figure 1 lists, for each method in the self-managed chapter, where Lambda retrieves credentials and how it presents them to the broker.

Where Each Lambda Kafka Authentication Method Keeps the Credential

2.2 `SourceAccessConfigurations` `Type`

Authentication settings for the ESM go in the SourceAccessConfigurations parameter of CreateEventSourceMapping. SourceAccessConfigurations is an array of SourceAccessConfiguration objects, with each object containing two fields: Type and URI. The Type field indicates the authentication method or the kind of VPC component, while the URI field often contains the ARN of a Secrets Manager secret.

The Type values for each method in the self-managed chapter are as follows:

SASL/SCRAM: SASL_SCRAM_256_AUTH or SASL_SCRAM_512_AUTH
SASL/PLAIN: BASIC_AUTH
mTLS: CLIENT_CERTIFICATE_TLS_AUTH
OAuth 2.0: OAUTHBEARER_AUTH
IAM: IAM_AUTH
IAM with SASL/OAUTHBEARER: IAM_OAUTHBEARER_AUTH

In addition to these, there are other types associated with specific methods. These include SERVER_ROOT_CA_CERTIFICATE (Section 3.3), which points to the root CA certificate for validating the broker's server certificate, and four types associated with OAuth: OAUTHBEARER_SCOPE, OAUTHBEARER_AUDIENCE, OAUTHBEARER_LOGICAL_CLUSTER, and OAUTHBEARER_IDENTITY_POOL (Section 4.3). The number of values that Type can take differs between the API reference and CloudFormation (see item 1 in Section 8.1).

2.3 Common Columns

The tables in Sections 3 through 6 share the following five columns:

Held By: The entity that holds the long-lived credentials.
Refreshed By: The entity that obtains new short-lived tokens.
Presented As: What Lambda presents to the broker.
What You Configure: What the user configures.
Where the Source Says So: The documentation supporting this entry, indicated by page title and section name.

The first column in the tables is Method. The Method column contains the name of the method and its Type value. This article adds one additional column:

What the Brokers Must Trust: What the brokers must accept.

2.4 Values for `Held By`, `Refreshed By`, and `Presented As`

The values for Held By are as follows:

Secrets Manager (your secret): Kept in a Secrets Manager secret that the user created. Lambda reads this secret using the function's execution role (see Section 3.4).
Not stored (the execution role): No secret is stored. Lambda uses the function's execution role.
Not applicable: No credentials are used. This is used with unauthenticated access to Amazon MSK.
The source does not say.: The documentation does not specify.

The values for Refreshed By are as follows:

Lambda: Lambda refreshes short-lived tokens before their expiration.
Not applicable: No short-lived tokens are refreshed. The user replaces the secret value, and this action is documented in the What You Configure column.
The source does not say.: The documentation does not specify.

The values for Presented As are as follows:

SASL/SCRAM credentials: SASL/SCRAM username and password.
SASL/PLAIN credentials: SASL/PLAIN username and password.
TLS client certificate: TLS client certificate.
OAuth access token (SASL/OAUTHBEARER): An OAuth access token presented over SASL/OAUTHBEARER.
AWS web identity token (JWT, SASL/OAUTHBEARER): An AWS web identity token presented over SASL/OAUTHBEARER.
Signed with the execution role's credentials: Signs the connection using the execution role's credentials for each connection.
Not presented: No credentials are presented.
The source does not say.: The documentation does not specify.

2.5 When the Source Does Not Say, and When the Sources Disagree

In each table cell, this article writes only what the source says, or a summary of it. Where the source does not say something, the cell reads The source does not say. Before writing that, this article searches the full text of the source, the pages its links point to, and other pages from the same provider. These related pages are both chapters of the Lambda developer guide, the API reference, CloudFormation, SAM, What's New, and the page describing using Kafka topics as an on-failure destination. The searches use the subject of the row along with the terms OAUTHBEARER, IAM_AUTH, GetWebIdentityToken, MSK, and self-managed. Where the source gives only a general rule and does not name the method of that row, the cell does not read The source does not say.; it states the general rule and notes that the method is not named.

When the sources disagree, this article lists both statements with their sources and does not lean toward either. It uses the phrase the sources disagree throughout. The places where the sources disagree are collected in Section 8.1. Statements that were searched for and not found are collected in Section 8.3, with where and how they were searched for. That a statement was not found does not mean that the behavior does not exist.

2.6 Where to Check Which Method Your ESM Uses

You can check which method your ESM uses to sign in to your brokers in two places.

First, check the ESM's SourceAccessConfigurations. The response from GetEventSourceMapping returns SourceAccessConfigurations. For self-managed ESMs, you will find one of the method Type values described in Section 2.2 within these configurations. The page for creating an ESM in Amazon MSK notes that when the cluster uses IAM authentication, the SourceAccessConfiguration object is not required (Section 6.1).

The second location is the ESM's system logs. The page for viewing Kafka ESM logs states that INFO log records contain the client configuration for Kafka consumers. The troubleshooting page indicates that the securityProtocol and saslMechanism fields in these records indicate the authentication method, providing an example. In this example, the eventSourceArn is the ARN of an Amazon MSK cluster.

"securityProtocol": "SASL_SSL",
"saslMechanism": "AWS_MSK_IAM",

The logs page states that this logging functionality is only available for Kafka ESMs operating in Provisioned mode. No statement was found on the saslMechanism values for ESMs that use OAUTHBEARER_AUTH or IAM_OAUTHBEARER_AUTH (Section 8.3).

3. Methods That Keep a Secret in Secrets Manager — SASL/SCRAM, SASL/PLAIN, and mTLS

This section covers three methods for managing credentials by storing them as secrets in Secrets Manager, which Lambda then reads and uses to authenticate with the broker. With these three methods, there is no short-lived token for Lambda to refresh.

3.1 SASL/SCRAM and SASL/PLAIN

The self-managed chapter states that Lambda supports SASL/SCRAM with TLS encryption (SASL_SSL), but does not support plain-text SASL/SCRAM.

Lambda doesn't support SASL/SCRAM with plaintext (SASL_PLAINTEXT).

The Type is either SASL_SCRAM_256_AUTH or SASL_SCRAM_512_AUTH. The API reference lists SASL_SCRAM_256_AUTH as a self-managed type, while SASL_SCRAM_512_AUTH is listed as a type supported by both Amazon MSK and self-managed configurations.

The same section also states that Lambda supports SASL/PLAIN. It notes that because SASL/PLAIN transmits credentials in plain text, the connection to the server must be encrypted using TLS. The Type for SASL/PLAIN is BASIC_AUTH. On the page listing configuration parameters, the line for SourceAccessConfigurations states For SASL_PLAIN, set to BASIC_AUTH.

In both methods, the user places the sign-in credentials in a secret within Secrets Manager and then specifies the ARN of that secret in the URI. Lambda reads the credentials from the secret and presents them to the broker. The user is responsible for replacing the credentials.

3.2 mTLS

In self-managed Kafka, Lambda acts as the client in mTLS. Users store the client certificate within a Secrets Manager secret. The documentation specifies that the client certificate must be signed by a CA that is present in the server's trust store. The Type is set to CLIENT_CERTIFICATE_TLS_AUTH.

Regarding the structure of the secret, the documentation details the following:

It must contain both a certificate field and a private key field. If the private key is encrypted, it must also include the password for decrypting the private key.
Both the certificate and the private key must be in PEM format.
The certificate field should contain a chain of certificates, starting with the client certificate, followed by any intermediate certificates, and ending with the root certificate.
The private key must be in PKCS #8 format.
Only the PBES1 algorithm is supported for encrypting the private key.

Concerning the final point, the documentation states:

Lambda supports the PBES1 (but not PBES2) private key encryption algorithms.

The documentation also notes that Secrets Manager supports secrets up to 65,536 bytes, which is sufficient for long certificate chains. The extended key usage (EKU) of the client certificate, and the use of a public CA certificate as the client certificate, are covered in Section 4.4 of the existing article When Public Certificates Stop Carrying clientAuth.

3.3 Private CA Brokers and `SERVER_ROOT_CA_CERTIFICATE`

Kafka clusters send server certificates to Lambda. The documentation states that for self-managed environments, the server certificate can be a certificate from a public CA, a private CA, or a self-signed certificate. If using a certificate from a public CA, it must be signed by a CA trusted within Lambda's trust store. When using a certificate from a private CA or a self-signed certificate, the user must store the broker's root CA certificate as a secret in Secrets Manager and pass it to Lambda using the Type field of SERVER_ROOT_CA_CERTIFICATE. Lambda will then use this root certificate to validate the broker.

The sources disagree on which methods SERVER_ROOT_CA_CERTIFICATE applies to (item 6 in Section 8.1).

3.4 Secrets Are Read with the Execution Role

Lambda reads secrets from Secrets Manager using the function's execution role. The OAuth secret section of the self-managed chapter states:

Lambda reads the secret with your function's execution role, so that role needs secretsmanager:GetSecretValue permission on the secret.

The self-managed Kafka network page also states, in the description of the Secrets Manager VPC endpoint policy, To call the Secrets Manager API, Lambda uses your function role, not the Lambda service principal.

The self-managed chapter includes two sentences regarding the Region where secrets are stored. The Important note in the SASL/SCRAM section reads:

To use Secrets Manager for authentication, secrets must be stored in the same AWS region as your Lambda Event Function.

The Important note in the OAuth secret section reads:

Store the secret in the same AWS Region as your Lambda function.

Both sentences require storing secrets in the same Region as the Lambda function. This article will cite these two sentences as statements of the sections where they are written.

4. OAuth 2.0 — `OAUTHBEARER_AUTH`

This section describes an OAuth method announced on October 9, 2026. In this method, Lambda obtains an OAuth access token from the user's identity provider (IdP), presents it to the broker, and refreshes it before it expires.

4.1 What Lambda Does, What the User Does

The self-managed chapter describes what Lambda does in the OAuth method as follows:

Lambda supports SASL/OAUTHBEARER authentication with TLS encryption (SASL_SSL). Lambda requests an access token from your OAuth 2.0 identity provider and presents that token to your Kafka brokers. Lambda refreshes the token before it expires. Configure your brokers to validate tokens from the same identity provider.

Separating the subjects of these four sentences gives the following:

Lambda is responsible for obtaining the OAuth access token. Lambda sends a request to the user's IdP token endpoint.
Lambda is responsible for presenting the OAuth access token to the broker. This is achieved using the SASL/OAUTHBEARER mechanism, and the connection is encrypted using TLS.
Lambda is responsible for refreshing the OAuth access token. The chapter says Lambda refreshes it before it expires. It does not specify how far in advance the refresh should occur.
The user is responsible for configuring the broker to validate tokens from the same IdP.

The user creates the OAuth secret in Secrets Manager and enters the ARN of that secret in the URI of the entry whose Type is OAUTHBEARER_AUTH.

4.2 The Grant Type Is Decided by the Keys in the Secret

Lambda supports two grant types for OAuth. Which one is used is determined by the fields configured in the secrets.

Lambda supports two grant types. The fields that you store in the secret determine which one Lambda uses.

Client Credentials: Lambda sends the client ID and client secret to the token endpoint. The chapter says to use this grant type when your IdP issues a client secret.
JWT Bearer: Lambda signs a JWT bearer assertion with a private key and sends that assertion to the token endpoint. The chapter says to use this grant type when your IdP expects a signed assertion instead of a client secret.

The secrets for OAuth are defined in a JSON document. The documentation lists the following fields:

Field	Required	Description
`oauthTokenEndpointUrl`	Yes	The identity provider's (IdP) token endpoint. `This must be an HTTPS URL.`
`oauthClientId`	Yes	The client ID issued by the IdP.
`oauthClientSecret`	Only Client Credentials	The client secret issued by the IdP.
`oauthClientEmail`	Only JWT Bearer	The service account ID that Lambda includes in the JWT bearer assertion.
`oauthPrivateKey`	Only JWT Bearer	The private key that Lambda uses to sign the JWT bearer assertion. Must be in PEM format.
`oauthTokenExpirationSeconds`	Optional	The validity period for the JWT bearer assertion that Lambda signs. Applies only to the JWT bearer grant type.
`oauthIdpCaCertificate`	Optional	The root CA certificate for the IdP. Must be in PEM format. Provided when the IdP uses a private CA certificate.

The documentation specifies that only the fields for one grant type should be configured. If both are provided, the result is as follows:

If you provide fields from both grant types, Lambda returns a validation error.

oauthTokenExpirationSeconds does not represent the validity period of the OAuth access token. The chapter describes this field as The lifetime of the JWT assertion that Lambda signs. The chapter's table of secret fields has no field that sets the validity period of the OAuth access token.

Regarding the client credentials grant type, no statement was found on how Lambda sends the client secret to the token endpoint (whether in the HTTP Authorization header or in the request body) (Section 8.3). The chapter only states that Lambda sends a client ID and client secret to the token endpoint.

4.3 The Four Accompanying Entries

When an identity provider (IdP) or broker requests additional values, add one SourceAccessConfigurations entry for each value. In the URI field of these four entries, specify the value itself, rather than the ARN of the secret.

OAUTHBEARER_SCOPE: The scope requested by Lambda when requesting an OAuth access token.
OAUTHBEARER_AUDIENCE: The audience requested by Lambda when requesting a token.
OAUTHBEARER_LOGICAL_CLUSTER: The logical cluster ID sent to the broker by Lambda as a SASL extension.
OAUTHBEARER_IDENTITY_POOL: The identity pool ID sent to the broker by Lambda as a SASL extension.

The first two entries represent values requested by Lambda when requesting a token, while the latter two represent values sent by Lambda to the broker. The chapter calls their destination your brokers, while the API reference and CloudFormation call it a Confluent Cloud broker (item 5 in Section 8.1).

The documentation notes that these four entries require an OAuth 2.0 authentication type.

These four types require an OAuth 2.0 authentication type. Provide them with OAUTHBEARER_AUTH, or provide only OAUTHBEARER_AUDIENCE with IAM_OAUTHBEARER_AUTH. If you provide any of them without one of these authentication types, Lambda returns a validation error.

The OAuth 2.0 authentication types in this note are OAUTHBEARER_AUTH and IAM_OAUTHBEARER_AUTH. The rules for combining these are summarized in the table in Section 7.3.

If the IdP's token endpoint presents a certificate from a private CA, place the IdP's root CA certificate in the secret's oauthIdpCaCertificate. The documentation states that if the broker presents a certificate from a private CA, also provide the SERVER_ROOT_CA_CERTIFICATE entry. These two entries represent different certificates: Lambda uses the former to verify the IdP and the latter to verify the broker.

4.4 Replacing the Secret and Recreating the ESM

Even if you replace the values in the OAuth secret, there is no need to recreate the ESM. The chapter states:

You can rotate the credentials in the secret without recreating the event source mapping. Lambda picks up the new values and requests a new token.

Breaking down the subject of this sentence, the user is responsible for replacing the secret credentials, while Lambda reads the new values and requests a new OAuth access token. The time it takes for Lambda to read the new values is not specified.

Whether you can switch the existing ESM's authentication method – for example, from SASL/SCRAM to OAuth – is a separate issue. What the sources say about this is collected in Section 8.2.

5. Methods That Store No Secret — `IAM_AUTH` and `IAM_OAUTHBEARER_AUTH`

This section covers two methods that do not store any secrets within Secrets Manager. Both methods utilize the Lambda function's execution role. What differs is what they present to the broker.

5.1 `IAM_AUTH` — Amazon MSK Clusters Reached Through Your Own Networking Setup

The IAM authentication section of the self-managed chapter describes this method as follows:

If your cluster is an Amazon MSK cluster that you reach through your own networking setup, you can register it as a self-managed Kafka event source and authenticate with IAM.

The chapter gives two examples of a custom networking setup: a custom domain name placed in front of the brokers, and a connection from another account. With this method, Lambda signs each connection using the credentials of the execution role, eliminating the need for users to store credentials in Secrets Manager.

Lambda signs each connection with the credentials of your function's execution role, so you don't store any credentials in Secrets Manager.

According to the chapter, users need to perform the following three steps:

Enable IAM access control for the cluster.
Select IAM_AUTH for the Type and omit the URI field.
Grant the cluster permissions to the execution role (the sources disagree on which page lists those permissions; see Section 7.1 and item 2 in Section 8.1).

When the broker presents a certificate from a private CA, the SERVER_ROOT_CA_CERTIFICATE must also be provided. The chapter indicates that this is typically required when using a custom domain name in front of the broker.

No statement was found on which SASL mechanism Lambda uses to sign connections with IAM_AUTH (Section 8.3).

5.2 `IAM_OAUTHBEARER_AUTH` — Presenting an AWS Web Identity Token to the Brokers

The section on IAM authentication with SASL/OAUTHBEARER in the self-managed chapter states that if your brokers accept OAuth 2.0 tokens, you can use AWS as the identity provider. The chapter describes what Lambda does with this method as follows:

Lambda presents an AWS web identity token to your brokers over SASL/OAUTHBEARER. Lambda requests a short-lived token for your function's execution role with the sts:GetWebIdentityToken action, so you don't store any credentials in Secrets Manager. Lambda requests a new token before the current one expires.

Breaking down the sentence, the key elements are:

AWS STS issues the AWS web identity token. Lambda calls sts:GetWebIdentityToken with the function's execution role.
Lambda is responsible for presenting the AWS web identity token to the broker. This is achieved using the SASL/OAUTHBEARER mechanism.
Lambda is responsible for refreshing the AWS web identity token. The chapter states that before the current token expires, Lambda requests a new one.
Users do not store anything in Secrets Manager.

According to the chapter, users perform the following three steps:

Select IAM_OAUTHBEARER_AUTH for the Type and omit the URI field.
In the OAUTHBEARER_AUDIENCE entry, specify the audience that the broker expects.
Grant the execution role the permission to call sts:GetWebIdentityToken.

This method does not allow the use of any additional entries besides OAUTHBEARER_AUDIENCE.

IAM_OAUTHBEARER_AUTH supports only OAUTHBEARER_AUDIENCE. If you also provide OAUTHBEARER_SCOPE, OAUTHBEARER_LOGICAL_CLUSTER, or OAUTHBEARER_IDENTITY_POOL, Lambda returns a validation error.

Regarding the broker's prerequisites, the chapter states that these must be met before creating an ESM.

Configure your brokers to trust AWS as an OpenID Connect (OIDC) identity provider before you create the event source mapping. If your brokers don't trust the AWS issuer, they reject the token and Lambda can't read from your topics.

The subjects of these sentences are the user and the broker. The user is responsible for configuring the broker to trust AWS as an OIDC IdP, while the broker is responsible for rejecting AWS web identity tokens if it does not trust the AWS issuer. SAM's SelfManagedKafka page also writes your brokers must trust the AWS issuer and accept the audience value in its example for this method.

5.3 Prerequisites on the `GetWebIdentityToken` Side

IAM_OAUTHBEARER_AUTH relies on AWS STS's GetWebIdentityToken. The AWS STS API reference states the following about GetWebIdentityToken:

It returns a signed JSON Web Token (JWT) that represents the calling AWS identity (Returns a signed JSON Web Token (JWT) that represents the calling AWS identity.).
It is not available on the STS Global endpoint (The GetWebIdentityToken API is not available on the STS Global endpoint.).
If outbound web identity federation is not enabled in the account, it returns an error (HTTP 403) – OutboundWebIdentityFederationDisabled.
Audience is a required parameter and is included in the JWT's aud claim.
SigningAlgorithm is a required parameter, and its value must be RS256 or ES384.
DurationSeconds must be between 60 and 3,600 seconds. If not specified, the default value is 300 seconds.

Regarding enabling outbound web identity federation in the account, neither the Lambda chapter nor SAM's SelfManagedKafka page mentions this. The STS API reference states that GetWebIdentityToken will return an error if the account does not have outbound web identity federation enabled.

The 300 seconds in the last item is the default of the GetWebIdentityToken API. No statement was found on which DurationSeconds and SigningAlgorithm the ESM uses when it calls GetWebIdentityToken (Section 8.3). This article does not state that AWS web identity tokens for IAM_OAUTHBEARER_AUTH expire after 300 seconds. There are only two mentions: one in the Lambda chapter, which describes OAUTHBEARER_AUDIENCE as The audience that Lambda requests when it asks for a token., and another in the STS API reference, which describes Audience as the value that goes into the aud claim.

The process for enabling outbound identity federation on an account, as well as obtaining the issuer URL, is covered in Section 3.1 of the previously published AWS IAM Outbound Identity Federation. Section 3.4 of the same article says to get the issuer URL from the API rather than construct it. This article does not provide an example of the issuer URL. The algorithms used by AWS to sign JWTs are also discussed in the previously published JWT Signing Algorithms on AWS Endpoints.

5.4 What the Brokers Must Trust — Three Settings in Apache Kafka 4.3

This article does not detail the steps for building a broker. For reference, the "Broker Configs" page in the Apache Kafka 4.3 documentation (checked October 13, 2026) lists three settings relevant to configuring a broker to validate JWTs.

Setting	Description from Apache Kafka 4.3 Documentation
`sasl.oauthbearer.jwks.endpoint.url`	`The OAuth/OIDC provider URL from which the provider's JWKS (JSON Web Key Set) can be retrieved.`
`sasl.oauthbearer.expected.issuer`	`The (optional) setting for the broker to use to verify that the JWT was created by the expected issuer.`
`sasl.oauthbearer.expected.audience`	`The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.`

The Kafka documentation describes both issuer and audience verification as (optional) settings. The Lambda chapter says to configure the brokers to trust AWS as an OIDC identity provider before creating the ESM, and the SAM page says the brokers must trust the AWS issuer and accept the audience value. This article places the two side by side as what each source says. For a Kafka provider's managed service, that provider's documentation applies.

5.5 Differences Between MSK Replicator's `IAM JWT bearer` and `IAM_OAUTHBEARER_AUTH`

MSK Replicator can also connect to self-managed Kafka clusters using SASL/OAUTHBEARER. The MSK developer guide, on the prerequisites page for MSK Replicator, lists IAM JWT bearer as one mechanism for obtaining access tokens, stating that MSK Replicator obtains signed JWTs using the AWS identity of the service execution role and sends them to the token endpoint as an assertion.

However, the pathways differ. With MSK Replicator's IAM JWT bearer, the signed JWT is sent to the IdP's token endpoint, and MSK Replicator receives an access token in exchange. In contrast, with Lambda's IAM_OAUTHBEARER_AUTH, the AWS web identity token is presented directly to the broker. This article does not describe these two as using the same mechanism. Furthermore, it differs from Lambda's JWT bearer grant type, where the key used to sign the assertion is the oauthPrivateKey that the user stored in the secret.

5.6 The Self-Managed Methods Table

The following table lists seven rows, corresponding to Sections 3 through 5. The supporting documentation for each row can be found on the Lambda developer guide page "Configuring cluster authentication methods in Lambda." In the Where the Source Says So column, the name of the relevant section is provided.

Method	Held By	Refreshed By	Presented As	What You Configure	What the Brokers Must Trust	Where the Source Says So
SASL/SCRAM (`SASL_SCRAM_256_AUTH` / `SASL_SCRAM_512_AUTH`)	`Secrets Manager (your secret)`	`Not applicable`	`SASL/SCRAM credentials`	A secret containing the username and password. TLS (`SASL_SSL`) is required.	The brokers must accept SASL/SCRAM over TLS.	SASL/SCRAM authentication
SASL/PLAIN (`BASIC_AUTH`)	`Secrets Manager (your secret)`	`Not applicable`	`SASL/PLAIN credentials`	A secret containing the username and password. The connection must be encrypted with TLS.	The brokers must accept SASL/PLAIN over TLS.	SASL/SCRAM authentication
mTLS (`CLIENT_CERTIFICATE_TLS_AUTH`)	`Secrets Manager (your secret)`	`Not applicable`	`TLS client certificate`	A secret containing the certificate chain in PEM format and the PKCS #8 private key. Only PBES1 is supported for private key encryption.	The CA that signed the client certificate must be in the trust store.	Mutual TLS authentication
OAuth 2.0 client credentials (`OAUTHBEARER_AUTH`)	`Secrets Manager (your secret)`	`Lambda`	`OAuth access token (SASL/OAUTHBEARER)`	A secret containing `oauthClientSecret`. Include accompanying entries as needed.	The brokers must validate tokens from the same IdP.	OAuth 2.0 authentication
OAuth 2.0 JWT bearer (`OAUTHBEARER_AUTH`)	`Secrets Manager (your secret)`	`Lambda`	`OAuth access token (SASL/OAUTHBEARER)`	A secret containing `oauthClientEmail` and `oauthPrivateKey`. Include accompanying entries as needed.	The brokers must validate tokens from the same IdP.	OAuth 2.0 authentication
IAM (`IAM_AUTH`)	`Not stored (the execution role)`	`Not applicable`	`Signed with the execution role's credentials`	Exclude the `URI`. Enable cluster IAM access control and grant the execution role cluster permissions.	The cluster must be an Amazon MSK cluster with IAM access control enabled.	IAM authentication
IAM with SASL/OAUTHBEARER (`IAM_OAUTHBEARER_AUTH`)	`Not stored (the execution role)`	`Lambda`	`AWS web identity token (JWT, SASL/OAUTHBEARER)`	Exclude the `URI`. Pass `OAUTHBEARER_AUDIENCE` and grant the execution role the `sts:GetWebIdentityToken` permission.	The brokers must trust AWS as an OIDC IdP and accept the audience.	IAM authentication with SASL/OAUTHBEARER

The Refreshed By field in IAM_AUTH shows Not applicable because the chapter describes no token to refresh for this method. The chapter says Lambda signs each connection with the credentials of the execution role. How those execution role credentials are updated falls outside the scope of this article.

6. The Amazon MSK Event Source Mapping — Four Methods and How Lambda Chooses a Broker

This section discusses Amazon MSK. The set of methods available with the Amazon MSK ESM differs from the self-managed ESM.

6.1 Four Methods

The Amazon MSK chapter describes four methods:

Unauthenticated Access: The documentation states that this method can be used when there are no clients accessing the cluster from the internet.
SASL/SCRAM: This method involves SHA-512 hashing and TLS encryption. Users store their username and password as secrets in Secrets Manager and reference those secrets in the ESM. The Type is SASL_SCRAM_512_AUTH.
mTLS: Users store their client certificates as secrets in Secrets Manager and reference those secrets in the ESM. The Type is CLIENT_CERTIFICATE_TLS_AUTH, and the secret format is the same as in the self-managed chapter (PEM, PKCS #8, PBES1 only).
IAM: Lambda connects to the cluster using the permissions defined in the function's execution role and reads records.

The Amazon MSK chapter does not include SASL/PLAIN. The chapter states:

Amazon MSK doesn't support SASL/PLAIN authentication.

Regarding mTLS server certificates, the documentation states that Amazon MSK does not support self-signed server certificates.

Amazon MSK doesn't support self-signed server certificates.

The documentation states that all Amazon MSK brokers use public certificates signed by Amazon Trust Services CAs, and that Lambda trusts these certificates by default. Neither the Amazon MSK chapter nor the page listing the parameters for Amazon MSK's ESM includes a description of SERVER_ROOT_CA_CERTIFICATE.

Regarding IAM authentication, the chapter states that Lambda uses IAM authentication by default when IAM auth is active on the cluster and no secret is provided.

If IAM auth is active on your MSK cluster, and you don't provide a secret, Lambda automatically defaults to using IAM auth.

The page describing the creation of Amazon MSK's ESM also states that the SourceAccessConfiguration object is not required when the cluster uses IAM authentication. Unlike self-managed IAM_AUTH, Amazon MSK's ESM IAM authentication does not have a Type value to indicate the method.

Amazon MSK's ESM IAM authentication was announced on October 7, 2021, in the "What's New" announcement: "AWS Lambda now supports IAM authentication for Amazon MSK as an event source."

6.2 How Lambda Chooses a Broker, and Why It Does Not Fall Back to a Weaker Method

The final section of the Amazon MSK chapter describes how Lambda selects the bootstrap broker. When the user provides a secret for mTLS or SASL/SCRAM, Lambda chooses that method. If no secret is provided, Lambda will select the strongest method that is active on the cluster. The chapter lists the following order of preference, from strongest to weakest:

mTLS (a secret for mTLS is provided)
SASL/SCRAM (a secret for SASL/SCRAM is provided)
SASL IAM (no secret is provided, and IAM authentication is enabled)
Unauthenticated TLS (no secret is provided, and IAM authentication is not enabled)
Plaintext (no secret is provided, and neither IAM authentication nor unauthenticated TLS is enabled)

The chapter notes that Lambda will not fall back to weaker methods.

If Lambda can't connect to the most secure broker type, Lambda doesn't attempt to connect to a different (weaker) broker type. If you want Lambda to choose a weaker broker type, deactivate all stronger auth methods on your cluster.

The subjects of these sentences are Lambda and the user. If Lambda cannot connect using the strongest method, it will not attempt to use a weaker method. If a user wants Lambda to use a weaker method, they must disable all stronger methods on the cluster.

6.3 The Path to Registering an Amazon MSK Cluster as Self-Managed

As described in Section 5.1, the self-managed chapter states that an Amazon MSK cluster reached through your own networking setup can be registered as a self-managed Kafka event source, allowing authentication via IAM_AUTH. The chapter gives two examples of a custom networking setup: a custom domain name in front of the brokers, and a connection from another account.

Meanwhile, the cross-account page of the Amazon MSK chapter details the steps for connecting to a provisioned MSK cluster in another account using multi-VPC private connections, utilizing Amazon MSK's ESM. This page notes that when the cluster uses IAM authentication, the SourceAccessConfiguration object is not required. The sources thus document two paths for connecting to an Amazon MSK cluster in another account; however, this article does not specify which option to choose.

6.4 The Amazon MSK Methods Table

The following table lists the four methods in the Amazon MSK chapter. The source of every row is the Lambda developer guide page Configuring Amazon MSK cluster authentication methods in Lambda, and the Where the Source Says So column gives the name of the section.

Method	Held By	Refreshed By	Presented As	What You Configure	What the Brokers Must Trust	Where the Source Says So
Unauthenticated Access	`Not applicable`	`Not applicable`	`Not presented`	No authentication is used.	The brokers must allow unauthenticated access.	Unauthenticated access
SASL/SCRAM (`SASL_SCRAM_512_AUTH`)	`Secrets Manager (your secret)`	`Not applicable`	`SASL/SCRAM credentials`	A secret containing the username and password.	The brokers must accept SASL/SCRAM with SHA-512 over TLS.	SASL/SCRAM authentication
mTLS (`CLIENT_CERTIFICATE_TLS_AUTH`)	`Secrets Manager (your secret)`	`Not applicable`	`TLS client certificate`	A secret containing the PEM certificate chain and the PKCS #8 private key. Only PBES1 is supported for private key encryption.	The Certificate Authority (CA) that signed the client certificates must be in the broker's trust store.	Mutual TLS authentication
IAM (`Type` value not specified)	`Not stored (the execution role)`	`Not applicable`	`The source does not say.`	No secrets are passed. The execution role is granted permissions to the cluster.	The brokers must allow IAM authentication.	IAM authentication

The Presented As field in the IAM row is The source does not say. because the Amazon MSK chapter only states that Lambda connects by relying on the execution role's permissions, without specifying what it presents to the broker. The log examples on the troubleshooting page in Section 2.6 show the ESM's saslMechanism set to AWS_MSK_IAM when reading the Amazon MSK cluster.

6.5 No Statement About OAuth Was Found in the Amazon MSK Chapter

The term OAUTHBEARER was not found in the Amazon MSK chapter, the page for creating an Amazon MSK ESM, the parameter list page, the permissions page, or SAM's MSK page. The title and body of the What's New announcement of October 9, 2026, cover only the self-managed Apache Kafka ESM. The API reference and CloudFormation describe seven types, OAUTHBEARER_AUTH, the four accompanying types, IAM_AUTH, and IAM_OAUTHBEARER_AUTH, all as self-managed types.

Conversely, the page discussing using Kafka topics as on-failure destinations mentions OAuth 2.0, IAM_AUTH, and IAM_OAUTHBEARER_AUTH, and references both the self-managed chapter and the Amazon MSK chapter. No statement was found that says OAuth cannot be used with the Amazon MSK ESM (Section 8.3). This article only says that the sources describe OAuth as a method of the self-managed ESM.

7. Execution Role Permissions and the Combination Rules

This section covers the permissions that the execution role needs for each method, as well as the rules governing combinations of SourceAccessConfigurations entries.

7.1 Permissions for Each Method, and the Pages That List Them

The following table lists the permissions required for each method, along with the pages where those permissions are defined.

Method	Permissions Required for Execution Role	Page Describing Permissions
SASL/SCRAM, SASL/PLAIN, mTLS (self-managed)	`secretsmanager:GetSecretValue`. If the secret is encrypted with a customer-managed key, also `kms:Decrypt`.	Configuring Lambda execution role permissions (self-managed)
`OAUTHBEARER_AUTH`	`secretsmanager:GetSecretValue` for accessing the secret.	Configuring cluster authentication methods in Lambda (OAuth secrets section)
`IAM_AUTH`	The chapter refers to the self-managed permissions page as `the cluster permissions described in`. The description on that page corresponds to Section 8.1, item 2.	Configuring cluster authentication methods in Lambda (IAM authentication)
`IAM_OAUTHBEARER_AUTH`	`sts:GetWebIdentityToken`	Configuring cluster authentication methods in Lambda, SAM's `SelfManagedKafka`
Amazon MSK (SASL/SCRAM, mTLS)	Access to the SASL/SCRAM secret, permission to read the secret, and decryption using a customer-managed key. The page lists the following permissions: `kafka:ListScramSecrets`, `secretsmanager:GetSecretValue`, and `kms:Decrypt`.	Configuring Lambda permissions for Amazon MSK event source mappings
Amazon MSK (IAM)	`kafka-cluster:Connect`, `kafka-cluster:AlterGroup`, `kafka-cluster:DescribeGroup`, `kafka-cluster:DescribeTopic`, `kafka-cluster:ReadData`	Configuring Lambda permissions for Amazon MSK event source mappings

The Amazon MSK permissions page states that the Amazon MSK ESM needs kafka:DescribeClusterV2 (or kafka:DescribeCluster) and kafka:GetBootstrapBrokers for every method. This page states that the AWSLambdaMSKExecutionRole managed policy contains the minimum necessary permissions for Amazon MSK's ESM.

The troubleshooting page for Kafka ESM states that, for IAM access control, the SASL authentication failed error means the execution role is missing the kafka-cluster:Connect permission for the cluster. This page addresses both Amazon MSK and self-managed Kafka.

7.2 Permissions AWS SAM Adds, and Permissions It Does Not Add

The SelfManagedKafka section within AWS SAM details the permissions that SAM adds to and excludes from the execution role. SAM adds the secretsmanager:GetSecretValue permission for each specified secret ARN, including the SERVER_ROOT_CA_CERTIFICATE secret. When a VPC entry is specified, SAM also adds permissions related to Amazon EC2 network interfaces. SAM does not add permissions for IAM_AUTH or IAM_OAUTHBEARER_AUTH.

For IAM_AUTH and IAM_OAUTHBEARER_AUTH, AWS SAM adds no cluster or token permissions. Grant those permissions to your function's execution role yourself.

Separating the subjects of this sentence: the user, not SAM, grants the execution role the cluster permissions for IAM_AUTH and the sts:GetWebIdentityToken permission for IAM_OAUTHBEARER_AUTH.

7.3 Combination Validation — What Passes and What Results in Validation Errors

The combination of entries within SourceAccessConfigurations is subject to both Lambda and SAM validation rules. Figure 2 shows the combinations of the two OAuth 2.0 authentication types and the four accompanying entries.

Which Source Access Configuration Entries Can Be Combined

The following table lists the combination rules that the sources describe.

#	Combination	Result	Validator	Source
1	`OAUTHBEARER_AUTH` along with any of `OAUTHBEARER_SCOPE`, `OAUTHBEARER_AUDIENCE`, `OAUTHBEARER_LOGICAL_CLUSTER`, and `OAUTHBEARER_IDENTITY_POOL`.	Passes	Lambda	Self-managed chapter (note), API reference
2	`IAM_OAUTHBEARER_AUTH` and `OAUTHBEARER_AUDIENCE`.	Passes. `OAUTHBEARER_AUDIENCE` is required.	Lambda	Self-managed chapter, API reference
3	`IAM_OAUTHBEARER_AUTH` along with any of `OAUTHBEARER_SCOPE`, `OAUTHBEARER_LOGICAL_CLUSTER`, and `OAUTHBEARER_IDENTITY_POOL`.	Validation error	Lambda	Self-managed chapter (note)
4	Providing any of the four accompanying entries without either `OAUTHBEARER_AUTH` or `IAM_OAUTHBEARER_AUTH`.	Validation error	Lambda	Self-managed chapter (note)
5	Including both client credentials and JWT bearer fields within an OAuth secret.	Validation error	Lambda	Self-managed chapter (OAuth secret section)
6	Specifying two or more authentication types.	Error	AWS SAM	SAM's `SelfManagedKafka`
7	Specifying only one of `VPC_SUBNET` or `VPC_SECURITY_GROUP`.	Error	AWS SAM	SAM's `SelfManagedKafka`
8	Omitting `URI` for authentication types other than `IAM_AUTH` and `IAM_OAUTHBEARER_AUTH`.	Error	AWS SAM	SAM's `SelfManagedKafka`

Items 6 through 8 represent validation steps that SAM performs before deployment. The SAM page indicates that it only verifies the existence of the URI value and does not interpret it; instead, it states that Lambda validates the values and the combination of types when you deploy. No statement was found on how the Lambda API responds when more than one authentication type is specified (Section 8.3).

7.4 When Using a Kafka Topic as an On-Failure Destination

The page describing how to use a Kafka topic as an on-failure destination states that this destination does not have its own authentication settings. Since the destination topic is in the same cluster as the source topic, Lambda uses the authentication configured in the ESM's SourceAccessConfigurations to write failed records. The page further explains that when authenticating with OAuth 2.0, Lambda obtains an OAuth access token using the same identity provider (IdP) and secret, allowing it to write to the destination topic.

The page states that the identity used for authentication must not only be able to read from the source topic but also be able to write to the destination topic. For IAM authentication, this requires the kafka-cluster:WriteData permission for the destination topic. For other authentication methods, it requires equivalent write permissions in the cluster's authorization rules.

8. Where the Sources Disagree and Where No Statement Was Found

This section collects the places where the sources disagree and the statements that were not found, as mentioned in Sections 1 through 7 of this article. This article does not claim that either document contains errors. All verification dates are October 13, 2026.

8.1 Where the Sources Disagree

The following table lists the places where the sources disagree.

#	Topic	One Source Says	Another Source Says	Section
1	Number of possible values for `Type`	In Lambda's API reference, `Valid Values` for `SourceAccessConfiguration` lists 8 values, but excludes 7 values from `OAUTHBEARER_AUTH` to `IAM_OAUTHBEARER_AUTH`. The same page describes the `Type` field in a list, which includes those 7 values.	In CloudFormation, `Allowed values` for `SourceAccessConfiguration` lists 15 values, ending with `OAUTHBEARER_IDENTITY_POOL`, `IAM_AUTH`, and `IAM_OAUTHBEARER_AUTH`. In SAM, `Valid values` for `SelfManagedKafka` lists 14 values, excluding RabbitMQ's `VIRTUAL_HOST`.	2.2
2	Permissions for the execution role (related to `IAM_AUTH`)	The IAM authentication section of the self-managed chapter refers to a self-managed permissions page (`with-kafka-permissions.html`) as `the cluster permissions described in`. That page does not mention `GetWebIdentityToken` or permissions starting with `kafka-cluster:`.	Amazon MSK's permissions page lists the following permissions for IAM authentication: `kafka-cluster:Connect`, `kafka-cluster:AlterGroup`, `kafka-cluster:DescribeGroup`, `kafka-cluster:DescribeTopic`, and `kafka-cluster:ReadData`. A troubleshooting page also states that `kafka-cluster:Connect` is required for IAM access control.	5.1, 7.1
3	Previous authentication methods	A "What's New" document from October 9, 2026, states: `Previously, Kafka ESM supported only SASL/PLAIN, SASL/SCRAM, and mutual TLS (mTLS) as authentication methods`.	A "What's New" document from October 7, 2021, announced IAM authentication for Amazon MSK's ESM. The self-managed chapter mentions `IAM_AUTH`. No statement was found on when `IAM_AUTH` was added.	1.1, 6.1
4	Values for Amazon MSK's ESM `SourceAccessConfigurations`	SAM's `MSK` page lists only `CLIENT_CERTIFICATE_TLS_AUTH` under `Valid values` for `SourceAccessConfigurations`.	The Amazon MSK chapter mentions SASL/SCRAM, and the page for creating Amazon MSK's ESM provides a CLI example using `SASL_SCRAM_512_AUTH`. The API reference lists `SASL_SCRAM_512_AUTH` as a type for Amazon MSK.	6.1
5	Destinations for `OAUTHBEARER_LOGICAL_CLUSTER` and `OAUTHBEARER_IDENTITY_POOL`	The API reference and CloudFormation write `sends to a Confluent Cloud broker`.	The self-managed chapter writes `sends to your brokers as a SASL extension`.	4.3
6	Methods that can use `SERVER_ROOT_CA_CERTIFICATE`	The OAuth 2.0 and IAM authentication sections of the self-managed chapter state that when brokers present a certificate from a private CA, `SERVER_ROOT_CA_CERTIFICATE` should also be provided.	The same chapter's section on root CA secrets states: `You can use TLS encryption for VPC, SASL/SCRAM, SASL/PLAIN, or mTLS authentication.` The page for creating ESM states: `This setting applies to TLS encryption for SASL/SCRAM or SASL/PLAIN, and to mTLS authentication.` Neither list includes OAuth or IAM.	3.3, 4.3, 5.1
7	Values for the `URI` within `OAUTHBEARER_SCOPE`	SAM's OAuth example for `SelfManagedKafka` lists `read write` as the value for `OAUTHBEARER_SCOPE`'s `URI`. This value includes spaces.	Lambda's API reference and CloudFormation list the `URI`'s `Pattern` as `[a-zA-Z0-9-\/:_+=.@-]`. This character set does not include spaces.	4.3
8	Maximum number of entries in the `SourceAccessConfigurations` array	Lambda's API reference for `CreateEventSourceMapping`, `UpdateEventSourceMapping`, and `GetEventSourceMapping` lists a maximum of 22 entries.	CloudFormation's `AWS::Lambda::EventSourceMapping` lists `Maximum: 29`.	2.2

Regarding point 1, this article does not state that the API reference's Valid Values section is outdated. It only states that on the same page, the Valid Values row and the descriptive bullet points list different sets of values.

Regarding item 7, this article has not tested it. It is unclear from the documentation whether Lambda accepts scope values that contain spaces.

8.2 Whether the ESM Must Be Recreated — What the Sources Say and Do Not Say

This section separates what the sources say from what they do not say about the authentication of an existing ESM.

The sources say the following four things:

Changing the credentials for OAuth secrets does not require recreating the ESM (see Section 4.4).
The Lambda API reference for UpdateEventSourceMapping accepts the SourceAccessConfigurations parameter.
The What's New announcement of October 9, 2026, states that using OAuth authentication requires creating a new Kafka ESM with your authentication configuration (create a new Kafka ESM with your authentication configuration).
The Lambda API reference and the CloudFormation Type description include a single line of text immediately following the line describing RabbitMQ's VIRTUAL_HOST.

This property cannot be specified in an UpdateEventSourceMapping API call.

The documentation does not clarify whether this line refers only to VIRTUAL_HOST or to the entire Type property. CloudFormation lists Update requires for the Type property as No interruption. This refers to the classification for updating CloudFormation stacks and is a separate matter from whether the Lambda service accepts changes to an existing ESM's authentication method.

No statement was found on whether UpdateEventSourceMapping can change an existing ESM's authentication type, for example, from SASL_SCRAM_512_AUTH to OAUTHBEARER_AUTH. No statement was found that says it can, and none was found that says it cannot (see Section 8.3). This article will not make a definitive statement on this matter.

8.3 Statements That Were Not Found

The following table lists the statements that were searched for on October 13, 2026, and not found, along with where and how they were searched for. The inability to find a description does not necessarily mean that the behavior it describes does not exist.

What Was Looked For	Where and How It Was Looked For	Result
Whether OAuth can be used with Amazon MSK's ESM.	Searched the Amazon MSK chapter, the ESM creation page, the parameter list page, the permissions page, and the SAM `MSK` page using the terms `OAUTHBEARER`, `OAuth`, and `IAM_OAUTHBEARER`.	Not found. The on-failure destination page refers to both chapters (Section 6.5).
The dates when `IAM_AUTH` and `IAM_OAUTHBEARER_AUTH` were added.	Searched "What's New" using three different search terms (`IAM authentication`, `web identity token`, and a combination of `custom domain name` and `cross-account`), and searched the Lambda developer guide's "Document history" using `Kafka`, `OAuth`, and `web identity`.	Not found. The latest entry in the "Document history" is June 19, 2026, and there is no mention of Kafka's OAuth.
Whether existing ESM authentication methods can be switched.	Searched the sections on self-managed and Amazon MSK, along with `UpdateEventSourceMapping`, `SourceAccessConfiguration`, CloudFormation, and SAM, using the terms `recreat`, `switch`, `change`, and `authentication type`, and read the full text of "What's New."	Not found. One instance of `recreat` refers to replacing secrets (Section 8.2).
The `DurationSeconds` and `SigningAlgorithm` values when ESM calls `GetWebIdentityToken`.	Searched the self-managed chapter, SAM's `SelfManagedKafka`, the API reference, and CloudFormation using the terms `DurationSeconds`, `duration`, `SigningAlgorithm`, `RS256`, and `ES384`.	Not found. 300 seconds is the default value for the `GetWebIdentityToken` API (Section 5.3).
An example of an AWS issuer URL.	Searched the self-managed chapter and SAM's `SelfManagedKafka` using the terms `issuer` and `https://`.	Not found. The term `https://` is an example value for the IdP token endpoint and audience. A previously published article on outbound federation (Section 3.4) describes how to obtain the issuer URL via API.
The time at which OAuth access tokens are refreshed.	Searched the self-managed chapter using the terms `refresh`, `expire`, and `before`.	Found `before it expires`. The number of seconds before expiration was not found.
How to send the client secret to the token endpoint using the client credentials grant type.	Searched the self-managed chapter and the OAuth secrets section using the terms `Authorization`, `Basic`, `header`, `body`, and `client_secret_post`.	Not found. Found `Lambda sends a client ID and client secret to the token endpoint.`
The SASL mechanism that Lambda uses for signing with `IAM_AUTH`.	Searched the self-managed chapter, the troubleshooting page, and the ESM logs page using the terms `AWS_MSK_IAM`, `saslMechanism`, and `IAM_AUTH`.	Not found. The example in the logs, `AWS_MSK_IAM`, is an example of an ESM that reads an Amazon MSK cluster.
The `saslMechanism` values in the ESM logs for `OAUTHBEARER_AUTH` and `IAM_OAUTHBEARER_AUTH`.	Searched the ESM logs page and the troubleshooting page using the terms `OAUTHBEARER` and `saslMechanism`.	Not found.
The API response from Lambda when specifying two or more authentication types.	Searched the self-managed chapter, the API reference, and CloudFormation using the terms `more than one`, `only one`, and `multiple`.	Not found. SAM is described as returning an error before deployment (Section 7.3).

8.4 What Changes When the Sources Are Updated

The following parts of this article change when the sources are updated. When you read this article, reread the following pages.

SourceAccessConfiguration in the Lambda API reference, CloudFormation, and SAM's SelfManagedKafka and MSK (Section 8.1, items 1, 4, 5, 7, and 8).
The self-managed chapter and the self-managed permissions page (Section 8.1, items 2 and 6).
The section on Amazon MSK (Section 6.5). This section may change with the addition of information on OAuth.
The page on Apache Kafka Broker Configs (Section 5.4). The version referenced may change with the release of a new version following version 4.3.

9. Frequently Asked Questions about Lambda's Kafka Authentication

Q1. Can OAuth be used with Amazon MSK's ESM?

The sources describe OAuth as a method of the self-managed ESM. The Amazon MSK chapter and its related pages do not mention OAUTHBEARER, and the "What's New" update from October 9, 2026, only discusses self-managed ESM. However, there are also no statements explicitly stating that OAuth cannot be used with Amazon MSK's ESM (Section 6.5).

Q2. If you replace the client secret in the OAuth secret, do you need to recreate the ESM?

No. The self-managed chapter states that you can rotate the credentials in the secret without recreating the ESM. It also notes that Lambda will read the new value and request a new OAuth access token (Section 4.4).

Q3. Can an ESM that uses SASL/SCRAM be switched to OAuth as it is?

The sources do not say. While the UpdateEventSourceMapping operation accepts SourceAccessConfigurations, no statement was found on whether it can change the type of authentication method. The "What's New" document from October 9, 2026, states that using OAuth requires creating a new ESM (Section 8.2).

Q4. How long are AWS web identity tokens obtained using `IAM_OAUTHBEARER_AUTH` valid for?

The sources do not say. While the default value for DurationSeconds in the GetWebIdentityToken API is 300 seconds, no statement was found on which value the ESM uses when it calls this API. The chapter states that Lambda requests a new token before the current AWS web identity token expires (see Sections 5.2 and 5.3).

Q5. Does `oauthTokenExpirationSeconds` represent the validity period of an OAuth access token?

No. The documentation describes this field as the validity period of the JWT bearer assertion that Lambda signs, specifying that it applies only to the JWT bearer grant type (Section 4.2).

Q6. What is required on the broker's side to use `IAM_OAUTHBEARER_AUTH`?

The documentation states that before creating an ESM, you must configure the broker to trust AWS as an OIDC identity provider (IdP). If the broker does not trust the AWS issuer, it will reject AWS web identity tokens, preventing Lambda from reading the topic. The ESM must pass the audience that the broker expects in the OAUTHBEARER_AUDIENCE entry (Section 5.2). Regarding AWS, the STS API reference states that on accounts where outbound web identity federation is not enabled, GetWebIdentityToken returns an error. The Lambda chapter does not mention this enablement (Section 5.3).

Q7. Which should you use: OAuth or IAM?

This article does not recommend one over the other. With the OAuth method, users store their OAuth secrets in Secrets Manager, while the brokers validate tokens from the user's IdP. With IAM_OAUTHBEARER_AUTH, users do not store anything, and the broker trusts AWS as the OIDC identity provider (IdP). IAM_AUTH is the method for Amazon MSK clusters reached through your own networking setup (see the table in Section 5.6).

10. Summary

The set of methods differs between the two chapters: The self-managed chapter has sections for five authentication methods: SASL/SCRAM (including SASL/PLAIN), mTLS, OAuth 2.0, IAM_AUTH, and IAM_OAUTHBEARER_AUTH. The Amazon MSK chapter covers four methods: unauthenticated access, SASL/SCRAM, mTLS, and IAM. It does not include SASL/PLAIN.
Three methods keep credentials in Secrets Manager and have no token to refresh: With SASL/SCRAM, SASL/PLAIN, and mTLS, Lambda reads the secrets using the execution role and presents them to the broker. For mTLS, the private key is in PKCS #8 format, and encryption is only supported using PBES1.
In OAuth, Lambda obtains, presents, and refreshes OAuth access tokens: The grant type is determined by the fields stored in the secret. Using fields for both grant types will result in a validation error. oauthTokenExpirationSeconds defines the validity period for the assertion used with JWT bearer grants. You do not need to recreate the ESM even if you replace the secret credentials.
The two methods do not require storing secrets: IAM_AUTH is the method for Amazon MSK clusters reached through your own networking setup, where Lambda signs each connection with the credentials of the execution role. With IAM_OAUTHBEARER_AUTH, Lambda obtains an AWS web identity token using sts:GetWebIdentityToken and presents it to the broker, refreshing it before it expires. A prerequisite for using this method is that the broker trusts AWS as an OIDC identity provider (IdP) before creating the ESM.
Amazon MSK's ESM does not fall back to weaker methods: If a secret is not provided, Lambda will select the strongest method that is active on the cluster, and if that fails, it will not attempt to use a weaker method.
Combinations are subject to rules: The four accompanying entries require an OAuth 2.0 authentication type, and only OAUTHBEARER_AUDIENCE can be combined with IAM_OAUTHBEARER_AUTH.
The sources disagree: this article lists eight places: the number of Type values, the IAM_AUTH permissions page, the "What's New" Previously … only, the value of MSK in SAM, the destination of the accompanying entries, the scope of the root CA, the characters representing the scope value, and the upper limit of an array. Ten statements were not found, including those on Amazon MSK's ESM OAuth, the dates when IAM_AUTH and IAM_OAUTHBEARER_AUTH were added, changes in authentication methods, and the validity period when ESM calls GetWebIdentityToken.

11. References

References:
Tech Blog with curated related content

Written by Hidekazu Konishi