Threat Modeling Generative AI Applications on AWS - STRIDE for RAG, Agents, and Tool Use

First Published: 2026-08-12
Last Updated: 2026-08-12

Ask a security architect to review a generative AI application and you will usually get one of two answers. The first is a list of controls to turn on, which is useful but arrives without an argument for why those controls and not others. The second is a shrug, because the reviewer knows how to reason about a web application and does not yet know how to reason about a system whose instructions and data arrive on the same channel.

This article is about producing the argument. It is a threat modeling method, not a control catalog: how to enumerate the assets a generative AI application actually holds, where its trust boundaries actually fall, how to draw a data flow that makes the boundary crossings visible, and how to read each STRIDE category in a way that surfaces the failure modes specific to retrieval, agents, and tool use. Every mitigation named below is paired with what it does not cover, because a threat model whose mitigations have no residual risk is a document that has stopped being useful.

Every AWS fact below was verified against AWS official documentation on 2026-08-09, with the page linked at the point of use. Nothing here was measured or executed. No agent was run, no injection was attempted, no guardrail configuration was changed, because the value of a threat model does not depend on demonstrating the attacks and the demonstration would be irresponsible. No attack procedures, payloads, or evasion techniques appear in this article — threats are described as outcomes to design against, not as recipes. No pricing figures appear, and every argument below is made in terms of blast radius, evidence, and review effort.

1. Introduction: Deriving What to Worry About
2. Why a Generic Threat Model Misses Here
3. Assets in a Generative AI Application
4. Where the Trust Boundaries Actually Are
5. Drawing the Data Flow
6. Spoofing in Generative AI Terms
7. Tampering in Generative AI Terms
8. Repudiation in Generative AI Terms
9. Information Disclosure in Generative AI Terms
10. Denial of Service in Generative AI Terms
11. Elevation of Privilege in Generative AI Terms
12. Indirect Input Is the Hard Part
13. Mapping Threats to Controls
14. Running the Exercise
15. Keeping the Model Current
16. Failure Modes
17. Frequently Asked Questions
18. Summary
19. References

1. Introduction: Deriving What to Worry About

The decision this article supports is narrow and comes up constantly: you have been handed a generative AI application and asked whether it is safe to ship, and you have to decide what to worry about before you can decide what to install. The controls are the easy half. The hard half is producing a defensible list of concerns, in an order, with reasons, that someone else can review and that survives the next architecture change.

Threat modeling is the established answer to that problem, and AWS names it as a foundational practice. The Well-Architected Framework Security Pillar states it as SEC01-BP07 Identify threats and prioritize mitigations using a threat model, and gives four core steps: identify assets, actors, entry points, components, use cases, and trust levels and put them in a design diagram; identify a list of threats; identify mitigations for each; and create and review a risk matrix to decide whether each threat is adequately mitigated.

Those four steps are correct and they are not the difficult part. The difficult part is step one, because the asset list and the trust levels for a generative AI application are not the ones you would write for a web application, and everything downstream inherits that error if you get it wrong. This article spends its first half on step one and its second half on reading the threat categories through the resulting picture.

1.1 Scope

In scope, in the order the table of contents gives them: why a generic threat model under-covers this class of system, the asset inventory and what confidentiality, integrity, and availability mean for each entry, where the trust boundaries actually fall and the invariant that keeps them honest, how to draw the data flow so the crossings are countable, a reading of all six STRIDE categories with the AWS controls that attach and the residual risk that remains, why indirect input is the hardest path, how to hand the result to a control mapping, and how to run and maintain the exercise.

1.2 Out of scope, with delegation

Which defenses to place at which layer. The layered placement model for AI agent defenses, including the four-layer split and its per-layer metrics, is developed in AI Agent Defense in Depth Model (AIDDM). This article stops at naming the threat; that article decides where the control lives.
The list-to-control correspondence. Mapping a published agentic risk list onto specific AWS controls is the subject of a companion article, OWASP Top 10 for Agentic Applications Mapped to AWS Controls. This article is the method for deriving what to look for; that one is the table of what to apply to what.
RAG architecture itself. Query planning, retrieval grading, reranking, and the correction loop are developed in Agentic RAG Architecture on Amazon Bedrock. Here a retrieval pipeline is the subject of a diagram, not the topic.
Guardrail implementation detail. Policy types, evaluation order, detection modes, and cross-Region profiles are covered in Amazon Bedrock Guardrails Implementation Deep Dive. This article names policies as mitigations and does not configure them.
Edge defenses against injection. Rate-based rules, size constraints, and pre-screening at the edge are covered in AWS WAF for Generative AI.
Vetting a third-party skill or extension. Static inspection of bundled instructions and scripts is covered in Agent Skills Security Vetting Guide.
Pricing. No figures appear here.

1.3 What this article deliberately omits

The useful content of a threat statement is the causal chain from an actor to an impact on an asset. The specific string that triggers a given failure is not part of that chain, it dates badly, and publishing it helps the wrong reader more than the right one. Every threat below is therefore illustrated by its outcome.

2. Why a Generic Threat Model Misses Here

2.1 The questions still hold

The structuring questions of threat modeling are unchanged. The Threat Modeling Manifesto defines threat modeling as analyzing representations of a system to highlight concerns about security and privacy characteristics, and organizes the work around four questions: what are we working on, what can go wrong, what are we going to do about it, and did we do a good enough job. The Manifesto carries no version number, so the honest citation is the document plus the date you read it, which for this article is 2026-08-09.

AWS uses the same four questions for this exact class of workload. The AWS Security Blog post Threat modeling your generative AI workload to evaluate security risk, published 2024-11-18, walks a generative AI chatbot through them and supplies a threat grammar and a documentation tool. AWS Prescriptive Guidance repeats the instruction for agentic systems, where Conduct threat modeling is best practice 2.1 in the secure development chapter and points at the OWASP Threat Modeling Cheat Sheet for process.

So the process is settled and this article does not propose a replacement. What is unsettled is the content you put into question one and question two.

2.2 Three assumptions that break

A conventional threat model rests on assumptions that hold for almost all software and fail here. Naming the three explicitly is what prevents a competent reviewer from producing a competent-looking model that misses the interesting half of the system.

Instructions and data arrive on the same channel. In a conventional application, a SQL statement and the string it embeds are different things at different times, and the entire discipline of parameterization exists to keep them apart. A language model receives one sequence of tokens. Your system instruction, the user's question, a retrieved document, and a tool's return value are typographically distinct to you and are the same kind of object to the model. Any threat model that assumes a parser separates control from content is modeling a system that does not exist here.

Behavior is not deterministic. Conventional threat modeling reasons over code paths, and a reviewer can assert that a branch is unreachable. AWS makes the consequence explicit in Secure AI agent access patterns to AWS resources using Model Context Protocol, whose first security principle is to assume all granted permissions could be used: static analysis of agent code tells you which tools are available, not which will be invoked or how. The permission, not the code path, is the thing to reason about.

The system acts, and it acts fast. A retrieval-only assistant produces text. An agent with tools produces effects. The same AWS post notes that an agent can make thousands of API calls in seconds, so a misconfigured permission scales at machine speed rather than at the pace of a human operator noticing. Availability and integrity threats that would be theoretical in a conventional design become the ordinary case.

2.3 What that implies for the exercise

Each broken assumption changes a specific step of the exercise rather than the exercise as a whole.

Broken assumption	What it changes in the threat model
Instructions and data share a channel	Every source of text in the context window becomes an entry point, so the entry point list is longer than the network-facing surface suggests
Behavior is not deterministic	Threats are enumerated against granted capability rather than against observed code paths
The system acts autonomously and quickly	Tampering, denial of service, and elevation of privilege apply to the agent's own actions, not only to an external actor's

That is the whole argument for a generative-AI-specific reading. The categories are the same; the inputs to the categories are not.

3. Assets in a Generative AI Application

An asset is the thing whose loss you are trying to prevent, and the reason to write the list down first is that a threat statement without a named asset cannot be prioritized. AWS builds this into its threat grammar, which ends with the impacted assets. The list below is the one worth starting from for a generative AI application, and most of it is invisible on an infrastructure diagram.

3.1 The asset inventory

Asset	Where it typically lives on AWS
System instructions and prompt templates	Application code, or Amazon Bedrock Prompt Management
Conversation history for the current session	Application state, or the session context of the runtime
Embeddings and the vector store	The vector store backing an Amazon Bedrock knowledge base
Ingested source documents	Amazon S3 and the other data sources a knowledge base syncs from
Tool definitions, schemas, and descriptions	An Amazon Bedrock AgentCore Gateway target, or the tool configuration in application code
Agent memory that persists across sessions	Amazon Bedrock AgentCore Memory
The right to invoke the model and the tools	IAM roles, OAuth tokens in an AgentCore Identity token vault, API keys in AWS Secrets Manager
Traces, prompts and responses, and audit records	Amazon CloudWatch, Amazon S3, AWS CloudTrail

3.2 What confidentiality, integrity, and availability mean per asset

The inventory is only useful once each entry has its three properties spelled out, because the same word means something different for each asset and the differences are where the interesting threats hide.

Asset	Confidentiality means	Integrity means	Availability means
System instructions	The wording, and the business rules it encodes, are not disclosed	Only a reviewed change alters behavior, and the change is versioned	The correct instruction is present on every invocation
Conversation history	One user's turns never appear in another user's context	Earlier turns are not silently rewritten or reordered	Context needed to continue the task is still there
Embeddings and vector store	Vectors are not readable by parties who cannot read the source text	Retrieval returns what was indexed, and no one has inserted content	Retrieval succeeds, and index rebuilds do not blank the store
Ingested documents	Only entitled requesters can cause a chunk to be retrieved	The corpus contains what your ingestion pipeline approved	Sync failures are visible rather than silently degrading answers
Tool definitions	The catalog does not reveal internal systems to unentitled callers	A tool's description and schema change only through review	The tool the agent needs is discoverable when required
Agent memory	Memories are scoped to the actor they belong to	A false memory cannot be planted and later believed	Memory retrieval failure degrades gracefully instead of hallucinating
Right to invoke	Credentials are not readable by the reasoning layer or by tool code	The set of granted actions is what the design intended	Throttling does not become an outage for entitled users
Traces and audit records	Prompts and responses in logs are handled at the sensitivity of their content	Records cannot be altered or selectively dropped	Records exist for the retention period the investigation will need

Two rows in that table repay a second reading. Integrity of the ingested corpus is a security property, not a data quality property, because a document that reaches the context window is read as instructions by the model. And confidentiality of the right to invoke is a distinct asset from the credentials themselves, because in an agentic system the capability is exercised by a component whose decisions you cannot fully predict.

3.3 The model is not the only asset

The instinct on first pass is to write down "the model" and stop. That produces a short and unhelpful list. If you use a managed foundation model, the weights are not yours to lose, and the interesting assets are the ones in the table above: the instructions that shape behavior, the corpus that supplies facts, the tool definitions that grant reach, the memory that persists influence, and the credentials that convert a suggestion into an effect.

The practical consequence is that most of your threat statements will name an asset that lives in a data store or an identity system, and the model will appear in the threat action rather than in the impacted asset. That is the correct shape, and a threat model in which every impacted asset is "the model" has almost certainly not been decomposed far enough.

3.4 Assign an owner and a classification first

Before enumerating threats, put two more columns on the inventory: who owns the asset, and how it is classified under your existing data policy. This is unglamorous and it resolves arguments later. When the exercise reaches a question like whether prompts may be retained in logs, the answer follows from the classification of the content of those prompts, which someone has already decided, rather than from a fresh debate among the people who happen to be in the room.

4. Where the Trust Boundaries Actually Are

4.1 Five inputs that are not yours

A trust boundary is where the level of trust changes. The OWASP Threat Modeling Process page calls the element a privilege boundary and defines it as representing the change of trust levels as data flows through the application, noting that boundaries show any location where the level of trust changes. In a generative AI application there are five inputs that sit on the far side of such a boundary, and the last one is the one teams get wrong.

End-user input. Obvious, and usually the only one that gets a control.
Ingested documents. Anything your pipeline pulls in, including content that a user or a partner can influence.
Tool return values. The response body of an API the agent called, including one you own.
Messages from other agents. In a multi-agent design, another agent's output is text produced by a model.
The model's own output. Including the tool call it proposes and the arguments it fills in.

AWS names the same set. The Well-Architected Agentic AI Lens best practice AGENTSEC08-BP01 Multi-layer input validation and prompt injection defense states that direct user messages, tool outputs, inter-agent messages, retrieved external content, and memory reads are all paths by which data reaches the agent's context, and that each needs a validation control. It also names external content retrieval as the most commonly missed surface.

4.2 The invariant: model output stays outside the boundary

The single design rule that keeps the rest of the model coherent is this: treat the model's output as untrusted input to whatever consumes it next. Not as a suggestion to sanity-check, and not as trusted because your own system produced it. The consumer might be a tool invocation, a rendering surface, a downstream service, a memory write, or another agent, and each of those is a boundary crossing that deserves its own check.

The reason is structural rather than a judgment about model quality. The model's output is a function of its context, its context contains text from the five sources above, and at least three of those are outside your control. Anything that can influence the context can influence the output, so the output inherits the trust level of the least trusted thing that reached the context window. That is what makes it a boundary and not a formality.

This invariant has a useful diagnostic property. Once you adopt it, a whole class of design errors becomes a single question you can ask about any edge on the diagram: does something on this edge take the model's word for it? Every yes is a finding, and the finding is legible to people who do not otherwise know how language models fail.

4.3 Five places the boundary gets drawn in the wrong spot

"It is our own S3 bucket, so the documents are trusted." The bucket is yours. The provenance of what landed in it is a separate question, and if any upload path is reachable by a customer, a partner, or an automated feed, the corpus is partly attacker-influenced.
"The tool is internal, so its response is trusted." Internal describes the network path and the ownership, not the content. An internal ticketing system returns text that a customer typed.
"The model is ours, so its output is trusted." Covered above. Ownership of the endpoint says nothing about the content of the context window.
"Memory holds our own data, so it is trusted." Memory holds what the system chose to persist, and what it chose was influenced by earlier untrusted input. AWS treats this as its own control area: AGENTSEC01-BP02 Validate and sanitize memory inputs sits alongside memory isolation and integrity in the Lens.
"The gateway authenticated the caller, so the request is trusted." Authentication establishes who is calling. It does not establish that the text in the request, or the text the tool will return, is safe to read as instructions.

4.4 Session and tenant boundaries

The boundary between one user's session and another's is the one with the clearest platform support and the most misunderstood residual. In the default serverless mode, Amazon Bedrock AgentCore Runtime gives each session a dedicated microVM with isolated CPU, memory, and filesystem, and AWS states that after session completion the entire microVM is terminated and memory is sanitized. Sessions persist for up to 8 hours of total runtime and terminate after 15 minutes of inactivity, per the AgentCore Runtime documentation.

Record which mode the workload runs on, because the boundary is not the same in both. Since 2026-08-06, AgentCore Runtime Instances runs a session as an isolated Amazon EC2 instance provisioned from a capacity provider, for a maximum of 14 days rather than 8 hours, and stopping a session terminates the instance but retains the session's persistent volumes so that a later invocation with the same session ID re-attaches them. AWS presents it as complementing the microVM-based option rather than replacing it. For the data flow in section 3 this changes two things and leaves the rest alone: the session store stops being ephemeral and becomes an asset with a retention question of its own, and the interval over which a compromised session can be reached grows from hours to days. The credential residual below applies to both modes unchanged.

One more difference belongs on the diagram rather than in a footnote, because it changes what the boundary encloses. On the serverless mode a session and a microVM are the same thing, so a boundary per session and a boundary per agent are indistinguishable. On Instances they are not. The security model for Runtime Instances states that the session is the isolation unit, identified by the pair of capacity provider and session ID and mapped one to one to an EC2 instance, and that agents on an instance are not isolated from each other: they can share the instance filesystem, and running them in containers or as separate processes provides no security boundary between workloads on the same instance, so agents that share an instance must be mutually trusted. If your design co-locates agents with different trust levels, that is a boundary you drew on the diagram and the platform does not implement. The offsetting fact is that the instances run in your own account and VPC, which puts your service control policies, permission boundaries, and VPC controls back in scope.

That is a strong boundary and AWS is unusually explicit about what it does not cover. The security best practices page for AgentCore Runtime states two things worth copying directly into your threat model. First, AgentCore does not enforce session-to-user mappings, so your client backend must maintain the relationship between users and their session IDs and implement lifecycle management such as a maximum number of sessions per user. Second, any code or actor running inside the microVM can access execution role credentials by calling the metadata endpoint, which means the isolation boundary contains the credentials and the scope of the execution role is the real limit on what a compromised session can reach.

Both belong on the diagram as assumptions, in the sense the Manifesto and Threat Composer use the word: statements you are relying on, recorded so that the next reviewer can challenge them.

5. Drawing the Data Flow

5.1 The notation

Use the standard data flow diagram notation rather than inventing one, because the value of the diagram is that a reviewer who has never seen your system can read it. The OWASP Threat Modeling Process page defines the element types: an external entity represents any entity outside the application that interacts with it via an entry point; a process is a task that handles data within the application; a multiple process presents a collection of subprocesses that can be broken down in another diagram; a data store represents a location where data is stored and does not modify it; a data flow represents data movement, with direction shown by the arrow; and a privilege boundary represents the change of trust levels as data flows through the application.

AWS asks for the same artifact. Its generative AI threat modeling post lists data flow diagrams that clearly illustrate the critical data flows of the application, from request to response, detailing what happens at each component or hop, as a minimum deliverable for the first question, alongside documented assumptions and business context.

5.2 What to draw for a retrieval application

For a retrieval-augmented application the diagram is small enough to fit on one page and still be complete. External entities: the end user, and every upstream system that can put a document into the corpus. Processes: the application front end, the orchestrator, the retrieval call, and the model invocation. Data stores: the document source, the vector store, the conversation state, and the log destination. Then draw the boundaries: one around your application, one separating each external entity, and — this is the addition people forget — one that puts the model's response outside the application's trusted interior until it has been checked.

5.3 What changes when you add an agent

Adding tools changes the diagram in three specific ways, and enumerating them is faster than redrawing from scratch.

Each tool becomes an external entity and a data flow in both directions, because the call carries data out and the response carries data in, and the response is untrusted.
The agent loop becomes a multiple process, because a single user request now produces an unbounded number of internal iterations, and the iteration count is itself a property worth bounding.
Memory becomes a data store that is both read and written inside the loop, which creates a path by which content from one session influences a later one.

In a multi-agent design, add one more: every other agent is an external entity whose output is model output, so the invariant in section 4.2 applies to it. AWS gives this its own best practice, AGENTSEC06-BP03 Establish trust boundaries between agents, which sits with message signing and coordination anomaly detection in the multi-agent orchestration question.

5.4 Count the crossing edges, not the components

The habit that makes the diagram pay for itself is to enumerate threats against edges that cross a boundary, not against components. A component-by-component pass tends to conclude that each component is individually fine, which is frequently true and rarely reassuring, because the interesting failures in these systems are compositions: a document crosses one boundary, influences a decision, and the decision crosses another boundary as a tool call. Neither component was broken. The path was.

Practically: list every arrow that crosses a boundary line, and for each one ask the six STRIDE questions from sections 6 to 11. That produces a finite list, it is obvious when you have finished, and the list is reviewable by someone who was not in the room.

Trust boundaries in a generative AI application, with untrusted inputs and model output outside the boundary

6. Spoofing in Generative AI Terms

STRIDE supplies the six questions. The category definitions used here are Microsoft's, from Threats - Microsoft Threat Modeling Tool (page date 2017-08-17, last updated 2026-03-04, read 2026-08-09), which is the maintained page; the older archived page The STRIDE Threat Model carries the same six definitions and is the version AWS links from its own threat modeling material. The generative AI reading in sections 6 to 11 is this article's extension and is not part of either source.

6.1 How it shows up

Microsoft defines spoofing as illegally accessing and then using another user's authentication information. The generative AI reading has three distinct shapes. The caller is not who the agent thinks it is, which is the classic case. A tool is not the tool the agent thinks it is, which arises when tool discovery is dynamic and the catalog can change between the review and the invocation. A message claiming to come from another agent did not come from that agent, which is the multi-agent case and is the one with the least mature tooling.

A fourth shape is specific to this class of system and easy to miss: the agent's own identity is conflated with the user's. If the agent calls a downstream service with a single service role for every user, the downstream service cannot tell whose request it is serving, and every access decision it makes is wrong in the same direction.

6.2 Where AWS controls attach

Inbound authentication for the agent is the first attachment point. Amazon Bedrock AgentCore Gateway requires an inbound authorization configuration on every gateway, and the core concepts page documents four types: OAuth using JWT, IAM using AWS Signature Version 4, an authenticate-only mode that validates the token and delegates authorization to the target, and no authorization, which AWS scopes to development and testing.

Outbound identity is the second. AgentCore Identity handles both directions, and the AWS Security Blog post Securing AI agents with Amazon Bedrock AgentCore Identity describes a centralized agent identity directory that gives each agent a distinct ARN, plus a token vault that stores OAuth access and refresh tokens, API keys, and client secrets encrypted with AWS KMS keys including customer managed keys. AWS states that access is limited to individual agents, and that for user-specific credentials agents are restricted to accessing them solely on behalf of the associated user.

For the identity-conflation shape, the Lens is direct: AGENTSEC03-BP02 Separate agent and human user permission is a named best practice, and the anti-pattern listed under tool authorization is failing to propagate user identity context through tool invocations, so downstream services cannot enforce user-level access controls and every call runs with the agent's permissions.

6.3 Residual risk

Authentication establishes the caller, and nothing about it constrains the content the authenticated caller supplies. That is the whole reason section 4.3 lists the authenticated-gateway assumption as a boundary error.

Tool identity remains the weaker half. Verifying that a gateway is reachable over a trusted path is not the same as verifying that a tool definition is the one you reviewed, and the Lens acknowledges the gap by making AGENTSEC02-BP03 Maintain approved tool registry with security assessments a separate practice from tool authorization. A registry is a process control, and it holds only as long as the process runs. If your agent can discover tools it has never been reviewed against, record that as an accepted risk with an owner, not as a solved problem.

7. Tampering in Generative AI Terms

7.1 How it shows up

Microsoft defines tampering as malicious modification of data, including unauthorized changes to persistent data and alteration of data in flight. Four assets from section 3.1 are the ones that matter here.

System instructions. A change to the prompt template changes behavior everywhere, and if the template lives in application code without review discipline, the change looks like a routine commit.

The ingested corpus. This is the highest-value tampering target in most retrieval designs, because a modified document does not merely produce a wrong answer, it produces text that the model reads as instructions the next time it is retrieved.

Tool definitions. A tool's description is prompt input, so an edit to a description is an edit to the agent's instructions with none of the review that a prompt change would attract.

Memory. A planted memory persists and is believed later, and unlike a single-turn injection it survives the session in which it was introduced. AWS gives it a dedicated question, AGENTSEC01, with memory isolation and integrity controls, memory input validation, and monitoring for hallucination propagation as its three best practices.

7.2 Where AWS controls attach

Treat prompts as code. AWS Prescriptive Guidance makes this best practice 2.2, treat prompts as code artifacts, describing a hybrid where a version control system is the auditable source of truth and a registry provides accessible authoring. On the AWS side, Amazon Bedrock Prompt Management supports versions through CreatePromptVersion, where a version is a snapshot created when you are satisfied with a configuration and versions are numbered incrementally from 1.

For memory, Amazon Bedrock AgentCore Memory organizes short-term memory by actor and session and long-term memory by namespace, with the memory organization page describing hierarchical namespaces built from actor, session, and strategy identifiers so that extracted memories are scoped rather than pooled. Memory resources accept an event expiry duration and an optional customer managed KMS key.

For tool definitions, the attachment point is change control on the gateway target plus the approved-registry practice from section 6.2. For the corpus, it is the ingestion pipeline: versioning on the source bucket, an approval step that is a real gate rather than a log line, and a record of which source produced which chunk.

7.3 Residual risk

Versioning tells you that a change happened and lets you roll it back. It does not tell you that the change was safe, and a reviewed prompt change can still be a harmful one. The control converts an invisible risk into a reviewable one, which is the honest claim.

Corpus integrity has a sharper residual. Provenance metadata records where a chunk came from; it does not establish that the origin system was not itself influenced. If a feed originates outside your control, the corpus is partly attacker-influenced no matter how carefully you version the bucket, and the mitigation belongs in section 12 rather than here.

Memory scoping prevents one actor's memories from surfacing in another actor's namespace. It does not prevent a false memory being written within the correct namespace by the correct actor, which is precisely the case where memory input validation is doing the work and where AWS lists hallucination propagation monitoring as a separate, medium-risk practice rather than folding it into isolation.

8. Repudiation in Generative AI Terms

8.1 How it shows up

Microsoft's definition centers on users who deny performing an action without other parties having any way to prove otherwise. In a generative AI application the disputed action is frequently not a user's but the system's, and the question that arrives after an incident is some version of: what did the agent do, on whose behalf, with what context, and why did it decide that.

Answering it requires four things that are easy to have three of. You need the request and who authenticated it; the context the model actually saw, including which chunks were retrieved and which memories were read; the tool calls with their arguments and results; and the linkage that ties all of it to one identifier. Missing the fourth is the common case, and it converts a five-minute lookup into an afternoon of timestamp correlation.

8.2 Where AWS controls attach

Amazon Bedrock model invocation logging collects invocation logs, model input data, and model output data for model invocations in the account, with delivery to Amazon CloudWatch Logs, Amazon S3, or both, per the model invocation logging documentation. AWS notes that when model input or output exceeds 100 KB or is binary, that data is delivered to S3.

For the agent layer, AgentCore Observability emits distributed traces, span-level logs, and metrics under the bedrock-agentcore CloudWatch namespace following the OpenTelemetry protocol, viewable on the CloudWatch generative AI observability page. The configuration documentation explains that agents hosted on AgentCore Runtime are instrumented automatically and that agents running elsewhere need the AWS Distro for OpenTelemetry SDK added; CloudWatch Transaction Search is a one-time per-account prerequisite before traces and spans become available.

The linkage problem has its own best practice. AGENTOPS05-BP01 Establish end-to-end tracing and telemetry for agent operations states that W3C Trace Context propagation across all agent boundaries maintains continuity in distributed workflows, so a request passing through five agents produces one trace with five spans rather than five disconnected traces, and that without propagation multi-agent debugging becomes manual correlation by timestamp, which produces incorrect answers when concurrent requests overlap. On the security side, AGENTSEC05 pairs comprehensive logging and decision artifact storage with distributed tracing for agent interactions.

8.3 Residual risk

Logging prompts and responses turns the log destination into a store of whatever sensitivity the conversation carried, which moves the problem rather than removing it. The retention period, the encryption key, and the access policy on that destination are now part of the confidentiality analysis for conversation history in section 3.2, and it is common to see an organization tighten its application-side data handling while leaving the invocation log open to a broad internal audience.

Sampling is the second residual. AGENTOPS05-BP01 recommends capturing all error traces and a configurable percentage of successful ones to balance visibility against cost. That is sound operationally and it means the trace for the request you eventually care about may not exist. If your dispute-resolution story depends on complete traces for a defined class of high-consequence action, that class needs to be exempt from sampling by design, and the exemption should be written down.

Finally, a trace records what the system did, and it does not record why the model produced the reasoning it did. A complete trace narrows an investigation to a decision point. It does not explain the decision.

9. Information Disclosure in Generative AI Terms

9.1 How it shows up

Microsoft defines information disclosure as exposure of information to individuals not supposed to have access to it. The generative AI reading has four paths worth separating, because they have different controls.

Retrieval returns chunks the requester is not entitled to. The most common serious finding in enterprise retrieval deployments, and a pure authorization problem wearing a search costume.

Cross-session or cross-tenant leakage. One user's context appearing in another's, whether through shared memory, cached state, or a session mapping error.

The response discloses sensitive content the corpus legitimately contained. The retrieval was authorized; the rendering was not appropriate.

System instructions are disclosed. Whether this matters depends on whether the instructions encode anything you would not publish, which is a question for section 3.4 rather than a universal yes.

9.2 Where AWS controls attach

For entitlement at retrieval time, Amazon Bedrock Knowledge Bases supports metadata filtering on Retrieve and RetrieveAndGenerate through the retrievalConfiguration.vectorSearchConfiguration.filter field, with comparison operators and logical operators that combine up to five filters, documented in Configure and customize queries for managed knowledge bases. AWS describes using this for access control by defining metadata fields for attributes such as user roles, departments, or sensitivity, and deriving the filter from the caller's authenticated identity.

For output handling, Amazon Bedrock Guardrails provides sensitive information filters that detect personally identifiable information in standard formats or custom regex entities in user inputs and model responses, and can block or mask them, per the guardrail components documentation. The Lens names output filtering for sensitive information as AGENTSEC08-BP02.

For cross-session leakage, the session isolation in section 4.4 plus the actor and namespace scoping of AgentCore Memory are the platform-side controls, with the session-to-user mapping remaining your responsibility. On Runtime Instances, add the retained session volumes to the same question: isolation between sessions still holds, but the data no longer disappears when the session ends.

9.3 Residual risk

Metadata filtering is only as good as the metadata, and the metadata is applied at ingestion time by a pipeline that is usually less scrutinized than the retrieval path. A document whose sensitivity label was never set, or was set by a default, will be retrieved by anyone whose filter does not exclude it. The filter is a positive control over what is returned and it is not a negative control over what was indexed incorrectly.

The filter is also constructed by your application, which means it is only trustworthy if it is derived from an authenticated claim rather than from anything the model produced. Deriving a filter from model output is a direct violation of the invariant in section 4.2, and it is a pattern that appears in prototypes because it is convenient.

Sensitive information filters are probabilistic. AWS describes the sensitive information filter as a probabilistic machine learning based solution that is context dependent. Treat it as a second line behind entitlement, not as the reason entitlement can be loose.

Embeddings deserve one explicit sentence, because teams frequently assume the opposite: a vector is a derived representation of the source text, not an anonymization of it, so the access controls on the vector store should be reasoned about as controls on the underlying content.

10. Denial of Service in Generative AI Terms

10.1 How it shows up

Microsoft's definition is attacks that deny service to valid users. Three shapes matter here, and only the first resembles the conventional case.

Conventional flooding of the endpoint. Handled by edge controls, and delegated in this article to AWS WAF for Generative AI.

Resource exhaustion through amplification. One request that expands into a large retrieval, a long context, and many model invocations. The multiplier is the property to bound, and it is a design parameter rather than a traffic property.

The agent denying service to itself. An unbounded loop, a tool that fails in a way the loop retries indefinitely, or a plan that consumes the session budget before reaching the goal. No external actor is required, which is why this shape is missed by a threat model that only considers adversaries.

10.2 Where AWS controls attach

Bound the loop in the design. The Lens question AGENTOPS05 includes tracking iteration count as a span field, and AGENTREL08 covers deciding when and where graceful degradation is appropriate; the practical control is that the orchestrator has a hard iteration limit and a hard tool-call budget, and that exceeding either is an event rather than a silent stall.

Bound the session. AgentCore Runtime terminates sessions after 15 minutes of inactivity and at 8 hours of total runtime, which caps the duration of a runaway loop even when the application has not implemented its own bound. Treat that as a backstop, not as the control.

Bound the fan-out. Retrieval configuration decides how many chunks enter the context, and tool authorization decides which tools an iteration may call at all, which is why the tool authorization control in section 11 also does availability work.

10.3 Residual risk

Every bound is a tradeoff against task completion, and a limit tight enough to stop a pathological loop will sometimes stop a legitimate long task. Decide which failure you prefer, write the decision down, and make the truncation visible to the user rather than silently returning a partial answer as though it were complete.

Quotas protect the platform and they do not protect a tenant from another tenant inside your own account. If one customer's workload can consume the shared throughput your account has available, you have a noisy-neighbor availability threat that no AWS-side control resolves on your behalf, and it belongs in the model as a design question about isolation.

11. Elevation of Privilege in Generative AI Terms

11.1 How it shows up

Microsoft defines elevation of privilege as an unprivileged user gaining privileged access sufficient to compromise or destroy the system. In an agentic system the elevation frequently happens without any privilege being granted at the moment of the attack, because the privilege was granted at design time and is simply exercised in a way nobody intended.

Three shapes. The agent acts with more authority than the requesting user has, which is the confused deputy in its classic form and the direct consequence of not propagating user identity. Tools compose into an outcome no single tool authorizes, where each call is permitted and the sequence achieves something that was never reviewed. Untrusted content steers a permitted action, where nothing is bypassed and the reasoning is redirected.

AWS Prescriptive Guidance describes the composition shape plainly in its generative AI security reference architecture, noting that agents can escalate privileges by combining low-privilege tools to achieve high-privilege outcomes and that the autonomous nature of agents makes it difficult to predict all possible execution paths.

11.2 Where AWS controls attach

Least privilege on the execution role is the floor, and it does more work here than in a conventional design because of the credential exposure noted in section 4.4: any code in the microVM can reach the execution role credentials, so the role's policy is the effective boundary. The Generative AI Lens best practice GENSEC05-BP01 names excessive agency as the risk and recommends permissions boundaries in addition to least-privilege role policies.

Authorization outside the agent's code is the control that addresses the composition shape. Policy in AgentCore intercepts agent traffic through AgentCore Gateway and evaluates each request before allowing tool access, using the Cedar policy language, with a default of denying all actions. The documentation describes enforcement at the boundary outside the agent's code, so it holds regardless of how the agent is implemented, and states that every enforcement decision is logged through CloudWatch metrics and logs. A Cedar policy names a principal, an action, and a resource, and can add conditions over the tool's input parameters. The following is illustrative of the shape rather than copied from a deployment; the action names come from the Cedar schema AWS generates from the gateway's tool input schemas.

permit(
    principal,
    action == AgentCore::Action::"TicketTarget___close_ticket",
    resource == AgentCore::Gateway::"<gateway-arn>"
) when {
    context.input.assigned_team == principal.team
};

AWS also documents a second, open-source policy language for the same engine, Dogwood, whose policies can combine input-based rules with session-aware temporal conditions that decide based on what has already happened in the same session — requiring that an approval was granted before a transfer, blocking an action after it has run a set number of times, or keeping a running total under a budget — and with conditions that consult an information provider such as Guardrails and act on the emitted score. The temporal form is the one that matters for this threat category, because the composition shape is precisely a threat about sequences rather than about individual calls.

Human review for consequential actions is the third control. AGENTSEC02-BP01 Implement tool authorization states the desired outcome as authorizing every tool invocation against a defined policy before execution, with agent identity and user context propagated through the authorization chain, and with human-in-the-loop checkpoints intercepting high-risk mutating operations. It lists skipping those checkpoints because they add latency as an anti-pattern that accepts unbounded risk for actions that are difficult or impossible to reverse.

11.3 Residual risk

Policy enforcement at the gateway covers what passes through the gateway. Anything the agent reaches by another path — a direct SDK call from inside the runtime using the execution role, for instance — is governed by IAM alone, and the completeness of your tool authorization story depends on there being no such path. That is an assumption to write down and verify, not one to presume.

Attribute-based conditions can only reference attributes that reach the policy engine. A condition on the requesting user's team requires that claim to be propagated; if the agent calls with its own identity, the condition has nothing to evaluate and the policy is weaker than it reads.

Human-in-the-loop degrades with volume. A reviewer approving a handful of consequential actions per day reads them; a reviewer approving hundreds does not. The Lens treats this as a real failure mode rather than an implementation detail, with AGENTSEC07 covering cognitive load management, confidence indicators and manipulation warnings, and multiple reviewers for critical operations. If your design depends on human review, the review rate is a security parameter.

12. Indirect Input Is the Hard Part

12.1 Three paths, three difficulties

Direct user input is the easy case: it is one edge, it is obviously untrusted, and it gets a control. The three indirect paths are harder for reasons that differ, and the differences determine what you can do about them.

Path	Why it is hard	What follows for design
Ingested documents	The content enters long before the request, so the request-time check has no signal that anything is unusual	Validate at ingestion as well as at retrieval, and record provenance per chunk
Tool return values	The volume is high, the format is machine-oriented, and teams treat their own APIs as trusted	Apply a check at the tool boundary rather than only at the user boundary
Messages from other agents	The sender is authenticated, so the message looks trustworthy at exactly the layer that usually decides	Authenticate the sender and still treat the payload as model output

AWS singles out the first as the commonly missed one. AGENTSEC08-BP01 states that when an agent uses retrieval, web browsing, or API calls to gather information during task execution, the retrieved content becomes part of the agent's context, and that adversarial instructions embedded in that content influence the agent's behavior as effectively as a direct user injection.

12.2 What detection can and cannot do

The available detection is a classifier, and the honest framing is that it reduces the rate of a class of input reaching the context, not that it establishes the input is safe.

Amazon Bedrock Guardrails offers prompt attacks as a category within content filters, described in the guardrail components documentation as helping detect and filter prompt attacks including jailbreaks, prompt injections, and prompt leakages. AGENTSEC08-BP01 recommends applying Guardrails with prompt attack detection to all retrieved content before it enters the agent's context, and notes that the ApplyGuardrail API provides a unified mechanism so one configured guardrail can be applied at each input boundary with surface-specific tuning through versioning.

The same best practice distinguishes two assessment modes and gives a rule for choosing. Block mode returns a binary allow or deny; detect mode returns confidence scores without blocking. AWS recommends block mode for prompt attack detection, where even low-confidence matches warrant intervention given the severity of potential impact, and detect mode for content safety filters on internal or lower-risk applications where the application can make risk-proportionate decisions from the scores.

Because detection is a classifier, the boundary between what you enforce and what you observe is a design decision that should be recorded with the reasoning behind it, and the classifier's failure to fire should never be the sole reason a downstream action is permitted.

12.3 The isolation moves

Detection reduces the rate. Isolation reduces the consequence, and it is where the durable engineering is.

Label provenance and carry it. A chunk that came from an externally influenced source should be distinguishable inside the context, and the distinction should survive into the decision about what the agent may do next.
Reduce capability after an untrusted read. The simplest robust pattern in this whole area: once the context contains externally influenced content, the set of permitted actions narrows, and anything mutating requires confirmation. Policy in AgentCore expresses this as a session-aware temporal condition rather than as application logic, which matters because application logic is inside the blast radius and policy at the gateway is not.
Constrain egress from the components that execute. AgentCore Code Interpreter supports sandbox, public, and VPC network modes, described on the resource management page, where sandbox mode provides limited external network access to AWS services and VPC mode connects to private resources while maintaining isolation from the public internet. Choosing public mode because a package install needed it during development is a decision that should reappear in the threat model, not one that quietly persists into production.
Check at the tool boundary, not only at the user boundary. AgentCore Gateway supports request and response interceptor Lambda functions for non-streaming MCP operations, per the gateway target documentation, which gives a place to apply a check to a tool's response before it reaches the agent.
Require confirmation for the irreversible. Reserved for the small set of actions where the cost of being wrong exceeds the cost of the latency, chosen deliberately so the reviewer's attention is not spent on routine approvals.

12.4 What stays unsolved

No control in the list above prevents a model from being influenced by content it reads. The claim available today is that you can lower the probability that adversarial content arrives, lower the capability available when it does, and raise the probability that you find out afterward. Anyone who tells you the problem is solved is selling something, and a threat model that records prompt injection as fully mitigated is a threat model that will be embarrassing later.

The practical implication for the exercise: for each threat in this family, write the residual risk as a sentence about what an attacker could still achieve given the controls, and get someone with authority to accept it. That sentence is the deliverable, and it is far more useful in six months than a checkbox.

13. Mapping Threats to Controls

13.1 Write the threat down in a fixed grammar

An enumerated threat is only reviewable if it is written consistently. AWS publishes the grammar it uses, in the generative AI threat modeling post: a threat statement follows the syntax of a threat source with prerequisites performing a threat action, leading to a threat impact, negatively impacting the impacted assets.

A [threat source] with [prerequisites]
can [threat action]
which leads to [threat impact],
negatively impacting [impacted assets].

The grammar does two things that matter for this article's method. It forces you to name the impacted asset, which is why section 3 comes first; and it forces you to state the prerequisites, which is where the trust boundary from section 4 becomes explicit rather than assumed. AWS notes that its customers most commonly add the STRIDE category and a priority as metadata on each statement, and recommends associating at least one preventative and one detective control with each threat.

13.2 The asset and category matrix

Crossing the asset inventory with the six categories gives a grid that is finite, that shows its own gaps, and that is a better starting point than a blank page. The point of the grid is not that every cell is a real threat in your system; it is that an empty cell is a claim you have made, and claims are reviewable.

Assets in a generative AI application mapped to the six STRIDE categories

The same content in table form, for readers who arrive without the diagram:

Asset	Spoofing	Tampering	Repudiation	Information disclosure	Denial of service	Elevation of privilege
System instructions	Impersonated persona	Unreviewed edit	No version trail	Instruction leakage	Missing on invoke	Role override
Conversation history	Wrong session bound	Turns rewritten	Turn not logged	Cross-user leakage	Context lost	Prior turn escalates
Embeddings and vector store	Fake index source	Inserted content	Change unlogged	Vector reveals text	Index rebuild loss	Filter bypass
Ingested documents	Forged source	Poisoned corpus	Provenance missing	Unentitled retrieval	Sync failure	Instructions in text
Tool definitions	Substituted tool	Description edited	Change unlogged	Catalog exposure	Not discoverable	Scope widened
Agent memory	Wrong actor scope	Planted memory	Write unattributed	Cross-actor read	Retrieval fails	Memory grants trust
Right to invoke	Shared service role	Policy widened	Actor not propagated	Credential read	Throttled out	Tool composition
Traces and audit records	Forged trace ID	Records altered	Records dropped	Prompts in logs	Retention too short	Log access abused

13.3 Where the AWS-side mapping lives

Once the threats are written, they need controls, and the useful move is to map onto a published control set rather than inventing one, so that a reviewer can check your coverage against something external.

On the AWS side, the Well-Architected Agentic AI Lens security pillar organizes its best practices under nine questions — secure agent memory and state, secure agent tool usage, agent identity and permission management, agent goal alignment and manipulation prevention, agent observability and non-repudiation, secure multi-agent orchestration, human oversight protection and agent containment, secure agent inputs and outputs, and agent vulnerability scanning and penetration testing — with the complete list of best practice identifiers and risk levels in Appendix A. Citing an AGENTSEC identifier next to a mitigation is cheap and makes the mitigation auditable.

Mapping onto the published agentic risk lists is the companion article's job rather than this one's: OWASP Top 10 for Agentic Applications Mapped to AWS Controls. The division is deliberate. This article produces the threat list from your architecture; that one supplies the correspondence between named risks and named AWS controls. Use them in that order, because a control list consulted before the assets are enumerated tends to become the scope rather than the response to it.

13.4 Record it somewhere a reviewer can open

AWS publishes Threat Composer, an open-source tool for documenting a threat model, and both the generative AI threat modeling post and Prescriptive Guidance best practice 2.1 name it. Whether you use it or a document in your repository matters less than that the artifact contains the four things a later reviewer needs: the data flow diagram, the assumptions, the threat statements with their assets and categories, and the mitigations with identifiers so they can be referenced from more than one threat.

14. Running the Exercise

14.1 Who is in the room

A threat model built by the security reviewer alone produces threats the reviewer can imagine, which is a smaller set than the ones the system has. Four perspectives cover most of the space: the person who can describe the data flow without guessing; the person who knows what the prompts and tool definitions actually say today; the person who knows what the downstream systems will do when called; and the person who can decide that a residual risk is acceptable. The last one is frequently omitted, and omitting them is why threat models end in a list of open items nobody closes.

AWS treats the facilitation role as a mechanism rather than an accident. Its generative AI threat modeling post notes that at AWS a threat model is a required input to the application security process and that builder teams get support from a Security Guardian to build threat models for their features or services.

14.2 Where to stop

The Manifesto's anti-patterns name the two failure directions precisely: admiration for the problem, meaning going deeper into analysis than into fixes, and perfect representation, meaning polishing the diagram past the point of usefulness. A workable stopping rule for this class of system is to stop when every boundary-crossing edge on the diagram has been asked the six questions, every threat you kept has at least one preventative and one detective control, and every control has a residual risk sentence with a name against it.

Depth is a separate axis from coverage. Go one level deeper only where the impact is high, and record the decision not to decompose further as an assumption rather than leaving it implicit.

14.3 What to keep

Five artifacts, and the fifth is the one most often skipped.

The data flow diagram with boundaries drawn.
The assumptions, written as statements someone could later prove wrong.
The threat statements in the fixed grammar, with asset and category metadata.
The mitigations with stable identifiers, so one control can be referenced from several threats. AWS notes that discovering a single control appears in most of your mitigations is itself a prioritization signal.
The residual risk register: for each mitigation, what it does not cover, who accepted that, and when it should be revisited.

14.4 Connecting to decision records

A threat model produces decisions, and decisions that live only inside the threat model document are invisible to the people making the next architectural change. Where a threat leads to a structural choice — putting authorization at the gateway rather than in the agent, running the interpreter without public network access, requiring confirmation for a class of action — record it as an architecture decision record with a link back to the threat that motivated it, using the patterns in Architecture Decision Records. The link is what stops a later change from quietly reversing a security decision whose reason nobody remembers.

15. Keeping the Model Current

15.1 The triggers

SEC01-BP07 states that a threat model is a living document that should continue to evolve as your workloads change, and that you should revisit it when there is a major change, a change in the threat landscape, or when you adopt a new feature or service. For this class of system the triggers are specific enough to enumerate, which makes them automatable as pull request checks.

Trigger	What to re-review
A tool is added or its schema changes	The whole of section 15.2
A data source is added to the corpus	Provenance, entitlement metadata, ingestion-time validation
The system instruction changes materially	Whether any assumption in the model depended on the old wording
A new external content source is introduced	Section 12, and whether egress constraints still hold
A second agent is added	Section 4.1 item 4, message authentication, and trace propagation
The model or inference configuration changes	Guardrail behavior, output schema assumptions, latency budgets for human review
An IAM policy on the execution role widens	Section 11 in full

15.2 The new-tool checklist

Adding a tool is the highest-frequency change and the one most likely to be treated as routine. Seven questions, answerable in a few minutes each, catch most of what matters.

What can this tool do that no existing tool could, and is that new capability inside the agent's intended scope?
Does it mutate anything, and if so, is it reversible?
Whose authority does it act with, and does the caller's identity reach the downstream system?
What does its response contain, and does that response now need a check at the tool boundary?
Can it reach the network, and if so, where?
Does it compose with an existing tool into an outcome neither authorizes alone?
Which authorization policy governs it, and was that policy updated in the same change?

Question six is the one that requires thought rather than lookup, and it is the reason a tool addition deserves a review rather than a checklist alone.

15.3 What to automate

Some of the re-review is mechanical and belongs in a pipeline rather than in a meeting. The tool registry can be diffed against the deployed gateway targets so that an undeclared tool is a build failure. The execution role's policy can be compared against an expected set so that a widening is visible. Guardrail attachment can be enforced rather than reviewed: the bedrock:GuardrailIdentifier condition key lets an IAM policy deny inference requests that do not include the configured guardrail, and AWS documents its use with Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream in Enforce the use of specific guardrails in model inference requests.

{
  "Sid": "DenyInferenceWithoutTheApprovedGuardrail",
  "Effect": "Deny",
  "Action": [
    "bedrock:InvokeModel",
    "bedrock:InvokeModelWithResponseStream"
  ],
  "Resource": ["arn:aws:bedrock:us-east-1::foundation-model/*"],
  "Condition": {
    "StringNotEquals": {
      "bedrock:GuardrailIdentifier": "arn:aws:bedrock:us-east-1:123456789012:guardrail/guardrail-id:1"
    }
  }
}

The value of moving a check into a policy is that it stops being a thing a reviewer might notice and becomes a thing the platform enforces. That is worth doing for any assumption in your model that is expressible as a condition, and the ones that are not expressible are the ones your review time should be spent on. AGENTOPS02-BP02, configuration drift detection and remediation, is the Lens name for this general move.

16. Failure Modes

16.1 Treating model output as trusted downstream

The most common and the most consequential. It appears as a tool call whose arguments were not validated against a schema, a rendered link the model composed, a downstream call that forwards the model's text unexamined, or a memory write of content the model produced. Each is one edge, each is cheap to fix, and the fix only happens if the invariant in section 4.2 is written down where designers will see it.

16.2 Granting broad tool permissions on day one

Prototypes get an execution role that works, and the role survives into production because narrowing it is a task with no deadline. AWS's framing is the useful one: assume all granted permissions could be used, because you cannot predict which the agent will invoke. The mitigation is procedural — the role is part of the change, reviewed with the change — and it is easier to establish at the start of a project than at the end.

16.3 Treating all ingestion paths as equally trusted

A corpus assembled from an internal wiki, a customer-facing upload form, and a partner feed has three different trust levels and usually one retrieval path. Without provenance per chunk, the design cannot distinguish them at request time even if the team understands the difference.

16.4 Building the model once

A threat model produced for a launch review and never reopened describes a system that no longer exists. The tool set in particular changes faster than any other part of the architecture, and it is the part that most directly determines blast radius. Section 15.1 exists to make the revisit a trigger rather than an intention.

16.5 Recording mitigations without residual risk

A mitigation column with no residual column reads as though the threats are closed. They are not, and the reader who most needs to know that is the one deciding whether to expand the system's autonomy next quarter. The residual sentence is the part of the artifact with the longest useful life.

16.6 Modeling components instead of compositions

Each component passes its own review, and the system fails through a path that crosses several of them. This is the specific reason section 5.4 recommends enumerating boundary-crossing edges rather than components, and the specific reason session-aware policy conditions are more interesting here than per-call conditions.

16.7 Letting the diagram drift from the deployment

A diagram that no longer matches the system is worse than no diagram, because it is believed. Tie it to something that changes when the system changes: keep it in the repository next to the code, review it in the same pull request that adds a tool, and let the tool registry diff in section 15.3 fail the build when the two disagree.

17. Frequently Asked Questions

Do I need a separate threat model for the AI parts, or one model for the whole application?

One model for the workload. AWS is explicit that threat modeling is most effective at the workload or workload feature level so that all context is available, and its generative AI post makes the point that large language models are only one portion of a larger workload and that well-known threats such as injection and credential compromise still apply. What changes is the content of the asset list and the trust levels, not the number of documents.

Is STRIDE the right framework for this, or should I use something AI-specific?

STRIDE is a categorization aid, and its value here is that it is six prompts that produce coverage, not that it was designed for this technology. AWS's own guidance suggests using STRIDE alongside repositories of AI-specific threat knowledge rather than instead of them. The reading in sections 6 to 11 is this article's extension for that purpose; the six category definitions themselves are Microsoft's and unmodified.

Where does the AI-specific risk list fit relative to STRIDE?

They answer different questions. STRIDE asks what kind of thing could go wrong at this boundary. A published risk list tells you what the community has already seen go wrong, so you are not relying on the imagination of the people in the room. Use STRIDE for systematic coverage of your own diagram, and the list as a cross-check against it. The correspondence between such a list and specific AWS controls is the subject of a companion article, OWASP Top 10 for Agentic Applications Mapped to AWS Controls.

How is this different from the defense in depth model on this site?

Different stage of the same workflow. This article derives what to worry about. AI Agent Defense in Depth Model (AIDDM) decides which layer each chosen defense belongs to and how to tell whether that layer is working. Enumerate first, then place.

Should I write attack steps for each threat?

For the highest-priority threats only. AWS describes attack steps as an optional activity and recommends identifying some for the highest-priority threats, on the grounds that specific mitigations attach to specific attack steps. Note that an attack step in a threat model is a description of a mechanism at design level, not a working procedure, and there is no reason for a threat model to contain a payload.

Is prompt injection a Spoofing threat or a Tampering threat?

It depends on the asset and the effect, which is a feature of the question rather than a defect in the framework. Content that redirects the agent into acting outside its intended authority is elevation of privilege against the right to invoke. Content planted in the corpus is tampering against the ingested documents. Content that causes disclosure is information disclosure against whatever was disclosed. If the category argument takes more than a minute, record both categories and move on; the category is metadata for sorting, not the finding.

Do I have to treat my own internal API's response as untrusted?

Treat the content as untrusted; you can still treat the transport and the identity as trusted. The distinction matters because the two get different controls. Mutual authentication and a private path address who you are talking to. Neither addresses the fact that the response body may contain text a customer typed into a form last week.

Does session isolation mean I do not need to think about cross-user leakage?

No, and AWS says so in the documentation for the feature. The session boundary is strong, and AgentCore does not enforce session-to-user mappings, so maintaining the relationship between users and session IDs is your backend's job. A mapping bug produces cross-user exposure inside a perfectly isolated session. Check which runtime mode you are on as well: the serverless mode discards session state when the session ends, while Runtime Instances retains the session's volumes across stop and resume.

How long should a first pass take, and what is the minimum useful output?

For one well-understood workload with a handful of tools, a focused session with the right four people plus a day of writing up is realistic. What makes it take longer is not the analysis; it is discovering mid-session that nobody present can say what the current tool definitions are, at which point the rest is guesswork and you should stop and go find out. If time is genuinely short, the minimum worth producing is the diagram with boundaries drawn, the asset list with owners, and a residual risk sentence for the three highest-impact threats. That is not enough for an audit and it is far better than nothing.

Do I need to re-run the whole exercise when the model version changes?

No. Re-check the parts that depended on model behavior: guardrail tuning, output schema assumptions, and any latency budget that a human review step depends on. Log the check as an update to the existing model rather than producing a new document, so the history stays in one place.

18. Summary

The process is not the hard part. The four questions and the four steps in SEC01-BP07 are settled, AWS publishes a threat grammar and a tool for recording the output, and none of that needs replacing for generative AI. What needs replacing is the content of the first two questions.

Enumerate the assets that are specific to this class of system. System instructions, conversation history, embeddings and the vector store, ingested documents, tool definitions, agent memory, the right to invoke, and the audit records. For each, say what confidentiality, integrity, and availability actually mean, because the words differ per asset and the differences are where the threats are. A model in which every impacted asset is "the model" has not been decomposed.

Draw the trust boundaries where they really are. Five inputs sit outside: user input, ingested documents, tool return values, messages from other agents, and the model's own output. The last is the invariant that keeps everything else honest, and it converts a class of subtle design errors into one reviewable question you can ask about every edge.

Enumerate against boundary-crossing edges, not components. Component-by-component review concludes that everything is individually fine, which is usually true and rarely relevant, because the failures in these systems are compositions.

Read each STRIDE category in generative AI terms and pair every mitigation with a residual risk. Spoofing includes tool identity and the conflation of agent identity with user identity. Tampering is most valuable to an attacker against the corpus, because a modified document is read as instructions. Repudiation turns on whether one identifier links the request, the context, the tool calls, and the result. Information disclosure is usually an entitlement problem at retrieval time wearing a search costume. Denial of service includes the agent denying service to itself. Elevation of privilege happens mostly through composition, using privileges granted at design time.

Indirect input is the part that stays open. Detection lowers the rate; isolation lowers the consequence; neither prevents a model from being influenced by what it reads. Write the residual as a sentence about what an attacker could still achieve, and have someone accept it by name.

Then keep it current. A tool addition is the highest-frequency change and the one that most directly moves the blast radius, so it gets a checklist and a trigger rather than a good intention. Move every assumption you can express as a policy condition into a policy, and spend the review time on the ones you cannot.

19. References

References:
Tech Blog with curated related content

Written by Hidekazu Konishi