Amazon Bedrock Knowledge Bases Retrieval Quality Engineering - Chunking Strategies, Hybrid Search, Metadata Filtering, Reranking, and Query Decomposition

First Published: 2026-07-13
Last Updated: 2026-07-13

When a Retrieval-Augmented Generation (RAG) application answers badly, the instinct is to reach for a bigger generation model or a longer prompt. Most of the time, that is the wrong lever. In a grounded RAG system the model can only be as good as the passages it is handed, and those passages are chosen by the retrieval component long before the language model sees a single token. If the right document never enters the context window, no amount of prompting will recover it; if the context is full of near-duplicates and off-topic noise, a stronger model will simply summarize the noise more fluently.

This article is a working reference for engineering the retrieval component of Amazon Bedrock Knowledge Bases. It treats retrieval as a set of tunable knobs — chunking strategy, hybrid search, metadata filtering, reranking, and query decomposition — and shows, for each, what it does internally, which API parameters control it, what the defaults and limits are, and how to tell whether it actually helped. The organizing idea is diagnostic: start from the symptom you observe ("exact terms are not matched", "the answer is cut off mid-thought", "results are noisy"), and map it to the specific setting that addresses the root cause.

The whole-system view of RAG on Bedrock — ingestion, generation, guardrails, and evaluation as one architecture — is covered in Amazon Bedrock RAG Architecture Guide. Self-correcting, agent-driven retrieval loops (retrieval grading, correction, re-query) are covered in Agentic RAG Architecture on Amazon Bedrock. This article deliberately stays one layer below those: the quality of the single retrieval call itself.

1. Introduction: Retrieval Is Where RAG Quality Is Decided

RAG has two halves. The generation half turns retrieved passages into an answer. The retrieval half decides which passages exist to be turned into an answer. The second half is where recall and precision are won or lost, and it is almost entirely a matter of configuration rather than model choice.

The scope of this article is narrow on purpose. It covers the levers that change what comes back from a Retrieve or RetrieveAndGenerate call, and how to reason about them from symptom to setting. It does not re-derive the end-to-end architecture, the agentic self-correction loop, graph-based retrieval, or vector store selection — those are each their own topic and are delegated to dedicated articles:

Whole RAG architecture (ingestion to generation to evaluation): Amazon Bedrock RAG Architecture Guide
Self-correcting, agent-driven retrieval: Agentic RAG Architecture on Amazon Bedrock
Graph-augmented retrieval: GraphRAG Architecture on AWS with Amazon Neptune and Amazon Bedrock
Choosing a vector store and understanding index internals: Amazon S3 Vectors Design Decision Guide and Vector Database Selection on AWS
Terminology: Amazon Bedrock Glossary

A note on scope: two Knowledge Bases products now coexist

Before going further, one scoping point that changes which knobs are available to you. As of 2026-06-17, AWS offers two distinct Knowledge Bases products on Amazon Bedrock:

Customer-managed Knowledge Bases — the original product (generally available since 2023). You choose the vector store, the embedding model, the chunking strategy, and you call the retrieval APIs directly. This is the product that exposes every tuning knob discussed in this article.
Amazon Bedrock Managed Knowledge Base — a newer, more fully managed product (generally available since 2026-06-17) in which AWS manages the vector store, embeddings, and reranker for you. It supports a smaller retrieval-tuning surface: built-in (default) or fixed-size chunking only, and it is queried with Retrieve and the newer AgenticRetrieveStream API rather than RetrieveAndGenerate.

This article covers Customer-managed Knowledge Bases, because retrieval-quality engineering is only possible where the knobs are exposed. Where a technique does not apply to the managed variant, that is called out inline. If your priority is minimal operational overhead rather than fine-grained control, evaluate the managed product first (see Introducing Amazon Bedrock Managed Knowledge Base); the tuning techniques below assume you have deliberately chosen the customer-managed path.

2. How Retrieval Works in Knowledge Bases

To tune retrieval you need a mental model of the pipeline and of where each knob acts. There are two pipelines: an offline ingestion pipeline that builds the index, and an online query pipeline that reads it.

Amazon Bedrock Knowledge Bases retrieval pipeline and where each tuning lever acts

Ingestion (offline, runs on sync). A data source — most commonly an Amazon S3 prefix — is crawled. Each document is parsed into text (with optional foundation-model parsing for complex layouts). The text is chunked into passages according to the chunking strategy configured on the data source. Each chunk is turned into a vector by the embedding model, and the vector plus the chunk text and its metadata are written into the vector store (the index). This entire pipeline runs during an ingestion job (a "sync"), not at query time.

Query (online, runs per request). A user query arrives at Retrieve or RetrieveAndGenerate. The query is embedded by the same embedding model used at ingestion. The vector store is searched — semantically, or with a hybrid of semantic and keyword search. Metadata filters, if present, constrain the search to a subset of the index. The candidate passages may then be reranked by a dedicated reranking model. For RetrieveAndGenerate, the passages are finally handed to a generation model to produce a grounded answer with citations. Optionally, before any of this, a complex query can be decomposed into sub-queries.

Two of these stages are set-once decisions worth calling out. Parsing defaults to a built-in text extractor, but for documents whose meaning lives in tables, forms, or figures you can enable foundation-model parsing so that layout survives into the chunk text — a chunk that keeps its table intact retrieves far better than one flattened into a run-on line. The embedding model is chosen at knowledge base creation and is used identically at ingestion and query time, which is exactly why it cannot be swapped without re-indexing: the stored vectors and the query vector must come from the same model. A current model such as Amazon Titan Text Embeddings V2 also lets you trade index size against fidelity by choosing 256, 512, or 1024 output dimensions.

The five tuning levers map onto these stages cleanly:

* You can sort the table by clicking on the column name.

Lever	Stage it acts on	Set at
Chunking strategy	Ingestion (chunk)	Data source creation (immutable afterward)
Embedding model	Ingestion (embed) and query (embed)	Knowledge base creation
Hybrid search	Query (search)	Per-request (`overrideSearchType`)
Metadata filtering	Query (search)	Per-request (`filter`) plus ingestion (sidecar metadata)
Reranking	Query (post-search)	Per-request (`rerankingConfiguration`)
Query decomposition	Query (pre-search)	Per-request (`orchestrationConfiguration`, `RetrieveAndGenerate` only)

Two consequences fall out of this table immediately, and both are common sources of confusion. First, chunking and the embedding model are baked into the index at ingestion time; changing them requires re-building the index. Second, everything else — search type, filters, reranking, decomposition — is a per-request decision you can change without re-ingesting anything. The cheap knobs are the query-time ones. Reach for them first.

3. Chunking Strategies

Chunking decides the unit of retrieval. Too large, and each hit drags in irrelevant text that dilutes the signal and wastes context budget; too small, and the passage that matches loses the surrounding context needed to answer. Bedrock Knowledge Bases supports several strategies, configured through the chunkingConfiguration object on the data source.

The documented values of chunkingStrategy are FIXED_SIZE, HIERARCHICAL, SEMANTIC, and NONE; custom logic is layered on with a Lambda transformation. If you omit chunkingConfiguration entirely, the default (standard) behavior applies: chunks of approximately 300 tokens that respect sentence boundaries.

As a rule of thumb: use fixed-size for uniform prose, semantic when a document interleaves unrelated topics, hierarchical for long structured documents where a precise match still needs its surrounding section, custom when the document has a structure no native strategy respects (source code, deeply nested records), and no chunking only when each file is already a single retrievable unit. The subsections below cover each in turn.

3.1 Fixed-size chunking

Fixed-size chunking splits text into approximately equal chunks, with a configurable overlap so that a sentence spanning a boundary is not lost. Both parameters are required when you select this strategy.

{
  "chunkingConfiguration": {
    "chunkingStrategy": "FIXED_SIZE",
    "fixedSizeChunkingConfiguration": {
      "maxTokens": 300,
      "overlapPercentage": 20
    }
  }
}

maxTokens is the target chunk size in tokens (bounded by the embedding model's context length); overlapPercentage is the percentage of tokens shared between consecutive chunks. Fixed-size is the right default for homogeneous prose where topic shifts are gradual. Its weakness is that it cuts on token count, not on meaning, so it can split a table, a list, or a tightly-coupled argument down the middle.

3.2 Semantic chunking

Semantic chunking uses embeddings to find natural breakpoints: it groups adjacent sentences by similarity and cuts where the similarity drops, so a chunk tends to hold one coherent idea.

{
  "chunkingConfiguration": {
    "chunkingStrategy": "SEMANTIC",
    "semanticChunkingConfiguration": {
      "maxTokens": 300,
      "bufferSize": 1,
      "breakpointPercentileThreshold": 95
    }
  }
}

maxTokens caps the chunk size; bufferSize controls how many surrounding sentences are considered when computing the semantic boundary (a buffer of 1 compares each sentence with its immediate neighbors); breakpointPercentileThreshold is the percentile of the sentence-distance distribution above which a boundary is placed — a higher value produces fewer, larger chunks. AWS's published guidance recommends starting near maxTokens 300, bufferSize 1, and breakpointPercentileThreshold 95. Note that semantic chunking runs an embedding pass during ingestion to compute the boundaries, so it adds foundation-model inference work at sync time relative to fixed-size chunking. Reach for it when documents interleave distinct topics that fixed-size chunking would blend.

3.3 Hierarchical (parent-child) chunking

Hierarchical chunking maintains two layers of chunks: small child chunks that are precise enough to match a specific query, and larger parent chunks that carry enough surrounding context to answer it. At search time the child chunks are matched, but the corresponding parent chunk is what gets returned to the generation model. You get the precision of small chunks and the context of large ones at the same time.

Hierarchical parent-child chunk structure and retrieval-time parent expansion

{
  "chunkingConfiguration": {
    "chunkingStrategy": "HIERARCHICAL",
    "hierarchicalChunkingConfiguration": {
      "levelConfigurations": [
        { "maxTokens": 1500 },
        { "maxTokens": 300 }
      ],
      "overlapTokens": 60
    }
  }
}

The first entry in levelConfigurations is the parent layer, the second is the child layer. Unlike fixed-size chunking, the overlap here is an absolute token count (overlapTokens), not a percentage. Hierarchical chunking is the strongest default for long, structured documents — technical manuals, contracts, regulatory filings — where a query matches a specific clause but the answer needs the clause in its section context. One caveat when the vector store is Amazon S3 Vectors: parent-child relationships are stored as non-filterable metadata, and very large combined parent-plus-child token counts can bump into that store's per-vector metadata size limit, so keep parent sizes moderate on that store.

3.4 No chunking and custom chunking

NONE treats each file as a single chunk. Use it only when your files are already chunk-sized (for example, one FAQ entry per file), because it forgoes page-level citation granularity and the ability to filter on the built-in per-page metadata.

Custom chunking gives you full control by combining NONE (or a native strategy) with a Lambda transformation that runs at the post-chunking step. You own the chunking logic entirely, or you use it purely to attach chunk-level metadata after a native chunker has run.

{
  "vectorIngestionConfiguration": {
    "chunkingConfiguration": { "chunkingStrategy": "NONE" },
    "customTransformationConfiguration": {
      "intermediateStorage": {
        "s3Location": { "uri": "s3://my-kb-bucket/intermediate/" }
      },
      "transformations": [
        {
          "stepToApply": "POST_CHUNKING",
          "transformationFunction": {
            "transformationLambdaConfiguration": {
              "lambdaArn": "arn:aws:lambda:us-east-1:<account-id>:function:my-chunker"
            }
          }
        }
      ]
    }
  }
}

Bedrock writes chunks to the intermediateStorage S3 location, invokes your Lambda to transform them, and reads the result back. Reserve this for document types no native strategy handles well — for example, code, where you want to chunk on function and class boundaries.

3.5 The immutability rule you must plan around

The single most important operational fact about chunking: the chunking configuration is fixed once a data source is created. UpdateDataSource will not let you change chunkingConfiguration; you must pass the existing value. There is no in-place re-chunking of already-ingested content. To change the strategy you delete and recreate the data source (or add a new one) and run a full re-sync so every document is re-parsed, re-chunked, re-embedded, and re-indexed under the new strategy.

Plan chunking experiments accordingly: stand up a separate data source (or a separate knowledge base) per candidate strategy, ingest a representative sample, and compare retrieval quality before committing production data. Treat chunking as a design-time decision, not a runtime dial.

Managed Knowledge Base note: the managed variant supports only built-in (default) or fixed-size chunking. Semantic, hierarchical, and custom-Lambda chunking are customer-managed features. If you need any of them, you need a customer-managed knowledge base.

4. Hybrid Search

Semantic (vector) search matches on meaning, which is exactly what you want for paraphrases and conceptual questions — and exactly what fails for identifiers. A part number like A-4471-X, a drug name, a legal citation, an error code, or an internal acronym has almost no useful semantic neighborhood; it either appears verbatim in the source or it does not. Pure vector search will happily return semantically "nearby" passages that do not contain the term at all.

Hybrid search fixes this by running both a semantic (vector) search and a keyword (full-text) search over the same query, then merging the two result sets. The exact-term hits from keyword search rescue the identifiers; the semantic hits keep the conceptual recall. You control it per request with overrideSearchType inside vectorSearchConfiguration:

import boto3

client = boto3.client("bedrock-agent-runtime")

response = client.retrieve(
    knowledgeBaseId="KB1234567890",
    retrievalQuery={"text": "What is the torque spec for bolt A-4471-X?"},
    retrievalConfiguration={
        "vectorSearchConfiguration": {
            "numberOfResults": 20,
            "overrideSearchType": "HYBRID"
        }
    }
)

The documented values are HYBRID and SEMANTIC. There is no literal AUTO value: if you omit overrideSearchType, Bedrock decides which strategy suits your vector store configuration. That automatic choice is subject to a hard constraint — hybrid search runs only on vector stores that support it and contain a filterable text field. As currently documented, that means Amazon OpenSearch Serverless, Amazon Aurora PostgreSQL (referred to as "Amazon RDS" in the current documentation), and MongoDB Atlas. On any other store, or on a supporting store without a filterable text field, the query silently falls back to semantic-only search — even if you explicitly ask for HYBRID.

* You can sort the table by clicking on the column name.

Vector store	Hybrid search
Amazon OpenSearch Serverless	Supported
Amazon Aurora PostgreSQL (pgvector)	Supported
MongoDB Atlas	Supported
Amazon S3 Vectors	Semantic only
Pinecone	Semantic only
Redis Enterprise Cloud	Semantic only
Amazon Neptune Analytics (GraphRAG)	Semantic only

This "silent fallback" is the diagnostic trap to watch for: if you turned on hybrid search and saw no change in identifier recall, confirm your vector store actually supports it and has a filterable text field, rather than assuming the setting had no effect. Hybrid search is the first thing to reach for in domains dense with proper nouns and codes — medicine, law, engineering, finance — because that is precisely where pure semantic search underperforms. Because the two result sets are merged rather than tried in sequence, hybrid search can also lift precision: a passage that scores well on both the vector and the keyword signal is a stronger candidate than one that scores well on only one of them.

5. Metadata Filtering

Reranking and better chunking improve which of the retrieved passages is best. Metadata filtering changes which passages are eligible to be retrieved at all. It restricts the search to a subset of the index — this year's documents, this tenant's documents, this department's documents — before similarity is ever computed. That both raises precision (irrelevant partitions can no longer surface) and, on most stores, lowers latency (a smaller space is searched). Of the query-time levers it is often the highest-leverage first move, because narrowing the candidate universe helps every downstream stage at once — the reranker has fewer distractors to sort through, and generation sees a cleaner context.

5.1 Attaching metadata at ingestion

For Amazon S3 data sources, metadata is attached with a sidecar JSON file that sits next to the source document. For a document at documents/legal/contract.pdf, the metadata file is documents/legal/contract.pdf.metadata.json:

{
  "metadataAttributes": {
    "department": "legal",
    "year": 2023,
    "language": "en",
    "classification": "internal"
  }
}

The attribute values become filterable fields on every chunk derived from that document. Design this schema deliberately — the filters you will want at query time have to exist as attributes at ingestion time. Common, high-value dimensions are time (year, effective_date), organizational (department, product, region), tenant (tenant_id), language, and sensitivity (classification). (CSV data sources also support a separate record-based metadata mechanism that maps some columns to content and others to metadata; that is distinct from the sidecar format described here.)

5.2 Filter operators

At query time, the filter field takes a RetrievalFilter, a union whose members are the comparison and logical operators. The full set is:

* You can sort the table by clicking on the column name.

Category	Operators
Equality	`equals`, `notEquals`
Numeric range	`greaterThan`, `greaterThanOrEquals`, `lessThan`, `lessThanOrEquals`
Set membership	`in`, `notIn`
String / list	`stringContains`, `listContains`, `startsWith`
Logical	`andAll`, `orAll`

A filter combining several conditions nests the comparison operators inside andAll or orAll:

{
  "andAll": [
    { "equals": { "key": "department", "value": "legal" } },
    { "greaterThanOrEquals": { "key": "year", "value": 2023 } },
    { "in": { "key": "language", "value": ["en", "ja"] } }
  ]
}

Attached to a Retrieve call, this restricts results to legal-department documents from 2023 onward in English or Japanese, before ranking. Per the console and managed knowledge base documentation, logical operators accept up to five conditions each and can be nested one extra level; the RetrievalFilter API reference itself documents only the two-condition minimum, so validate deeper filter trees against current limits (see Section 10.3).

Operator support is not uniform across stores, and this is a frequent source of "my filter is ignored" bugs. startsWith is supported only on Amazon OpenSearch Serverless. in, notIn, and stringContains are best supported on OpenSearch Serverless (with Neptune Analytics GraphRAG supporting the string form of stringContains but not the list form). Fully-managed / quick-create knowledge bases, and knowledge bases backed by an S3 vector index, do not support startsWith or stringContains — use equals, greaterThan, lessThan, in, or notIn there. Always verify operator support against your specific store before designing a filter scheme around an operator.

5.3 Implicit (model-inferred) metadata filtering

Explicit filters assume your application already knows which partition to search. Sometimes the constraint is buried in the natural-language query itself — "what did the 2024 policy say about remote work?" contains a year filter that a human would apply automatically. Implicit metadata filtering lets Bedrock do the same: given a metadata schema, it uses an Anthropic Claude model to infer a retrieval filter from the query and apply it. It is configured on the Retrieve request through implicitFilterConfiguration:

{
  "vectorSearchConfiguration": {
    "numberOfResults": 20,
    "implicitFilterConfiguration": {
      "metadataAttributes": [
        { "key": "year", "type": "NUMBER", "description": "Publication year of the document" },
        { "key": "department", "type": "STRING", "description": "Owning department, e.g. legal, hr, finance" }
      ],
      "modelArn": "<claude-model-arn>"
    }
  }
}

Each schema entry gives the model the attribute name, its type (STRING, NUMBER, BOOLEAN, or STRING_LIST), and a natural-language description it uses to decide when and how to apply the filter. The clearer the description, the more reliable the inference. Implicit filtering trades a small amount of added latency (an extra model call to build the filter) for the ability to apply constraints the user expressed only in prose. It is documented for the Retrieve API; confirm current support before relying on it in a RetrieveAndGenerate flow. Metadata filtering — implicit or explicit — is also the mechanism behind tenant isolation in multi-tenant knowledge bases; the isolation patterns themselves are covered in Amazon Bedrock Knowledge Bases Ingestion and Isolation Patterns.

6. Reranking

First-stage vector (or hybrid) search is optimized for speed over a large index; it uses an approximate nearest-neighbor structure and a single embedding per passage. That is good enough to pull a broad candidate set but imperfect at ordering it, because a single embedding compresses a whole passage into one point and cannot model the fine interaction between this query and this passage. Reranking adds a second, slower, more accurate stage: a cross-encoder model that looks at the query and each candidate passage together and produces a fresh relevance score, then re-orders the set and keeps the top few.

The two-stage flow is: retrieve a generous candidate set (bounded by numberOfResults), then rerank it down to a smaller, better-ordered set (bounded by numberOfRerankedResults). Retrieve wide, rerank narrow.

6.1 Reranking models and configuration

Two reranking models are available in Amazon Bedrock, with different Region coverage that you must account for:

* You can sort the table by clicking on the column name.

Model	Model ID	Regions
Amazon Rerank 1.0	`amazon.rerank-v1:0`	ap-northeast-1, ca-central-1, eu-central-1, us-west-2 (not us-east-1)
Cohere Rerank 3.5	`cohere.rerank-v3-5:0`	ap-northeast-1, ca-central-1, eu-central-1, us-east-1, us-west-2

Both require an explicit Model Access grant, the same as any Bedrock foundation model. To integrate reranking into a knowledge base retrieval call, add rerankingConfiguration to vectorSearchConfiguration:

response = client.retrieve(
    knowledgeBaseId="KB1234567890",
    retrievalQuery={"text": "How do I rotate a database credential without downtime?"},
    retrievalConfiguration={
        "vectorSearchConfiguration": {
            "numberOfResults": 25,
            "rerankingConfiguration": {
                "type": "BEDROCK_RERANKING_MODEL",
                "bedrockRerankingConfiguration": {
                    "modelConfiguration": {
                        "modelArn": "arn:aws:bedrock:us-west-2::foundation-model/amazon.rerank-v1:0"
                    },
                    "numberOfRerankedResults": 5
                }
            }
        }
    }
)

The only current value of type is BEDROCK_RERANKING_MODEL. numberOfRerankedResults is the count returned after reranking, valid from 1 to 100. An optional metadataConfiguration controls which metadata fields are passed to the reranking model, which matters when a passage's relevance depends partly on its attributes and not only its text. The knowledge base first fetches numberOfResults candidates, then the reranker re-scores them and returns the best numberOfRerankedResults.

Two sizing decisions matter here. The first is how wide to retrieve: a larger numberOfResults gives the reranker more chances to surface a buried passage, but every extra candidate is another (query, passage) pair the cross-encoder must score, so latency grows with the candidate count — retrieve wide enough to include the answer, not wider. The second is what the reranker sees: by default it scores passage text, but metadataConfiguration can fold selected metadata fields into its input when relevance depends on attributes such as recency or source authority rather than wording alone.

The same reranking capability is also available as a standalone Rerank API (in the bedrock-agent-runtime namespace) that reranks an arbitrary list of documents you supply — useful when you are merging results from several retrievers, or reranking outside a knowledge base entirely.

6.2 When reranking earns its cost

Reranking is not free: it adds a second model invocation between search and generation, so it adds latency, and it needs bedrock:Rerank and bedrock:InvokeModel permissions plus model access. It earns that cost when your first-stage results are relevant but poorly ordered — the right passage is in the top 25 but sits at rank 12, so with a small numberOfResults for generation it would be dropped. Retrieving 25 and reranking to 5 recovers it. If, instead, the right passage is not in the candidate set at all, reranking cannot help — that is a recall problem to fix with chunking, hybrid search, or filtering, not a re-ordering problem. Diagnose which you have before adding a reranker: inspect a wide Retrieve (say numberOfResults 50) and check whether the answer-bearing passage is present but low, or absent entirely.

7. Query Decomposition and Reformulation

Some queries fail retrieval not because of chunking or search type but because they pack several distinct information needs into one string. "Compare our 2023 and 2024 parental-leave policies and note any changes to eligibility" is really three retrievals: the 2023 policy, the 2024 policy, and the eligibility rules. A single embedding of the whole sentence is a blurry average of all three intents and matches none of them cleanly. In practice the service rewrites the single request into a small set of focused retrievals — roughly one per distinct information need it detects — runs them, and passes the pooled, de-duplicated passages to generation as if they had come from one search.

Query decomposition breaks such a query into simpler sub-queries, retrieves for each independently, and pools the results before generation. It is enabled through orchestrationConfiguration on a RetrieveAndGenerate call:

{
  "retrieveAndGenerateConfiguration": {
    "type": "KNOWLEDGE_BASE",
    "knowledgeBaseConfiguration": {
      "knowledgeBaseId": "KB1234567890",
      "modelArn": "<claude-generation-model-arn>",
      "orchestrationConfiguration": {
        "queryTransformationConfiguration": {
          "type": "QUERY_DECOMPOSITION"
        }
      }
    }
  }
}

QUERY_DECOMPOSITION is the current value of the transformation type. Decomposition is off by default — you must enable it explicitly. It is available on the RetrieveAndGenerate orchestration flow (which has an orchestration step to decompose against), not on the plain Retrieve API. Turn it on for multi-part and comparative questions; leave it off for simple lookups, where the extra sub-query round trips add latency for no benefit.

Query decomposition is a bounded, single-shot transformation: split, retrieve, generate. It is not the same as an agentic loop that grades what it retrieved and decides to re-query. That iterative pattern — retrieval grading and correction — belongs to agent-driven RAG and is covered in Agentic RAG Architecture on Amazon Bedrock.

8. A Retrieval Tuning Workflow: From Symptom to Setting

The five levers are most useful when you approach them by symptom rather than by feature. Below is the diagnostic flow: observe what is wrong, identify the root cause, apply the matching setting, and measure before moving on.

Retrieval quality diagnostic flow from symptom to setting

* You can sort the table by clicking on the column name.

Symptom	Likely root cause	First setting to try
Exact terms, codes, or names are not matched	Pure semantic search has no signal for identifiers	Enable hybrid search (`overrideSearchType: HYBRID`) on a supporting store
Answers are cut off or miss surrounding context	Chunks are too small or split mid-idea	Switch to hierarchical chunking, or increase chunk size / overlap (requires re-ingestion)
Results are on-topic but noisy or duplicated	Candidate set is broad and poorly ordered	Add reranking (retrieve wide, rerank narrow)
Results include the wrong partition (year, tenant, department)	Search space is not constrained	Add a metadata filter
Search feels broad and slow	Whole index is searched every time	Add a metadata filter to shrink the search space
Multi-part or comparative questions answered partially	One embedding averages several intents	Enable query decomposition (`RetrieveAndGenerate`)

8.1 Change the cheap knobs first

Order your changes by cost to apply, not by expected impact. Query-time settings — search type, filters, reranking, decomposition — cost nothing but a config change and can be tried and reverted in minutes. Ingestion-time settings — chunking strategy and embedding model — require a full re-index, so they are the last resort, not the first. A rational tuning sequence for a struggling retriever is:

Widen numberOfResults and inspect the raw candidates to see whether the answer-bearing passage is present at all. This tells you whether you have a recall problem or a ranking problem.
If identifiers are missing, enable hybrid search.
If the wrong partitions appear, add a metadata filter.
If the right passage is present but low, add reranking.
If multi-part questions are half-answered, enable query decomposition.
Only if the answer-bearing content is fundamentally split or too coarse, revisit the chunking strategy and re-ingest.

8.2 Measure every change

Every one of these levers can help one query and hurt another, so never change two at once and never trust a single anecdote. Fix a representative test set of real queries with known-good answers, and after each change measure whether the answer-bearing passage is retrieved and where it ranks. Cheap query-time changes especially deserve an A/B check, because a filter that seems obviously correct can exclude a document whose metadata was mislabeled at ingestion. The next section covers what to measure.

9. Measuring Retrieval Quality

Tuning without measurement is guessing. Retrieval quality is measured on two axes. Recall asks whether the passages that contain the answer were retrieved at all — the ceiling on everything downstream, because a passage that is never retrieved can never be reranked or generated from. Precision / relevance asks how much of what was retrieved is actually on-point — the signal-to-noise ratio handed to the generation model.

For a lightweight in-house loop you can build this from the retrieval response directly: run a fixed query set through Retrieve, and for each query check whether the known answer-bearing document appears in the results and at what rank. Tracking hit rate (recall proxy) and mean rank over a set of tens to low-hundreds of queries is enough to tell whether a change helped, and it is cheap enough to run on every configuration change. Concretely, a golden set is a list of (query, expected-document) pairs curated from real user questions and their verified answers; for each configuration you run every query through Retrieve, record the rank at which the expected document appears (or a miss), and summarize the run with hit rate at k and mean reciprocal rank. Because the same set runs against every configuration, the comparison is apples-to-apples, and a regression from a well-meaning change surfaces immediately as a drop in hit rate.

For a managed, repeatable evaluation, Amazon Bedrock Evaluations includes a dedicated RAG evaluation capability (generally available since 2025). Its "Retrieval only" evaluation type evaluates retrieval quality on its own — computing metrics such as context relevance and context coverage using an LLM-as-a-judge — against either a Bedrock knowledge base or a bring-your-own retrieval pipeline. That lets you gate configuration changes on quantitative retrieval metrics rather than spot checks. The full treatment of Bedrock evaluations — automatic metrics, LLM-as-a-judge, RAG evaluation jobs, and wiring quality gates into CI/CD — is a topic of its own, covered in Amazon Bedrock Model Evaluation Practical Guide.

10. Cross-Cutting Concerns

A few concerns cut across every retrieval call regardless of which levers you have turned on.

10.1 Least-privilege IAM

The identity that calls retrieval needs only the retrieval actions, scoped to the specific knowledge base — plus, if reranking is on, the reranking and model-invocation actions. Keep the knowledge base action scoped to its ARN; the model-invocation and rerank actions are scoped to the model ARNs you actually use.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "KnowledgeBaseRetrieve",
      "Effect": "Allow",
      "Action": ["bedrock:Retrieve", "bedrock:RetrieveAndGenerate"],
      "Resource": "arn:aws:bedrock:us-east-1:<account-id>:knowledge-base/KB1234567890"
    },
    {
      "Sid": "InvokeGenerationAndRerankModels",
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:Rerank"],
      "Resource": [
        "arn:aws:bedrock:us-east-1::foundation-model/<generation-model-id>",
        "arn:aws:bedrock:us-west-2::foundation-model/amazon.rerank-v1:0"
      ]
    }
  ]
}

If you use RetrieveAndGenerate, the generation model needs bedrock:InvokeModel; a plain Retrieve that does no generation and no reranking needs only the knowledge base action.

10.2 Latency budget

Every lever you add to a retrieval call spends part of a latency budget. Reranking adds a model call between search and generation. Implicit metadata filtering adds a model call before search. Query decomposition adds one retrieval round trip per sub-query. Hybrid search runs two searches and merges them. None of these is expensive on its own, but stack all of them on a low-latency path and the sum is noticeable. Turn each on because a symptom justifies it, measure its latency cost, and keep the interactive path lean.

10.3 Quotas and limits worth knowing

A handful of documented limits shape retrieval design. numberOfResults accepts 1 to 100, and the documented default is 5 results; for the two-stage retrieve-then-rerank pattern, set it explicitly to a wider value instead of relying on the default. Logical filter operators (andAll, orAll) each require at least two conditions; the API reference does not document an upper bound or nesting depth, so validate complex filter trees against the current service limits before relying on them. When Amazon S3 Vectors is the store, Bedrock allows up to 1 KB of custom metadata and 35 metadata keys per vector — tighter than S3 Vectors' standalone limits — which interacts with both metadata-filter design and hierarchical chunking (whose parent-child links consume non-filterable metadata). Ingestion is separately bounded: concurrent ingestion jobs are limited (documented as a small number per account, per knowledge base, and per data source), and direct document ingestion via the API is bounded per call. Confirm the exact current values in the Amazon Bedrock service quotas reference and the Service Quotas console before you build capacity assumptions on them; treat the numbers here as shape, not as SLA. Re-ingestion is itself an operational cost — because chunking and embedding changes force a full re-sync, budget for the ingestion window when you plan a strategy change. Ingestion strategy, sync modes, and tenant isolation are covered in depth in Amazon Bedrock Knowledge Bases Ingestion and Isolation Patterns.

10.4 Multi-turn sessions and multimodal queries

RetrieveAndGenerate supports multi-turn conversations through a sessionId. You do not set it yourself — Bedrock generates it on the first call and returns it, and you pass that value back on later turns so the service carries conversational context forward. Retrieval can also be multimodal: retrievalQuery accepts an image as well as text on knowledge bases configured for it, so a screenshot or diagram can drive retrieval the same way a text query does. Both are per-request concerns that layer on top of the tuning levers above without changing how any of them behave.

10.5 A note on cost and on safety

Two honest caveats. First, several of these levers change the cost profile of a request — semantic chunking adds ingestion-time inference, reranking and implicit filtering add per-request model calls, query decomposition multiplies retrievals — but this article deliberately states cost only qualitatively; consult the official Amazon Bedrock pricing page for figures. Second, a metadata filter that scopes results to a tenant or a classification is a retrieval-quality control, not by itself a complete security boundary; it must be paired with correct metadata labeling at ingestion, least-privilege IAM, and defense in depth. A filter that returns clean results is not proof that an incorrectly-labeled document cannot leak — verify the labels, not just the outputs.

11. Frequently Asked Questions

Can I change the chunking strategy on an existing data source?
No. The chunking configuration is immutable after the data source is created. Create a new data source (or knowledge base) with the new strategy and run a full re-sync, then cut over.

Why did enabling hybrid search change nothing?
Most likely your vector store does not support hybrid search, or the supporting store lacks a filterable text field, so the query silently fell back to semantic-only. Hybrid search is currently supported on Amazon OpenSearch Serverless, Amazon Aurora PostgreSQL, and MongoDB Atlas with a filterable text field.

Reranking or better chunking — which should I try first?
Diagnose the failure. If the answer-bearing passage is present but low in a wide Retrieve, that is a ranking problem: add reranking. If it is absent from a wide Retrieve, that is a recall problem: fix chunking, hybrid search, or filtering. Reranking cannot recover a passage that was never retrieved.

Does query decomposition work with the Retrieve API?
No. Query decomposition is configured through orchestrationConfiguration on RetrieveAndGenerate, which has an orchestration step; the plain Retrieve API does not.

What is the difference between explicit and implicit metadata filtering?
Explicit filtering means your application supplies the filter object. Implicit filtering means Bedrock uses a Claude model to infer a filter from the natural-language query against a metadata schema you declare. Use explicit when the application already knows the constraint; use implicit when the constraint is expressed only in the user's prose.

Do these techniques apply to the new Amazon Bedrock Managed Knowledge Base?
Partially. The managed variant (generally available since 2026-06-17) supports built-in or fixed-size chunking only and is queried with Retrieve / AgenticRetrieveStream rather than RetrieveAndGenerate. Hierarchical, semantic, and custom chunking, and the RetrieveAndGenerate orchestration features such as query decomposition, are customer-managed capabilities.

Which embedding model should I use?
Use a current-generation embedding model such as Amazon Titan Text Embeddings V2, which supports configurable output dimensions (256, 512, or 1024) and float or binary embeddings, or a Cohere Embed v3 model (English or Multilingual — the Cohere generation currently listed as supported for knowledge bases). Avoid the first-generation Titan model for new builds. The embedding model is fixed at knowledge base creation, so choosing it well matters as much as any query-time knob.

12. Summary

Retrieval quality in Amazon Bedrock Knowledge Bases is engineered, not wished for, and almost all of the engineering happens in configuration rather than model choice. The retrieval pipeline has two halves — an offline ingestion pipeline that builds the index and an online query pipeline that reads it — and five levers act at specific points on it. Chunking (fixed-size, semantic, hierarchical, custom, or none) sets the unit of retrieval at ingestion time and is immutable afterward, so it is a design-time decision that costs a re-index to change. The four query-time levers are cheap to try and revert: hybrid search rescues exact-term recall on supporting stores; metadata filtering constrains the search space for precision and speed; reranking re-orders a wide candidate set into a precise short list; and query decomposition splits multi-part questions into clean sub-queries. The disciplined way to use them is diagnostic — read the symptom, apply the cheapest matching knob, measure against a fixed query set, and only reach for a re-index when a query-time change cannot fix a genuine recall gap. For the broader architecture these components live inside, and for the evaluation and ingestion practices that surround them, follow the delegated articles referenced throughout.

13. References

References:
Tech Blog with curated related content

Written by Hidekazu Konishi