pgvector Can Return 4 of Your 10 Matches—Where Did the Rest Go?

Your retrieval query can succeed and still come up short

pgvector documents an awkward piece of arithmetic: when a filter matches 10% of rows and HNSW uses its default hnsw.ef_search of 40, about four rows match on average. Ask for ten results and you may receive fewer, even when enough eligible documents exist. That figure is a documentation example, not a benchmark of your application. The project README explains the mechanism.

For a retrieval augmented generation app, this creates an easy misdiagnosis. Imagine a support assistant searching one customer’s manuals inside a shared document table. Its database call succeeds, its answer arrives, and nobody sees an exception. Yet the assistant may have received a thin slice of the available evidence. Spending more on generation is an expensive response to a retrieval problem.

This is a practical guide for November 7, 2026, rather than a claim that the fix appeared today. Iterative index scans came with pgvector 0.8.0; PostgreSQL’s announcement was posted on November 11, 2024. The useful question now is whether your deployed queries use them, and whether they help your particular filters.

LIMIT 10 is a ceiling, not a delivery promise

The critical distinction is between eligible rows in the table and eligible rows found by an approximate search. An HNSW index helps find nearby vectors without scoring the entire collection. A separate condition, such as WHERE tenant_id = 42, can then discard candidates. A large collection of relevant rows somewhere in the database does not ensure they were among the candidates examined.

Supabase gives a concrete version of this warning: an indexed query filtered by category_id = 123 can return fewer than five rows despite five matching rows existing. Its documentation points builders toward iterative search. This applies to both HNSW and IVFFlat approximate indexes. Supabase’s pgvector guide describes the failure without dressing it up as an AI mystery.

Think of this as two separate checks in your retrieval contract. First, did the query fill its requested result count when enough eligible records existed? Second, did it find the nearest eligible records? Ten returned chunks can pass the first check and fail the second. Counting results is useful instrumentation; it is not a relevance evaluation.

For an application, the distinction matters whenever evidence is narrow: one account, one language, one product version, one date range. A broad demo query tells you little about those combinations. A builder should include the filters users will apply when evaluating retrieval, rather than measuring only an unrestricted search across the whole corpus.

Let the index keep looking, within a budget

Iterative scans allow the database to continue searching after filtering leaves too few results. They still have stopping conditions. PostgreSQL’s 0.8.0 announcement names hnsw.max_scan_tuples and ivfflat.max_probes as configurable thresholds. Turning the feature on does not promise that every query will return its requested count.

Start by checking the installed extension rather than assuming a database provider exposes the feature:

SELECT extversion
FROM pg_extension
WHERE extname = 'vector';

For HNSW, strict_order preserves distance ordering; relaxed_order allows some disorder in exchange for better recall. Neither turns an approximate search into an exhaustive one. The settings address how the search proceeds and emits candidates, which is different from guaranteeing that no nearer eligible vector was missed.

An illustrative application query can scope the change to one transaction. Here, $1 is a bound tenant identifier and $2 is a bound embedding with the same dimensions as the stored vectors:

BEGIN;
SET LOCAL hnsw.iterative_scan = strict_order;

SELECT id, content
FROM documents
WHERE tenant_id = $1
ORDER BY embedding <=> $2::vector
LIMIT 10;

COMMIT;

This assumes an existing documents table and a compatible cosine HNSW index. It is a query pattern, not a complete migration. Run the transaction on one checked-out database connection. PostgreSQL specifies that SET LOCAL expires when the transaction ends; outside a transaction it emits a warning and has no effect. That lifetime makes it useful for applying retrieval settings deliberately. PostgreSQL documents SET here.

More searching has a bill attached

AWS’s production guide lists an HNSW iterative-scan tuple budget of 20,000 by default and a scan memory multiplier of 1 relative to work_mem. These are resource controls, not a latency guarantee. A busy database can experience very different response times from a development instance running the same SQL. AWS’s pgvector production guide discusses both settings.

Increase a budget only when a measured failure justifies it. If a selective filter routinely leaves the result short, record what happens to completeness and response time as you adjust the search. If additional work barely changes the answer set, stop turning knobs and inspect the data and execution plan.

AWS also gives the planner an ordinary B-tree on the tenant column in its baseline schema. That provides an alternative route for finding eligible documents. Its guide recommends omitting HNSW when a bounded exact scan already meets the workload’s needs, and warns that distance operators must match their index operator classes. For example, cosine distance uses <=> with vector_cosine_ops.

That makes the engineering decision less glamorous and more useful. Compare candidate retrieval strategies under your actual traffic and data distribution. Buying a separate vector service, increasing database capacity, or raising scan limits should follow evidence about the bottleneck. None of those decisions can be settled by a screenshot of one fast query.

Read the plan before celebrating the fix

Use EXPLAIN to see the execution strategy PostgreSQL chooses. Add ANALYZE when you want actual execution measurements, remembering that it runs the statement. The distinction matters when investigating a query on a busy system. PostgreSQL’s EXPLAIN guide explains estimates, actual rows, scan nodes and filter conditions.

For the read-only retrieval query, inspect EXPLAIN (ANALYZE, BUFFERS) with representative parameter values. Check whether the database used the approximate index, filtered a smaller set using another access path, or performed a sequential scan. Changing an HNSW setting is not evidence of improvement if the query never used HNSW.

Keep the comparison controlled: the same query vector, filter, data snapshot and requested count. Record the rows returned and execution time alongside the plan. Where useful, retain machine-readable JSON plans so that differences can be reviewed later instead of reconstructed from a terminal screenshot.

The experiment should answer a decision. Does iterative scanning recover eligible results at acceptable latency? Does an exact route already satisfy the request? Does one tenant behave differently from another? A plan is useful because it connects observed behavior to a concrete execution strategy, rather than leaving the team to speculate about embeddings.

Evaluate the evidence your assistant receives

My recommended evaluation starts with a small, fixed set of real retrieval requests. Include common questions, restrictive filters and accounts with different collection sizes. For each, preserve the approximate result IDs and compare them with an exact-search reference under the same eligibility rules. The reference establishes nearest neighbors for the chosen distance metric; editorial judgments about usefulness still need separate evaluation.

For example, if an exact reference has ten results and an approximate run shares eight of their IDs, its recall@10 is 0.8 for that request. This is an illustrative calculation, not a reported pgvector score. Handle ties consistently and specify what your metric does when fewer than ten eligible documents exist.

Then inspect the content. Two retrieved chunks might repeat the same paragraph while omitting the troubleshooting step a user needs. Nearest-neighbor recall can expose search losses, but it does not establish that chunking, embedding choice or document freshness is right. Keep those failures distinguishable so that the team fixes the component responsible.

Finally, log retrieval behavior alongside answer evaluations. A proposed rollout gate could require acceptable latency, adequate nearest-neighbor recall and a clear response when evidence is insufficient. Set those thresholds from the product’s requirements; there is no universal percentage supplied by this article. The useful outcome is an assistant whose evidence pipeline you can explain when an answer goes wrong.

Frequently asked questions

Is this a new pgvector feature?

No. Iterative index scans were introduced in 0.8.0. Check the extension version installed in your database.

Does strict ordering guarantee perfect recall?

No. It preserves distance ordering of returned results; approximate search can still miss eligible neighbors.

Will iterative scans always return ten results?

No. Too few eligible rows or scan limits can leave the result short. Measure returned counts and recall separately.