Next.js Search Index Sync: Events, Rebuilds & Permission Safety

Treat the search index as a rebuildable projection

A search engine makes application data easy to discover, but it should not become an accidental second source of truth. Define which database owns each record, which fields enter the index, and how updates, deletions, and permission changes reach that derived representation.

Set a freshness objective by operation. A delayed description update may be acceptable while a delayed access revocation is not. Model those consequences before choosing batch schedules or queue capacity.

Publish committed changes through a durable boundary

Writing the database and then calling the search provider creates a failure gap: the record can commit while indexing fails. Record an outbox event in the business transaction or use a suitable change-data capture stream with durable offsets.

Carry a stable event ID, tenant, entity type, entity ID, source revision, operation, and schema version. Choose whether events contain an approved snapshot or instruct the worker to reread current state. A reread can collapse redundant updates, but the worker must index the revision it actually fetched.

Our transactional outbox guide explains reliable delivery from committed records.

Prevent old events from overwriting newer documents

Assume duplicate and reordered delivery. A stable document ID avoids duplicate documents but does not prevent an old payload from replacing a newer one. Use source revision checks supported by the destination or serialize work per entity with durable revision tracking.

document identity: tenant + entity type + entity ID
projection metadata: source revision + schema version

apply only when incoming revision is newer;
acknowledge duplicates without changing content;
retry transient failures within a bounded policy.

The revision comparison and destination write need a strategy that remains correct under concurrent workers. Checking a local version and then issuing an unconditional remote update can still race. Confirm the guarantees of the selected search engine and worker design.

Keep deletion evidence long enough to reject stale updates

A delete event must remove the searchable document and its derived content. Preserve a tombstone or equivalent revision evidence so a delayed update cannot recreate it. Retain that evidence for the supported replay and rebuild window.

Propagate archive, publication, membership, and visibility changes when they affect discovery. Verify deletion from every active index generation, cache, and export path. The privacy workflow guide covers coordinating deletion across derived systems.

Authorize search results before exposing private content

Resolve tenant and user context on the server and inject mandatory scope filters. Do not accept an arbitrary tenant filter from the browser as authorization. Restrict provider credentials so clients cannot query private indexes directly beyond their intended scope.

Indexed access metadata can become stale. For sensitive results, recheck authoritative authorization before returning snippets or fields, and define how that affects counts, facets, and pagination. A filtered list can still leak information through totals or suggestions if those were calculated over unauthorized records.

Version document shaping and minimize indexed data

Keep a deterministic projection function that validates fields, normalizes agreed values, and excludes secrets or unnecessary personal content. Store stable identifiers and source revisions. Define locale, date, numeric, and missing-field behavior consistently.

Changes to analyzers, field types, permissions, or document shape may require a new index generation. Treat ranking changes separately from correctness checks so a relevance improvement does not conceal missing records or broken access filters.

Rebuild beside the live index with a catch-up plan

Create a new generation and capture a durable change-stream position. Backfill from a consistent snapshot or use a version-aware scan designed to converge while writes continue. Retain and replay changes from the captured position so modifications during the scan are not lost.

Keep backfill and live updates subject to the same revision and deletion rules. Verify expected coverage, representative queries, tenant isolation, permission changes, and lag. Cut over through a controlled routing or alias change only after catch-up meets the stated target.

Retain the previous generation briefly for rollback while ensuring it still receives required changes or is caught up before reuse. Rolling back to an abandoned stale index can reintroduce deleted or inaccessible content.

Measure freshness and reconcile documents against the source

Track oldest pending event age, commit-to-search latency, retries, dead letters, source revision gaps, delete latency, and failed projection counts. Monitor by tenant and entity type so one noisy stream cannot hide another stalled pipeline.

Run bounded reconciliation that compares source IDs and revisions with indexed documents, repairs missing or stale projections, and removes verified orphans. Counts alone cannot prove equality. Use resumable checkpoints and limit provider concurrency to protect interactive search.

Our background-work guide covers the execution controls for reconciliation jobs.

Test event histories that ordinary CRUD checks miss

Deliver updates in reverse order, replay duplicates, delete then replay an older update, revoke membership during pipeline delay, and change a record while backfill scans it. Crash workers after destination success but before acknowledgment.

Verify counts and snippets never cross tenant boundaries, and test facet leakage independently of result filtering. Exercise rebuild cutover, rollback, invalid schemas, and provider outages. Keep a synthetic record whose expected revision can measure end-to-end freshness continuously.

Next.js search synchronization checklist