Fresh enough to trust, without hammering your source systems.
An AI answer built on last month's index is a liability. Data.co Control keeps indexed content current using scheduled sync or change data capture, chosen per source, and runs every change through classification and lineage before it can be retrieved.
Choose freshness per source.
The right mode depends on what the source supports and how quickly changes must be reflected in answers.
| Mode | How it works | Good fit | Consideration |
|---|---|---|---|
| Change data capture | Reads the source's change log and applies inserts, updates, and deletes as they occur | Operational databases where freshness matters | Requires CDC support and appropriate permissions on the source |
| Event-driven | Applies changes from webhooks, change APIs, or message queues | SaaS and document systems with change notifications | Depends on the source's event delivery guarantees |
| Scheduled incremental | Reads items changed since the last run on a schedule | Document stores and apps with modified timestamps | Freshness bounded by the schedule |
| Scheduled full | Re-reads the full source on a schedule and reconciles | Small sources or ones without change tracking | Most load on the source; also catches missed deletes |
Access revocation does not wait for the next sync.
Sync keeps content current. Permissions are a different matter: source ACLs are evaluated at query time, so if a user loses access to a folder or a record, their next AI request reflects it regardless of when the content was last synced.
Deletes are handled as first-class changes. When a source record or document is removed, its index chunks are removed on the next sync, and scheduled full reconciliations catch deletes that a change feed did not report.
Sync policy per source.
source: claims
mode: cdc
cdc:
method: logical_replication # where the source supports it
slot: dataco_claims
reconcile:
mode: scheduled_full
every: 7d
window: "Sun 01:00-05:00" # off-hours for the source
on_change: [classify, index, lineage]Illustrative example · final API surface documented in /docs
What sync does to protect sources and indexes.
- Runs entirely inside your network
- Respects rate limits and maintenance windows
- Classifies new content before it is retrievable
- Propagates deletes, with periodic reconciliation
- Resumes from the last checkpoint after failure
- Records every job for lineage and audit
- Reports lag and failures locally
- Works without internet access
What happens when a record is updated at the source.
- 1
Detect
The change arrives from the source's change log, an event, or the next scheduled scan, and is tied to a sync job ID.
- 2
Classify
The new content is classified before anything else. If it now contains a sensitive identifier, the new label applies from its first retrievable moment.
- 3
Replace
Chunks derived from the old version are replaced in the index, and the catalog's sync state is updated, so retrieval never mixes old and new versions of the same item.
- 4
Link
Lineage records the source version, the job, and the resulting chunks, so later answers can be traced to the version they actually used.
The details that decide whether sync stays healthy.
Change data capture is efficient, but it is not free for the source. Log-based CDC typically holds a position in the database's change log, and if the consumer stops for long enough the source retains log data on its behalf, which uses disk on a production system. Treat CDC lag as an alert, agree with the database team how long a pause can last, and know how to drop and rebuild the capture if it must be abandoned. A periodic full reconciliation then repairs anything the rebuild missed.
For scheduled modes, pick intervals from the question each use case answers, not from a default. A policy library that changes monthly does not need hourly scans; a ticket queue an agent triages throughout the day probably does. Deletes deserve their own attention: change feeds and modified-time scans can miss them, which is why reconciliation exists, and why its frequency should reflect how quickly removed content must disappear from answers.
Finally, make staleness visible. The catalog records the last successful sync for every source, which lets reviewers, users, and policies see when an answer may be built on content that is older than expected.
Questions buyers ask
Which sources support change data capture?
CDC depends on the source system, its version, and its configuration. Where it is not available, event-driven or scheduled incremental sync is used. We confirm the mode for each of your sources during scoping.
How fresh will AI answers be?
As fresh as the chosen mode allows. CDC reflects changes shortly after they occur, while scheduled modes are bounded by their interval. We do not quote a single number because it depends on the source and your configuration.
Will sync put load on production systems?
Sync jobs honor rate limits and maintenance windows you set, and CDC reads change logs rather than re-querying tables. Full reconciliations can be scheduled off-hours.
What happens if a sync job fails?
Jobs resume from their last checkpoint, failures are reported locally, and the catalog shows the last successful sync so stale sources are visible to users and policies.
Are permission changes picked up by sync?
Permissions are checked at query time against the source, so revocations take effect on the next request. Sync also refreshes stored ACL metadata used for retrieval filtering.
How quickly do deleted documents disappear from answers?
With CDC or event-driven sync, deletes are applied as they are reported. Scheduled modes apply them on the next run, and full reconciliation catches deletes a change feed missed. Set reconciliation frequency to match your obligations.
See it against your own use case.
A 30-minute technical walkthrough, policy simulation on your scenarios, and a deployment recommendation for on-prem, private cloud, or hybrid.