Retrieval that checks permissions before the model reads.
Retrieval-augmented generation is how most enterprise AI gets useful. It is also how most enterprise AI leaks. Data.co builds RAG pipelines where access filtering happens before context is assembled, every answer carries citations, and every lookup is logged.
Filtering after generation is too late.
A common RAG pattern retrieves the top results from a shared index, sends them to the model, and hopes a system prompt keeps the answer appropriate. If a restricted document ranks highly, its content is already in the model's context. Some systems try to filter the answer afterward, which cannot reliably remove what the model has paraphrased.
Data.co applies permission and label filters inside the retrieval step. The candidate set is limited to content the requester may see before ranking, so restricted content never reaches the context window. The audit log records which chunks were retrieved for each answer.
Each stage is governed.
Ingest
Connectors pull content with ACLs, labels, and lineage attached to every chunk.
Embed locally
Embedding models run in your environment, so vectors never leave it.
Retrieve with filters
Hybrid keyword and vector search constrained by identity and label at query time.
Generate with citations
Answers reference specific passages, which users can open if they have access.
Evaluate
Test sets of questions with expected sources help catch regressions when models or content change.
Log
Each request records identity, retrieved chunk IDs, model, and decision.
Retrieval as a governed call.
Applications call the retrieval API with the end user's token. Results are already filtered; the application does not need to reimplement access control.
curl https://ai.internal.example.com/v1/retrieve \
-H "Authorization: Bearer $USER_TOKEN" \
-d '{
"collection": "policies",
"query": "What is the travel reimbursement limit for client visits?",
"top_k": 6
}'Illustrative example · final API surface documented in /docs
What a production RAG system needs.
- Permission filtering before ranking, not after generation
- Labels on chunks, not just documents
- Citations users can verify
- Re-indexing when permissions change, not only content
- Evaluation sets tied to known source documents
- Retrieved content treated as untrusted input to the model
What to measure, and who sets the bar.
There is no universal accuracy number for RAG. These are the measures teams track; the thresholds are yours, set against your own questions and documents.
| Measure | How it is checked | Typical owner |
|---|---|---|
| Retrieval recall | Do expected source passages appear in the candidate set for each test question? | Platform team |
| Citation faithfulness | Does each cited passage actually support the sentence that cites it? | Domain reviewers |
| Access correctness | Do test identities ever receive chunks they should not? Expected result: never | Security team |
| Freshness | How long after a source edit or permission change does retrieval reflect it? | Data team |
| Refusal behavior | Does the system say it cannot answer when sources are missing, instead of guessing? | Business owner |
From one collection to many applications.
- 1
One collection, one audience
Start with a well-permissioned source such as policies or product documentation, and a defined user group.
- 2
Build the test set first
Collect real questions with known sources, plus access tests using identities that should be refused.
- 3
Launch with citations visible
Show sources in the interface from day one, so users learn to check and report bad answers.
- 4
Open the API
Let internal applications call retrieval with the end user's token once the first collection is stable.
- 5
Add sources deliberately
Each new source brings its own permission model; add it to the evaluation set before exposing it.
Questions buyers ask
Which vector database do you use?
Data.co Control includes a vector index that runs in your environment. Integration with an existing vector store can be discussed during scoping, provided it can enforce the same filters.
Can RAG answers be wrong?
Yes. Retrieval and citations make answers checkable, not infallible. Evaluation sets and citation display help users and teams catch mistakes.
How do you defend against prompt injection in retrieved documents?
Retrieved content is treated as untrusted. Because access decisions happen outside the model, an injected instruction cannot widen what is retrieved or which tools are allowed.
Can we use RAG inside our own applications?
Yes. The retrieval API and the OpenAI-compatible gateway can be called from your applications using the end user's identity.
What happens when a document is deleted or its permissions change?
Incremental sync removes or re-labels the affected chunks and updates their access lists. Permission sync age is monitored per source, so a lagging sync is visible rather than silent.
See it against your own use case.
A 30-minute technical walkthrough, policy simulation on your scenarios, and a deployment recommendation for on-prem, private cloud, or hybrid.