Data services · Custom pipelines

Pipelines built and operated for you.

When an off-the-shelf dataset is not enough, we design a pipeline that combines marketplace data, your sources and enrichment — and run it with versioning, monitoring and provenance.

Talk to the marketplace team Private deployment

Engagement

From scope to operation

1
Scope
Sources, licenses, outputs, refresh schedule and quality targets. The scope names every input and the license that governs it.
2
Build
Versioned transformations with tests and sample outputs you review before anything runs on a schedule.
3
Deploy
In our environment, your cloud, or on-premises, with secrets kept in your secret store where you host it.
4
Operate
Monitoring, alerting and change logs for every run, with a named owner on each side.

Stages

What each stage does and records

Stage	Does	Records
Ingest	Pulls marketplace data through the API or a private connection, and your sources through their native interfaces	Source, dataset version, request ids, license id
Clean	Validation and repair rules	Change log per rule
Normalize	Maps fields onto the target schema	Mapping table versions, unmapped values
Enrich	Classification, tagging, summaries	Generator, transformation version, quality score
Validate	Row counts, schema checks, freshness and quality thresholds	Pass or fail per check; failed runs do not publish
Publish	Writes to warehouse tables, files or an API	Output version and delivery log

Patterns

Pipelines we design for

Retrieval for AI applications: a pipeline that keeps a search index or vector store aligned with marketplace data and your own documents, where each chunk carries its source record id, license and retrieval time so an answer can cite it. Where a license forbids storing copies, the pipeline stores references and retrieves at query time instead.

Analytics: a pipeline that lands cleaned, normalized tables in your warehouse for BI tools, with provenance columns kept alongside business columns rather than dropped at load time.

Monitoring: a scheduled query that watches a dataset for new records matching your criteria and posts them to a queue or a ticketing system, subject to the API's rate limits and your plan's quota.

Ingest step

Pulling marketplace records with provenance intact

An ingest step pages through a dataset with cursor pagination, honors Retry-After on 429, stops on rights or configuration errors instead of retrying, and keeps each record's provenance and the response's request id for reconciliation. Illustrative Python with standard libraries; no SDK is required.

ingest.py

import os, time, requests

URL = "https://api.data.co/v1/datasets/lawco-court-opinions/query"
H = {"Authorization": f"Bearer {os.environ['DATACO_API_KEY']}"}
body = {"q": "easement", "date_from": "2015-01-01", "limit": 100}

while True:
    r = requests.post(URL, headers=H, json=body, timeout=30)
    if r.status_code == 429:
        time.sleep(int(r.headers.get("Retry-After", "5")))
        continue
    env = r.json()
    if env["error"]:
        # rights_restricted / not_configured: stop and alert, do not retry
        raise RuntimeError(f"{env['error']['code']} {env['error']['request_id']}")
    if env["meta"]["demo"]:
        print("demo records: do not publish downstream")
    for rec in env["data"]:
        write_row(rec, rec["provenance"], env["meta"]["request_id"])
    if not env["meta"]["next_cursor"]:
        break
    body["cursor"] = env["meta"]["next_cursor"]

Illustrative example · final API surface documented in /docs

Standards

Every pipeline ships with

Provenance on every output record
Pinned transformation versions
License checks on each input
Run history and failure alerts
Documented schema
Tests that run before each deployment
An exit path: code and configs you can take over

Licensing comes first.

We confirm that each input's license allows the intended use before building. Some LAW.co layers, for example, cannot be bulk-exported until the provider agreement is signed, and LAW.co's terms prohibit training models on its corpus without a written agreement.

Downloads and rights

FAQ

Questions buyers ask

Who owns the pipeline code?

Set in your agreement. We can deliver code and configuration so your team can run it independently.

Can a pipeline feed our AI agents?

Yes. Outputs can be exposed through the Data.co API and MCP server, or through your own endpoints.

What happens when a run fails validation?

The run does not publish. The previous output stays in place, an alert goes to the named owners, and the failed checks are listed in the run history.

Can a pipeline use LAW.co data today?

Yes, through REST and MCP, which serve live LAW.co records one request at a time. Bulk delivery is awaiting rights confirmation, so pipelines that need a full copy wait on that. Demo records stay labeled wherever they appear.

How is pricing set?

Pipelines are quoted per engagement. API usage by the pipeline counts against your marketplace plan, such as Pro at $149/month or Business at $799/month, or against an enterprise agreement.

Can we start small?

Yes. Many pipelines start as a single dataset, one transformation and one output, then add sources once the first version has run reliably for a while.

Data Enrichment

Add classification, tags, relationships

Private Data Deployment

Datasets inside your network

Dataset Integration

Into your stack and tools

Enterprise