One format for dates, names, codes and units.
Normalization maps differently formatted sources onto a shared schema so they can be joined and queried together — while keeping the raw value next to every normalized one.
What gets normalized
| Field kind | Raw examples | Normalized form |
|---|---|---|
| Dates | “March 3, 1998”, 03/03/98, 1998-3-3 | ISO 8601 (1998-03-03) |
| Jurisdictions | “Cal.”, “California”, CA | A controlled code list you approve |
| Court names | Abbreviated and full forms | A canonical court id plus display name |
| Citations | Spacing and punctuation variants | A normalized citation string for lookup |
| Units and currency | Mixed units, symbols, scales | SI or agreed units with the original retained |
| Names | Case, spacing and suffix variants | A comparison key, with the display form unchanged |
Raw values stay with the record.
Normalized values are written as derived fields. The raw value, the mapping table version and any low-confidence flag travel with them, so a downstream user can always see what the source actually said.
This matters most at the edges. A two-digit year, an ambiguous abbreviation or a court that was renamed can each be normalized in more than one defensible way. Keeping the raw value means a reviewer can see the choice that was made and change the mapping table without re-ingesting the source.
A normalized field next to its raw value
An illustrative delivered record. The raw value is untouched, the normalized value is derived, and the mapping version and confidence explain how one became the other.
{
"decision_date_raw": "March 3, 1998",
"decision_date": {
"value": "1998-03-03",
"is_derived": true,
"method": "date-parser",
"mapping_version": "dates@<version>",
"confidence": "high"
},
"court_raw": "<abbreviated court name>",
"court": {
"value": "<canonical court id>",
"is_derived": true,
"method": "court-table",
"mapping_version": "courts@<version>",
"confidence": "review"
}
}Illustrative example · final API surface documented in /docs
How we build the mapping
- 1
Inventory the variants
Profile every distinct raw value per field across all sources, with frequencies, so the long tail is visible from the start.
- 2
Agree the target
Choose the target form for each field: a standard such as ISO 8601, a code list you already use, or one we propose and document.
- 3
Write mapping tables
Deterministic tables and parsers first, with model-assisted suggestions only for the residue, and every suggestion reviewed before it enters a table.
- 4
Version and re-run
Each mapping table is versioned. A change to a table re-normalizes affected records and logs the before and after values.
How a normalization run is checked
Coverage
The share of values per field that mapped, mapped with low confidence, or stayed unmapped, reported for every run.
Round-trip review
A sample of normalized values is shown next to the raw values for a reviewer, field by field, before a mapping version is adopted.
Drift alerts
New raw variants that appear after a source update are listed, so the mapping tables grow deliberately rather than by guesswork.
Join tests
Where normalization exists to support a join, we test the join itself: how many records link before and after the run.
Deliverables
- Target schema and field dictionary
- Versioned mapping tables
- Normalized and raw values side by side
- Low-confidence mappings flagged
- Re-runnable on every source update
- Coverage report per field
- A list of unmapped values for decision
Normalization is what makes matching possible.
Entity resolution and matching compare values. If one source says “Cal.” and another says “California”, every comparison starts from a disadvantage. Normalizing first keeps matching rules simpler and their decisions easier to audit.
Questions buyers ask
Can you normalize to our internal schema?
Yes. We can map to a schema you provide, or propose one based on the sources involved.
What happens to values that cannot be mapped?
They are left unmapped and flagged, never guessed. You decide whether to map them manually or exclude them.
Do you use AI for normalization?
Mostly not. Dates, codes and units are handled by deterministic parsers and tables. Models may suggest mappings for rare variants, and those suggestions are reviewed before they are added to a table.
Does normalization change the citation text?
No. A normalized citation string is added as a lookup key. The citation as printed in the source stays as the display value.
Can we take the mapping tables with us?
Yes. Mapping tables for your engagement are delivered as data you can load into your own systems.
How do you handle values that change meaning over time?
Mapping tables can be date-aware. A renamed court or a merged jurisdiction maps differently depending on the record's date, and the rule that applied is recorded with the value.
Can normalization run on our own data as well as marketplace data?
Yes. The same mapping tables can be applied to your records and to external datasets, which is usually the point: both sides end up in one form, ready to join.
Bring your sources together.
Tell us which datasets need to line up and the schema you want them in.