> ## Documentation Index
> Fetch the complete documentation index at: https://docs.terces.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Quality

> How the Food Additives platform ensures accuracy and reliability of regulatory data

The Food Additives platform maintains data quality through automated testing, exception monitoring, and systematic classification of the source regulatory standard.

## Data Source

All regulatory data comes from a single authoritative source: the **Codex Alimentarius General Standard for Food Additives (CXS 192-1995)**, published by the joint FAO/WHO food standards body.

<Card title="Key Facts" icon="shield-check">
  * **No PII:** The platform contains no personally identifiable information — all data is public regulatory text
  * **Single source of truth:** One authoritative standard, not aggregated from multiple unofficial sources
  * **Structured extraction:** The 530-page PDF standard is systematically extracted and parsed into structured data
  * **Annual refresh cadence:** Data is refreshed when Codex publishes updates (typically annually)
</Card>

## Automated Quality Checks

The platform runs automated tests on every data refresh. These tests verify:

| Check                          | What it ensures                                                     |
| ------------------------------ | ------------------------------------------------------------------- |
| **Ingredient uniqueness**      | Every INS number maps to exactly one ingredient — no duplicates     |
| **Rule uniqueness**            | Every ingredient-category permission appears exactly once           |
| **Permission status validity** | All rules have a recognized status (Permitted, Conditional)         |
| **Difficulty score range**     | All scores fall within the valid 0-5 range                          |
| **Opportunity score range**    | All commercial scores fall within the valid 0-100 range             |
| **Coverage completeness**      | All key datasets contain data after every refresh — no empty tables |
| **Unresolved rule threshold**  | Less than 10% of rules have unrecognized limit formats              |

<Note>If any automated check fails, the data refresh is blocked until the issue is resolved. Data is never published in an inconsistent state.</Note>

## Exception Monitoring

Beyond automated tests, the platform monitors exceptions — data that is valid but unusual and may need investigation:

* **Unrecognized limit formats:** Usage limits that couldn't be automatically classified are flagged for manual review
* **Conflicting signals:** Ingredients that are both specifically permitted AND on the GMP exclusion list for a category are marked as "Conditional"
* **Heavily conditional rules:** Rules with 3+ regulatory conditions are flagged as requiring expert review

## Condition Classification

All 834 regulatory conditions from the Codex standard are classified by type:

| Type       | Count | Accuracy                        |
| ---------- | ----- | ------------------------------- |
| Scope      | 235   | Classified via keyword analysis |
| Exception  | 231   | Classified via keyword analysis |
| Exclusion  | 185   | Classified via keyword analysis |
| Condition  | 117   | Classified via keyword analysis |
| Basis      | 59    | Classified via keyword analysis |
| Carry-Over | 1     | Classified via keyword analysis |
| Other      | 6     | Unclassified (\< 1% of total)   |

**Coverage: 99.3%** of all conditions are classified into a meaningful type.

## Ingredient Name Normalization

INS numbers from the Codex standard are mapped to standardized ingredient names. Known data quality corrections include:

* **Shifted alignments** in the source PDF (INS numbers mapped to wrong names) — 20 corrections applied
* **Truncated names** from PDF extraction errors — restored to full names
* **Consistent formatting** — all names normalized to Title Case for readability

**Result:** 306 INS numbers map to 306 distinct, correctly named ingredients.

## How to Report Data Issues

If you notice data that appears incorrect — an ingredient that seems to be in the wrong category, a limit that doesn't match the published standard, or any other discrepancy — please contact **[support@terces.io](mailto:support@terces.io)** with:

1. The ingredient name and INS number
2. The food category in question
3. What you believe the correct value should be
4. A reference to the Codex standard section (if available)

We investigate all reported issues and apply corrections in the next data refresh.