Data policy

Result bins

SwapAI bins every result. Boolean and string classifiers use one bin per value. Numeric classifiers use the declared range and product decision boundaries.

const relevance = createClassifier({
  // ...
  result: { type: "number", min: 0, max: 1 },
  decisionBoundaries: [0.5],
});

A boundary creates dedicated near-boundary coverage because a prediction of 0.49 instead of 0.51 can change product behavior even when its numeric error looks small.

Retention is balanced by result bin, purpose and declared-facet group instead of merely keeping the newest examples globally. This prevents a common class from pushing rare but important outcomes out of the dataset.

Facets

Facets describe product slices whose distribution must remain visible during collection.

const relevance = createClassifier({
  // ...
  facets: ["documentFamily", "source"],
});

await relevance.classify(input, {
  documentFamily: "bank-statement",
  source: "gmail",
});

Declare only low-cardinality dimensions that change the task or population. Never put secrets or unbounded identifiers in a facet.

Inspection reports every observed value, an explicit (unlabelled) group, totals and per-purpose counts for each declared facet. Facets do not independently gate readiness or promotion: SwapAI cannot infer every value your product expects. Your application must review that distribution and decide whether it covers the population it intends to serve.

Stable assignment

Assignment uses classifier identity and the input string—not its current answer. This has two consequences:

• duplicate inputs cannot leak across training and protected tests;
• correcting a result does not move that input into a different purpose.

Default minimums

Defaults are deliberately conservative and can be overridden per classifier:

datasetRequirements: {
  minimumTrainingExamples: 1_000,
  minimumTrainingExamplesPerResultBin: 50,
  minimumValidationExamplesPerResultBin: 20,
  minimumRepresentativeTestExamples: 200,
  minimumCoverageTestExamplesPerResultBin: 30,
}

These are readiness gates, not a promise of accuracy. A harder task may need substantially more data. inspect().deficits is the source of truth for the current classifier.

Protected evaluation can never be empty. Even when configured minima are zero, readiness still requires at least one validation example, at least one representative-test example and at least one coverage-test example across the retained dataset. These aggregate requirements appear with resultBin: null; per-bin requirements remain separate.

Legacy examples

Data created by older SwapAI releases did not distinguish all four purposes. SwapAI adopts it once, on the first configured open, according to whether the rows could have influenced a model.

If legacy rows were never exposed to training, SwapAI deterministically assigns them across all four purposes from classifier identity and input hash. This is allowed only when there is no local training attempt or examples-used count, previous generation, trained model, provider training run or model artifact.

If any of that exposure evidence exists, automatic schema migration maps the old training split to training and the old held_out split to validation only. It never turns exposed rows into representative-test or coverage-test evidence. An operator who can independently attest that held-out rows were never used for model selection can use the separate two-phase held-out migration.