Luganda PII detector, trained with differential privacy

Type or paste Luganda text and see which words look like personal information. The detector is one of the models from an MSc study of differentially private (DP) PII detection. Everything runs in your browser: your text is never uploaded.

A research demo, not a redaction tool. Even the least private of these models misses about half of the PII mentions in held-out test text, and stronger privacy misses more (see how good is it). They were trained on lower-case Luganda news sentences, so other kinds of text will do worse. Always review the output yourself.

Try it

Examples (invented):
Privacy level:

Why the model is private

Differential privacy is a property of how a model was trained, so it cannot be shown by looking at one prediction. What can be shown is the training recipe and its accounting, which anyone can check, plus an attack that fails.

Privacy certificate: ε =

Model , trained with DP-SGD (Opacus) from a warm start on public and synthetic data only. Guarantee: adding or removing any one training sentence changes the probability of any outcome of training by at most a factor eε, except with probability δ. ε is certified by the PRV accountant, which was used to calibrate the noise; the older RDP accountant gives a looser bound of ε = for the same training.

Check the arithmetic yourself

ε follows from four numbers alone. This calculator is a line-by-line port of the Opacus RDP accountant; it reproduces the RDP ε that Opacus recorded for every DP training run in the study to within 10−7. RDP is a looser, still valid bound than the certified PRV value. Change σ or T to see how the guarantee moves.

ε (RDP) = –

An attack that fails: membership inference

An attacker tries to tell whether a sentence was in the training data from the model's loss on it. If training sentences (solid) have lower loss than unseen ones (dashed), the model has memorised them. AUC 0.5 means the attacker does no better than a coin flip.

training sentences (members)   unseen test sentences

This attack is weak, so it gives a lower bound on leakage, not proof of its absence. The proof is the certificate above; the attack shows the effect is real.

What DP protects, and what it does not

How good is it?

Scores on 536 held-out test sentences (entity level, exact match). Privacy has a large cost in accuracy, and the rules recover part of it at no privacy cost.

ModelεF1FindsWith rules: F1 / findsAttack AUC

The browser copy of the ε-model is compressed to 8-bit weights; its test F1 is against for the full-precision model. The rules here are adapted to typed text from the evaluated version (they also accept spaced phone numbers and emails).

Share of each kind of PII found (ε = )

CategoryNon-privateDP modelDP model + rules

The test set has only 22 ID and 43 date mentions, and its IDs are unusually uniform, so the rule scores for those categories overstate what to expect on real text.