De-identifying company workspace exports in the data holder's browser: design and benchmark results

Abstract

Company records such as chat workspaces, mailboxes, documents and tables are proposed as training data for language models, and they carry identifiers of employees, customers and other organisations. This report describes a rule-based de-identification engine that runs entirely in the data holder's browser, so that raw records, the people learned from them and the key behind every pseudonym stay on the holder's computer. It combines the people an export names in its own structure, pattern detectors validated by check digits, cues and a public-domain name lexicon, and the holder's own lists. It writes stand-ins from a keyed HMAC-SHA-256, scoped to each file by default, holds every table row to a group of at least 11, screens content for special categories, and rereads its output with a second check that has no memory of the first pass. On the held-out split of the Text Anonymization Benchmark (127 court judgments) it masks every mention of 93.4% of direct identifiers. On the six human-annotated exports of PrivacyBench v2.5, not inspected while the rules were written, it reaches 86.8% F1 (precision 88.1%, recall 85.6%) using export structure, and 81.9% F1 on text alone. It is weakest on organisation names (49.9% recall), links (61.9%), account numbers (68.3%) and street addresses (87.4%). We report the protocol, the published comparison table and the threats to validity.

Keywords: de-identification, pseudonymisation, personally identifiable information, named entity recognition, workplace data, k-anonymity, privacy engineering, training data, benchmark