Why take this course?
When useful systems reveal too much
Privacy failures do not always look like someone stealing a password. They can happen when a useful product reveals more than its designers expected:
A map of ordinary runs
Each run looked harmless on its own. But when Strava combined millions of routes, bright lines traced roads and the perimeter of a military base. Nothing had to be hacked: the privacy failure emerged when individually ordinary records were aggregated and released.
The surprise is that no single runner had to disclose the base. The sensitive pattern appeared only after the service transformed and visualized many records together. This case asks where a release boundary should sit, what information should be minimized, and when an aggregate can still expose people or places.
Anonymous, until the datasets meet
Netflix removed names and account identifiers before releasing millions of movie ratings from roughly 500,000 subscribers. To someone looking at the file by itself, a row of movie IDs, scores, and dates could appear anonymous.
The researchers treated public IMDb ratings as background knowledge. A few overlapping movies, approximate scores, and rough dates could narrow the candidates to one Netflix record. Once that link was made, the person's other ratings in the released record became visible.
The lesson is broader than these two websites: anonymity depends on what other data exists and how distinctive a person's behavior is. You will study scoped linkage attacks in class and test how generalization, suppression, or differential privacy changes what an attacker can infer.
A model that remembered too much
GPT-2 never exposed its training files directly. Yet researchers found prompts that made the model reproduce memorized text, including the redacted contact details shown here. The model itself became a query interface to its training data.
The attack did not start with a known phone number. The researchers generated many completions, ranked strings that looked unusually likely under GPT-2, and then checked whether the strongest candidates appeared verbatim in the training sources. That verification step turned a surprising output into evidence of extraction.
This story connects model behavior back to the data pipeline: deduplication, the definition of a training record, memorization tests, and private training all affect the risk. You will learn to distinguish a plausible leak from a reproducible privacy claim.
From failures to defenses
There is no single privacy switch. The right defense depends on what a system will release: an answer, a trained model, or a reusable dataset.
Limit what one person can change
Differential privacy compares two neighboring worlds: one with a person's data and one without it. If the observable outputs remain close, an analyst cannot confidently determine whether that person participated or what their record contained. The privacy parameter controls how close those worlds must be.
This is stronger than deleting names because the guarantee anticipates outside knowledge and repeated analysis. It also forces precise questions: What counts as one person's record? Which output is protected? How much privacy budget is spent?
Bound influence during model training
DP-SGD limits each training example's influence before a model update. It computes per-example gradients, clips each one to a fixed norm, aggregates them, and adds Gaussian noise. A privacy accountant then tracks the cumulative cost across many training steps.
The diagram is simple; a correct implementation is not. Sampling assumptions, clipping location, record boundaries, noise calibration, and accounting must agree. You will use Opacus and JAX Privacy to trace those choices from configuration to code and test whether a claimed guarantee matches the training run.
Release new records, not the original people
Synthetic data replaces original rows with newly sampled records, but synthetic does not automatically mean private. In this NIST example, the original table is summarized, noise is added to the counts, and only then are new rows sampled from the privatized distribution.
The release can preserve useful population patterns while protecting individual contributions, but ranges, rare categories, and correlations still determine whether it is useful. With Google DPSynth, you will audit both sides of that claim: the formal release boundary and the empirical utility of the generated data.
From examples to engineering judgment
These cases raise the questions we will learn to answer: What information is actually sensitive? What does an attacker already know? Can a person be linked, reconstructed, or detected in a dataset? Does a model memorize its training records? Does a claimed defense match the data pipeline, code, configuration, and privacy accounting?
You will study privacy failures, measure their impact in scoped exercises, and test defenses ranging from data minimization and access boundaries to differential privacy, private machine learning, and synthetic data. We will also survey secure multi-party computation (MPC), homomorphic encryption (HE), trusted execution environments (TEEs), and network privacy tools. The emphasis is on technical understanding, experimental reasoning, and clear communication rather than graduate-level novelty.