#Privacy
Articles about Privacy — exploring patterns, best practices, and real-world implementations in production systems.
13 posts tagged with privacy. ← All posts
On-device AI's biggest promise is privacy — but that promise is only real if the architecture actually keeps data on the device. Privacy isn't a feature you add; it's a property of a design where sensitive data has no path off the phone. This post is about building that property in deliberately, and about the honest hybrid designs for when pure local isn't enough.
On-device AI's biggest promise is privacy — but only if the architecture actually keeps data on the device. Privacy isn't a feature you add; it's a property of a design where sensitive data has no path off the phone.
Compliance requires you to collect and keep a great deal of personal data; privacy law requires you to protect it, limit it, and sometimes delete it. Data privacy is the compliance domain that governs the data all the other controls depend on — and it turns "how you handle personal information" from a courtesy into a set of enforceable engineering obligations.
Compliance requires you to collect and keep a great deal of personal data; privacy law requires you to protect it, limit it, and sometimes delete it. Data privacy governs the data all the other controls depend on — turning data handling into enforceable engineering obligations.
A defender's tour of the attacks that target the model and its data — prompt and context extraction, training-data memorization, membership inference and model inversion, model stealing, poisoning and backdoors, and evasion — with what a red-teamer tests and what actually stops each one.
Attacks on the model and its data: system-prompt and training-data extraction, membership inference and model inversion, model stealing, and poisoning/backdoors — what to test and defend, and the honest risk difference between using hosted models and training your own.
Four techniques, each protecting against a different threat, each with real costs — how do you actually decide what to use? Privacy-preserving machine learning in practice isn't about picking one silver bullet; it's about understanding your threat model and composing the right techniques to meet it. This closing post is the decision framework the series was building toward: match the tool to the threat, pay the trade-offs honestly, and combine for defense in depth.
Four techniques, each for a different threat, each with real costs — how do you decide? Start with the threat model (what, from whom, at what stage), compose for defense in depth (federated learning + DP + secure aggregation), anchor on differential privacy's provable guarantee, and price the privacy-utility trade-off deliberately.
Part three of the AI Security Engineering series: protecting the data that flows through an LLM system — how sensitive information leaks out of prompts, logs, and retrieval, and the engineering controls (redaction, data minimization, per-user retrieval authz, residency choices) that actually stop it.
Protecting data in LLM systems: sensitive-information disclosure, PII in prompts and logs (your observability can be the leak), redaction with Presidio, data residency, and per-user access control on the retrieval layer so RAG doesn't leak across tenants.
Sometimes the goal isn't to compute privately on real data but to produce data that's safe to share, publish, or hand to a less-trusted team — data that looks and behaves like the real thing without being about any real person. Synthetic data and anonymization promise exactly this, and they're widely used — but they also hide the subtlest privacy traps in the whole field. This post covers how to make shareable data, and the honest truth about how well it actually protects privacy.
Sometimes the goal is data safe to share. Anonymization (generalization, suppression, k-anonymity) is fragile against re-identification; synthetic data looks safe but the generator can memorize real people. The fix is the recurring move — differentially private synthetic data — and the lesson that nothing is automatically private.
What if you could compute on data while it stays encrypted the entire time — never decrypting it, never seeing it, yet getting the correct result? It sounds impossible, but cryptography makes it real. Homomorphic encryption and secure multi-party computation let parties jointly compute on sensitive data that none of them can read, closing the one gap the other privacy techniques leave open: protecting data even during the computation itself. This post covers the most mathematically remarkable tools in the privacy toolkit.
What if you could compute on data while it stays encrypted, never seeing it, yet get the right result? Homomorphic encryption (math on ciphertext — private inference) and secure multi-party computation (secret sharing — the basis of secure aggregation) close the gap of protecting data in use — the strongest and most expensive tools.
The most direct way to protect data is to never move it. Federated learning trains a shared model across many devices or organizations without their data ever leaving home — the model travels to the data, learns locally, and only the lessons (not the data) come back. It's how your phone's keyboard improves without uploading what you type, and how hospitals can build a shared model without sharing patient records. This post is about learning from data you're never allowed to see.
The most direct way to protect data is to never move it. Federated learning trains where the data lives (phones, hospitals) and shares only model updates — data minimization by architecture. But updates leak too, so it's combined with differential privacy and secure aggregation; it changes the threat model, not the threat.
The theory of differential privacy is elegant; applying it to machine learning training is where it gets real. The key technique, DP-SGD, modifies the ordinary training loop so the resulting model carries a provable privacy guarantee — it demonstrably can't have memorized any individual's data too strongly. This post is about how differential privacy actually gets into a model, the knobs that control it, and the honest trade-offs that decide whether it's worth it.
DP-SGD modifies training — per-example gradient clipping plus calibrated noise, with privacy accounting across every data access — so the resulting model carries a provable guarantee it couldn't have memorized any individual. The knobs (ε, clipping, noise) and the honest costs (accuracy, fairness, compute).
Most privacy protections are hopes dressed as techniques — remove the names, add some noise, trust that it's enough. Differential privacy is different: it's a precise mathematical definition of what privacy means and a provable guarantee that an individual's data can't be inferred from a result. It is the gold standard of privacy, used by the US Census and major tech companies, and understanding its single core idea is understanding what it actually means to protect a person in a dataset.
Differential privacy defines privacy mathematically: the output is essentially the same whether or not any individual is included, so nothing specific about you can be inferred. Calibrated noise, the privacy budget ε, the privacy-utility trade-off, and why DP (US Census, Apple, Google) is the gold standard.
A trained model feels like a safe abstraction — it's just numbers, a learned function, surely it reveals nothing about the specific data it saw. That intuition is wrong, and a family of attacks proves it: models memorize their training data and can be made to give it up. Understanding these attacks is what turns privacy from a vague worry into a concrete threat you can defend against — and it's why the formal techniques in the rest of the series exist.
A trained model feels safe — it's just numbers — but models memorize their training data and can be made to give it up. Membership inference, model inversion, training-data extraction, and model stealing — many working with only black-box API access, which is why a model trained on sensitive data is itself sensitive data.
Machine learning runs on data, and the most valuable data is often the most sensitive: medical records, financial transactions, personal messages, location histories. This creates a fundamental tension — the models that could help most need exactly the data we're least willing to expose. Privacy-preserving machine learning is the set of techniques that resolve this dilemma, letting us learn from sensitive data without surrendering it. This series builds those techniques from first principles, starting with why the problem is real and harder than it looks.
ML wants more, centralized, detailed data; privacy wants less, distributed, minimized — and the highest-value ML sits where that tension is sharpest. Why the naive protections fail (anonymization → re-identification; securing data ≠ securing the model), and the four techniques that make privacy an engineered, provable property.
Handling user data used to be a technical matter; now it's a legal one, with real regulations, real penalties, and real obligations that engineers build software to satisfy. Privacy and compliance have become part of the job — and, along with the rest of this series' legal basics, they lead to the single most important lesson: legal literacy exists to tell you when you're out of your depth and need a real lawyer. This closing post covers privacy, compliance, and that essential meta-skill. (Educational, not legal advice.)
Handling user data used to be a technical matter; now it's a legal one, with real regulations, penalties, and obligations. Privacy and compliance are part of the job — and they lead to the most important lesson: legal literacy exists to tell you when you're out of your depth and need a real lawyer. (Educational, not legal advice.)
All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse all posts or explore related topics using the tag cloud above.