GDPR-Compliant AI Development: A Checklist for Product Teams
Short answer
GDPR-compliant AI development means treating every prompt, embedding and log that contains personal data as regular personal data processing: you need a legal basis, a data processing agreement with each model provider, data minimization before data reaches the model, a plan for data subject rights and documented retention. The easiest win is to keep personal data out of prompts entirely.
The GDPR does not have a special chapter for large language models. It does not need one: as soon as personal data is in a prompt, a retrieval index, a fine-tuning set or a log file, the normal rules apply. This checklist is organized by the phase in which product teams typically make each decision, so it can be used as a review list in sprint planning. It is practical guidance, not legal advice; involve your data protection officer for anything non-trivial.
Phase 1: Concept and design
- Map the data flows. Draw where user input goes: frontend, your backend, the model API, vector database, analytics, logs. Every arrow that carries personal data needs to be covered by your records of processing.
- Define the purpose precisely. "Improve the product with AI" is too vague. "Summarize support tickets for agents" is a purpose you can justify and limit.
- Choose the legal basis. Common candidates are contract performance (Art. 6(1)(b)), legitimate interest (Art. 6(1)(f), with a documented balancing test) and consent (Art. 6(1)(a)). Special categories such as health data need an additional Art. 9 condition.
- Check whether a DPIA is needed. New technologies, large-scale processing, profiling or special categories often trigger a data protection impact assessment. Doing it early is cheaper than retrofitting.
- Decide if personal data is needed at all. Many AI features (classification, summarization, drafting) work just as well on pseudonymized or anonymized text.
Phase 2: Vendor and model selection
Each external model provider is usually a processor. Before integrating, verify:
| Check | What to look for |
|---|---|
| Data processing agreement | Art. 28-compliant DPA covering the API or enterprise product you actually use (consumer chat apps often have different terms) |
| Training use | Contractual statement on whether inputs and outputs are used to train models; for business APIs this is commonly excluded, but verify the current terms |
| Retention | How long prompts and outputs are stored, including abuse-monitoring logs, and whether zero or reduced retention is available |
| Data location and transfers | Processing region, sub-processors, and the transfer mechanism for non-EU processing (adequacy decision, EU-US Data Privacy Framework certification, or standard contractual clauses) |
| Security | Certifications such as ISO 27001 or SOC 2 reports, encryption, access controls |
Self-hosted open-weight models remove the third-party processor question but move security and operations responsibility to you. They are not automatically "more compliant."
Phase 3: Building the feature
- Minimize before you send. Strip names, emails, phone numbers, addresses, IDs and free-text identifiers from prompts where the task does not need them. Running this step locally means the raw data never leaves your environment; Textwash Pro is one option that anonymizes text fully offline on the device. Our guide on anonymizing data before sending it to ChatGPT or Claude covers the techniques in detail.
- Separate identifiers from content. Use internal IDs in prompts and re-join the result with personal data in your own backend.
- Watch your retrieval index. Embeddings of personal data are still personal data. Plan how a record is deleted from the vector store when it is deleted from the source system.
- Control logs. Prompt and response logs are useful for debugging and quality, and they are a classic place where personal data lingers forever. Set retention periods, restrict access and redact where possible.
- Guard outputs. Models can repeat personal data from context into places it should not go, such as a reply to a different user. Scope retrieval to the requesting user's permissions.
- Do not train or fine-tune on personal data by default. Removing a person from a trained model is hard. If you must, document the legal basis and the deletion approach first.
Phase 4: Transparency and user rights
- Privacy notice: Name the AI processing, its purposes, the categories of recipients (including model providers) and any third-country transfers.
- Access and deletion: Make sure a data subject request covers prompts, chat histories, embeddings and logs, not just the main database.
- Rectification: If the AI generates statements about a person (for example in a CRM summary), there must be a way to correct them.
- Automated decisions: Art. 22 restricts decisions based solely on automated processing that have legal or similarly significant effects. Keep a meaningful human review for such cases.
- App store disclosures: For mobile apps, AI data flows also belong in Apple's privacy labels and Google Play's Data safety form; see our guide to privacy labels and the data safety form.
Phase 5: Operation and change
- Review the data flow map whenever you switch models, add a provider or introduce a new feature such as memory or file uploads.
- Monitor provider terms; DPAs and retention policies change.
- Include AI components in incident response: a leaked prompt log is a personal data breach like any other and may trigger the 72-hour notification duty.
- Coordinate with the EU AI Act: GDPR and AI Act obligations overlap but are separate. Our article on the EU AI Act for software teams explains what the AI Act adds.
Quick self-test for your next sprint review
If your team cannot answer one of these questions in a sentence, that is the next item for the backlog:
- Which personal data reaches which model provider, and why is each field necessary?
- Where is the signed DPA for each provider?
- How long are prompts and outputs stored, by us and by them?
- How would we delete one user's data from every AI-related store within a month?
- Does any feature make significant decisions about people without human review?
If you would like an outside view on a specific architecture, you can book a free intro call.
Frequently asked questions
Is it GDPR-compliant to send customer data to an LLM API?
It can be, if you have a legal basis, a data processing agreement with the provider, a lawful transfer mechanism for non-EU processing and you only send the data the task needs. Removing personal data from prompts is the simplest way to reduce risk.
Are embeddings personal data under the GDPR?
If embeddings are derived from personal data and can be linked back to a person, for example via the source record, they should be treated as personal data, including for deletion requests.
Do I need a DPIA for an AI feature?
Often, but not always. A data protection impact assessment is required where processing is likely to result in a high risk, which new technologies, large-scale processing, profiling and special categories of data frequently indicate. Your data protection officer should decide.
Is pseudonymized data outside the GDPR?
No. Pseudonymized data is still personal data. Only properly anonymized data, where individuals can no longer be identified by reasonably likely means, falls outside the GDPR.
Want a second opinion on your project?
Book a free 30-minute call with our team. We look at your idea, prototype or app and tell you honestly which route makes sense – whether you work with us or not.
Book a free call or write to [email protected]