Does CCPA give consumers the right to know if their data was used to train AI models?

Yes. Under CCPA's right to know (Section 1798.110), consumers can request disclosure of the categories and specific pieces of personal information collected, the business purpose for collecting it, and the categories of third parties to whom it was disclosed. If personal information was used to train an AI model, that is a 'business purpose' that must be disclosed. Consumers can also ask whether their data was sold or shared — and providing data to a third party to train their AI model may qualify as a 'sale' or 'share' under CCPA, triggering opt-out rights.

Can a California consumer opt out of their data being used for AI training?

Often, yes — but the mechanism depends on how the data flows. If you sell or share personal information with third parties for AI training purposes, the consumer has the right to opt out under 'Do Not Sell or Share My Personal Information.' If you use it for your own internal AI training, it may fall under a 'business purpose' exemption — but if that training supports automated decision-making or profiling, CPRA's ADMT regulations give consumers the right to opt out and request human review. Starting in 2026, the CPPA's ADMT rulemaking adds more granular opt-out rights for significant decisions made by AI trained on consumer data.

What happens if a consumer asks us to delete their data from a trained AI model?

CCPA's right to delete requires businesses to delete personal information they have collected — but there are exceptions. Data already used to train a model may qualify for the 'internal use' or 'compliance with legal obligation' exemptions if deletion is infeasible. However, you must stop using the consumer's data for future training, delete it from current datasets, and document why model-level deletion is technically infeasible. The CPPA is actively considering whether 'machine unlearning' or model fine-tuning should be required in specific circumstances. For now: honor the request for stored data, explain the model limitation, and keep records.

Does using customer data to fine-tune an AI model count as a 'sale' or 'share' under CCPA?

Internal fine-tuning (where you train a model on data you collected and the model stays within your organization) is typically a 'business purpose,' not a sale or share. However, if you send the data to a third party (e.g., a foundation model provider's API for fine-tuning) without the third party acting as a service provider under a written contract, that may be a 'share.' If you provide data to a vendor for training in exchange for value, that's likely a 'sale.' The key questions are: who controls the data, who benefits, and whether the recipient is a regulated 'service provider' or a separate 'third party.'

Home / Blog / CCPA Rights for AI Training Data

CCPA Rights for AI Training Data 2026: What Businesses Must Disclose and Honor

Q: Is personal data embedded in a trained AI model still 'personal information' under CCPA?

This is an emerging question. If the model can reproduce or extract identifiable personal information (through memorization, overfitting, or prompting), the CCPA likely still treats the model as containing personal information. The standard is whether the information 'identifies, relates to, describes, or can reasonably be linked with a particular consumer.' Models that output training data verbatim, or that can be probed to recover individual records, almost certainly retain regulated personal information. Proper differential privacy, deduplication, and reinforcement learning from human feedback (RLHF) reduce — but do not eliminate — this risk.

California's CCPA — strengthened by CPRA amendments and the California Privacy Protection Agency's ongoing automated decision-making technology (ADMT) rulemaking — gives consumers meaningful rights over personal information used to train AI models. If your company builds, fine-tunes, or deploys AI systems using data from California residents, you face obligations around disclosure, opt-out, deletion, and data minimization that go beyond traditional privacy compliance.

This guide explains how CCPA applies specifically to AI training data, what rights consumers hold, and how AI builders can reduce compliance risk without crippling their models.

Why this matters now: The California Privacy Protection Agency (CPPA) is finalizing ADMT regulations that give consumers the right to opt out of AI systems that make "significant decisions" about them. These rules apply to AI trained on personal information — meaning data governance choices made during model development become compliance obligations enforceable years later.

Why Training Data Triggers CCPA

CCPA's definition of "personal information" is intentionally broad: any information that "identifies, relates to, describes, or can reasonably be linked with a particular consumer or household." That covers email addresses, IP addresses, browsing history, geolocation, employment information, inferences, and — critically — combinations of seemingly anonymous data that can be re-identified.

When you train an AI model on datasets containing any of this information, two things happen that matter for CCPA compliance:

The training dataset itself is personal information. If a consumer asks what data you collected about them, training data derived from their personal information counts. You must be able to identify and produce it.
The model may memorize or reproduce identifiable data. Overfit models and large language models trained on repetitive data can recite training examples verbatim. If those examples are linked to identifiable individuals, the model itself contains personal information — and CCPA applies to it.

This creates a counterintuitive situation: a model you trained years ago may still hold regulated personal information even if you've deleted the original dataset. Compliance requires thinking about the model as a data store, not just a software artifact.

The Four Consumer Rights That Matter for AI Training

1. Right to Know (Section 1798.110, 1798.115)

Consumers can request: what categories of personal information you collected, the specific pieces of data (for a verifiable request), the source, the business purpose, and the categories of third parties you disclosed it to. If you collected customer support transcripts, browsing logs, or usage telemetry and used them to fine-tune a chatbot or recommendation model, the consumer can ask about that use — and you must be able to answer accurately.

Compliance implication: You need a data inventory that tracks which datasets fed which models, for what purpose, and on what date. "We don't know what data was in the training set" is not a defensible answer if a consumer exercises their right to know.

2. Right to Opt Out of Sale or Sharing (Section 1798.120)

Consumers can tell you to stop selling or sharing their personal information. "Sale" is broadly defined to include any exchange of personal information for monetary or other valuable consideration. "Sharing" includes disclosing data to third parties for cross-context behavioral advertising.

For AI training, this matters in two scenarios:

Sending data to a model provider's API for fine-tuning — If the provider isn't your "service provider" under a compliant contract, you may be sharing personal information. Check whether your model provider's terms qualify as a service provider relationship (data-use restrictions, confidentiality, no secondary use) or whether they're a separate third party.
Providing data to partners for joint AI development — If you exchange data with a research institution, a vendor, or a marketplace in exchange for access to their AI services, that may be a sale.

3. Right to Delete (Section 1798.105)

Consumers can ask you to delete their personal information. For AI training data, this raises the technical question: what does "delete" mean when the data has already been used to train a model?

CCPA recognizes that some exemptions apply. You may retain data to comply with a legal obligation, complete a transaction, or detect security incidents. But the consumer has the right to have their data removed from future training runs. For models already trained, you face a choice:

Delete the data from current training datasets and document why retraining the model to fully remove the influence is infeasible (a common position, but document the analysis).
Apply machine unlearning techniques if the data's contribution is identifiable and removable.
Retrain without the consumer's data, if the cost is justified and the model version is still in production.

4. Right to Limit Use of Sensitive Personal Information (Section 1798.121)

CPRA added the right to limit businesses' use of "sensitive personal information" — which includes Social Security numbers, driver's license data, financial account information, precise geolocation, racial/ethnic origin, religion, biometric data, health data, and sexual orientation. If you train AI models on any of these categories, consumers can demand you limit use to what is necessary to provide the service they requested. Using customer health data to train a general-purpose recommendation model — without a clear, disclosed, limited purpose — likely violates this right.

Model Extraction: When Outputs Become Personal Information

One of the hardest problems in AI privacy compliance is the risk of model extraction — when a user can prompt a model in ways that cause it to output specific individuals' personal information that was in the training data. Research has shown that large language models can be prompted to regurgitate memorized training examples, including names, phone numbers, and addresses that appeared many times in the corpus.

Under CCPA, if a model can be prompted to produce a specific consumer's personal information, the model contains that personal information — and the consumer's rights apply. This means:

The consumer can request deletion (creating the unlearning problem above).
The consumer can request disclosure of what personal information the model holds about them (can you answer?).
Regulators may treat systematic memorization as a data breach if you knew about it and didn't mitigate.

Mitigation strategies: Deduplicate training data to reduce memorization, apply differential privacy during training, use reinforcement learning to teach the model to refuse extraction attempts, rate-limit queries that trigger known extraction patterns, and periodically test your deployed models with adversarial prompts.

The ADMT Rulemaking: What's Coming in 2026-2027

The California Privacy Protection Agency's draft ADMT regulations add teeth to CCPA for AI builders. Key proposed provisions include:

Pre-use notice: Consumers must be notified before ADMT is used to make a decision about them, including what data the ADMT processes.
Opt-out right for significant decisions: Consumers can opt out of ADMT used for decisions that produce legal or similarly significant effects (hiring, lending, housing, healthcare).
Access to ADMT logic: Consumers can request meaningful information about the logic involved in automated decisions, including the expected outcome and the main factors weighted.
Human review: Consumers can request human review of automated decisions, with the reviewer having authority to override.
Cybersecurity audits for ADMT: Businesses deploying ADMT at scale may need to submit risk assessments and cybersecurity audits.

These rules apply to AI systems trained on personal information — so the compliance burden starts at the data layer, not at deployment. If you trained a model on CCPA-regulated data, your ADMT obligations follow the model wherever it's deployed.

Practical Compliance Checklist

✅ Map training datasets to source populations — Know which records came from which source, when they were collected, and under what notice. If you can't identify the source of a dataset, you can't honor deletion or access requests for it.
✅ Label training data with provenance metadata — Tag datasets with collection date, legal basis (consent, business purpose), intended use, and retention policy. This metadata is what you'll need to respond to access requests.
✅ Classify your model provider relationship — Is your foundation model provider a "service provider" under CCPA, or a "third party"? Read the data processing addendum. If they retain your data for their own model training, they're a third party — and you may be sharing.
✅ Implement opt-out infrastructure for training data — When a consumer opts out, you need to remove their data from current datasets AND flag it to exclude from future training runs. A "do not train on" list should be versioned alongside your data pipeline.
✅ Deduplicate training data — Repeated records drive memorization. Deduplicating reduces both memorization risk and the number of consumers whose data can be extracted from the model.
✅ Test deployed models for extraction — Run adversarial prompts targeting known individuals. If you can recover personal information, treat it as a security bug and patch (fine-tune, filter, retrain).
✅ Document why you're retaining training data — CCPA allows retention for legal compliance and to complete the purpose for which it was collected. If you're holding data for future model retraining, state that purpose in your privacy notice.
✅ Honor deletion requests at the data layer even if you can't unlearn at the model layer — Delete from stored datasets, flag the consumer's record as "requested deletion," document the limitation for the trained model, and retrain at the next scheduled cycle without their data.

Common Misconceptions

"Anonymized training data is exempt."

True anonymization — where data cannot reasonably be re-identified — is exempt. But most "anonymized" datasets aren't truly anonymous. Removing names from records that still contain zip codes, timestamps, and behavior patterns may not meet the standard. Hashed identifiers, pseudonymized data, and "de-identified" datasets that can be joined with other tables to re-identify individuals are all regulated personal information under CCPA.

"Public data is fair game for training."

CCPA exempts "publicly available information" — but defines it narrowly as information from federal, state, or local government records. Social media posts, blog comments, and forum data are not "publicly available" under this definition even if they're technically viewable by anyone. Scraping public posts and training on them may still trigger CCPA obligations toward California residents whose data you collected.

"We don't sell data, so we don't need an opt-out."

CCPA covers both "sale" and "share." Sharing includes providing data to third parties for cross-context behavioral advertising — and may extend to data partnerships where a vendor uses your customer data to improve their own AI products. Audit every data flow: if a third party receives consumer personal information and benefits from it beyond providing a service back to you, it may be a share or sale.

Frequently Asked Questions

We train on usage telemetry (clicks, page views, session duration). Is that personal information?

Yes, if it can be linked to a particular consumer or household. Usage telemetry tied to a user ID, device identifier, IP address, or session token is personal information under CCPA. Aggregate, de-identified statistics that cannot reasonably be linked back to individuals are not — but the bar for 'reasonably' is lower than most engineers assume. If you retain any linkage key (even hashed), the data is likely still personal information.

We use synthetic data for training. Does CCPA apply?

Truly synthetic data — generated programmatically with no link to real individuals — is not personal information and falls outside CCPA. But synthetic data generated from real consumer records (e.g., training a generative model on real data, then sampling from it) may inherit privacy risks. If a synthetic record can be traced back to or used to infer information about a real individual, it may be regulated. Use differential privacy guarantees or strict generative separation to reduce this risk.

What's the penalty for using consumer data in training without proper notice or opt-out?

CCPA civil penalties start at $2,500 per violation and $7,500 per intentional violation or violations involving minors. Each consumer whose rights were violated counts as a separate violation — so training on 10,000 California consumers' data without notice could theoretically mean $25M-$75M in statutory penalties. Consumers also have a private right of action for data breaches (Section 1798.150) with statutory damages of $100-$750 per consumer per incident. CPPA enforcement actions can be brought in addition to civil penalties.

Do we need to update our privacy policy to mention AI training?

Yes. CCPA requires privacy policies to disclose the categories of personal information collected, the business purposes for which each category is used, and whether each category is sold or shared. If AI training is a business purpose, it must be listed. The CPPA's ADMT regulations will add specific notice requirements for automated decision-making. Generic statements like 'we use data to improve our services' are insufficient — you need to specifically identify AI/model training as a purpose.

Build Privacy Controls Into the Training Pipeline

AI privacy compliance fails when data governance is bolted on after deployment. The companies that survive CCPA scrutiny are the ones that build provenance tracking, opt-out infrastructure, and extraction testing into their ML pipeline from day one — so that when a consumer exercises their rights, the answer is a lookup, not a forensic investigation.

Treat every training dataset as a regulated asset. Tag it, track who contributed what, flag which consumers opted out, and test your deployed models for leakage. The cost of doing this during model development is a fraction of the cost of responding to enforcement actions, retraining emergency patches, or notifying regulators that your chatbot is regurgitating customer data.