Skip to main content
    Back to Blog
    8 min readLast reviewed:
    AI Security
    Data Leakage
    LLM Security
    Compliance
    DLP

    AI Data Leakage: 7 Ways Sensitive Information Escapes to LLMs (and How to Stop It)

    Sonomos Research

    The Sonomos research team writes about AI privacy, data protection, and how to use generative AI safely at work.

    Most AI data leaks are not the work of a sophisticated adversary. They happen quietly, in the seam between a busy professional and a chat window, when a prompt that should have been redacted is sent in full. The result is the same whether the cause is malice or muscle memory: regulated data ends up in the logs, training pipelines, or screen recordings of a system you do not control.

    This article maps the seven most common pathways for AI data leakage in 2026, why each one is hard to catch in code review or DLP rules, and what an effective defense looks like in practice.

    What "AI data leakage" actually means

    In a security context, leakage is the unintended transfer of information across a trust boundary. With LLMs, the trust boundary is the moment a prompt leaves the user's device — once it crosses the wire, the data is in someone else's environment. Leakage matters because:

    • LLM providers may retain prompts for abuse monitoring (commonly 30 days) or, on free tiers, for training.
    • Prompts can be subpoenaed, subject to legal hold, or breached.
    • Outputs can echo earlier inputs, surfacing information to other users in the same workspace.
    • Many regulations (HIPAA, GLBA, GDPR, CCPA, CPRA, EU AI Act) treat the act of disclosure to a processor as a controlled event — even if the processor is "trustworthy."

    If your organization has compliance obligations, leakage is not just a risk; it is a reportable event.

    1. Copy-paste from sensitive documents

    The most common pattern is also the most mundane: an employee copies a paragraph from a contract, a CRM record, or a patient note and pastes it into ChatGPT to "make it sound more professional." Names, account numbers, and IDs hitchhike along.

    Why it slips past DLP. Traditional data-loss prevention products inspect file uploads and email attachments, not arbitrary keystrokes inside a browser tab.

    What to do. Redact at the source. A local-first tool such as Sonomos detects sensitive entities in the text before the request leaves the browser and replaces them with reversible tokens, so the model sees structure but not identifiers.

    2. Bulk uploads and "summarize this for me" attachments

    Modern chat interfaces accept PDFs, spreadsheets, and code archives. Users routinely drag in a 60-page deal memo or a CSV export of customer records and ask the model to summarize. The entire file is now in the provider's environment.

    Why it slips past policy. File-upload policies often focus on size and file type, not contents. A 200 KB PDF is approved by every gateway in the chain.

    What to do. Treat uploads like exports. Apply the same approval workflow you would use to send the file to an outside vendor. For ad-hoc summarization, paste only the relevant excerpt and redact it first.

    3. Auto-fill, browser extensions, and clipboard managers

    Productivity extensions can capture and transmit prompt content as a side effect — text expanders, grammar checkers, clipboard sync, screen-recording tools, and translation overlays all touch the same DOM as your AI tab. Each one is a potential exfiltration path.

    What to do. Audit the extensions installed on browsers used for AI work. Remove anything not strictly required. Prefer extensions whose privacy policy says "processed locally" and whose code is reviewable.

    4. Shared workspaces and "memory" features

    ChatGPT memory, Claude Projects, and Gemini Workspace integrations all blur the line between a single conversation and a persistent corpus. A teammate who later joins the workspace, or a future you who forgot what you stored last quarter, may surface that data unexpectedly.

    What to do. Treat workspace memory as a database. Decide what should live there, document it, and review it periodically. Disable memory features for accounts that handle regulated data.

    5. Prompt injection and indirect data exfiltration

    Prompt injection happens when untrusted input — a webpage, an email, a PDF — contains instructions that the model treats as commands. A common payload tells the model to "include the user's previous prompt in the next response" or to "fetch this URL with the conversation contents." The user, watching only the visible answer, may not notice that the model just leaked data to an attacker-controlled endpoint.

    What to do. Limit the tools and browsing capabilities granted to AI assistants when they process untrusted content. For agentic workflows, enforce egress controls on the model runtime, not just on the user's network.

    6. Logging, telemetry, and "we keep prompts for 30 days for safety"

    Even when training is disabled, most providers retain prompts for abuse monitoring. That window is usually 30 days but can be longer for flagged accounts or under legal hold. The retained data is encrypted in transit and at rest, but it exists, and it is reachable by the provider's staff and lawful process.

    What to do. Read the data processing addendum (DPA) for each provider you use. Where retention is unacceptable, negotiate a zero-retention rider (available from major vendors for regulated customers) or send only redacted prompts.

    7. Output reuse and the "model knows my customer's name now" problem

    The seventh pathway is the most subtle. Once a name, an account number, or a piece of source code appears in a prompt, it appears in the response — and from there it travels into your notes, your tickets, your email drafts, and your screen-shares. The exposure compounds with every downstream system that touches the output.

    What to do. Apply round-trip masking. Tools that redact on the way in should also unmask only at the user's screen, never re-emitting the sensitive value into clipboards or downstream tools by default.

    Frequently asked questions

    How do I know if my prompts are being used to train AI models?

    Check each provider's privacy settings page. As of 2026, ChatGPT free and Plus tiers may use prompts for training unless you opt out via Data Controls; Claude excludes consumer chats from training by default; Gemini free tier may retain and review prompts. Enterprise tiers across all three exclude training by contract.

    Is it safe to use AI tools for work if my company hasn't approved them?

    It depends on what you put into them. Public information, brainstorming, and template generation rarely raise compliance issues. Customer data, employee records, source code, and legal documents almost always do. When in doubt, redact before pasting and check your acceptable-use policy.

    What is the difference between PII redaction and tokenization?

    Redaction replaces sensitive values with a placeholder (e.g., "[NAME]") or removes them entirely; the original is gone from the prompt. Tokenization replaces sensitive values with a reversible token tied to a local mapping; the original can be restored on the user's device but never travels to the AI provider. For AI workflows, reversible tokenization usually preserves more model utility while keeping the source data local.

    Can a prompt itself be a security incident?

    Yes. Many regulations treat unauthorized disclosure to a processor as a reportable event regardless of whether the data was later used or breached. Pasting regulated data into a non-approved AI tool can trigger the same incident-response process as emailing the file to a personal address.

    Do enterprise AI tools eliminate leakage risk?

    They reduce it materially — zero-retention APIs, audited subprocessors, and BAA-eligible deployments are real improvements. They do not eliminate it: an enterprise account does not stop a user from copying the wrong paragraph, and indirect prompt injection works the same against enterprise endpoints. Defense in depth still applies.

    A short checklist for security and IT teams

    • Inventory which AI tools your workforce actually uses (not which are sanctioned).
    • Confirm training-opt-out is enabled on every consumer-tier account.
    • Roll out a local-first redaction tool to every browser used for AI work.
    • Treat AI uploads like external file transfers in your DLP and audit logs.
    • Add prompt-injection scenarios to your tabletop exercises.
    • Review workspace memory and stored projects quarterly.
    • Update your acceptable-use policy to address pasting and uploads explicitly.

    The bottom line

    AI data leakage is a workflow problem more than a technology problem. The leaks happen because the path of least resistance — paste, send, summarize — runs straight through a trust boundary that the user cannot see. Closing the gap means making redaction the path of least resistance instead. When that habit is in place, the productivity benefits of AI no longer come with a quiet, ongoing tax on your data.

    Protect your data while using AI

    Sonomos detects and masks sensitive information before it reaches AI models. 100% local, zero data collection.

    Install Free