Source Code in AI Coding Assistants: Keeping Secrets Out of Copilot, ChatGPT, and Claude

Short answer: Source code is one of the highest-value, easiest-to-leak assets in a modern engineering organization, and AI coding assistants — GitHub Copilot, ChatGPT, Claude, Cursor, Windsurf, Amazon Q Developer, Codeium — all create new exposure paths that traditional secret scanning and DLP were never designed to catch. This guide explains, in plain English, the specific ways code leaks into AI tools in 2026, what each provider actually does with prompts and completions, and the controls that engineering and security teams can deploy without slowing developers down.

Why source code is harder to protect than customer data

For most companies, source code carries three kinds of risk simultaneously:

1. Trade secrets. Algorithms, infrastructure designs, and product internals that derive their value from being non-public. Disclosure can be irreversible — once a competitor sees an architecture diagram, you cannot un-disclose it.
2. Embedded secrets. Hard-coded API keys, database URLs, OAuth tokens, JWT signing keys, third-party credentials, and PII test fixtures. A leaked key may grant access to production within minutes of the leak.
3. Vulnerabilities pre-disclosure. Code that contains a security bug not yet fixed. Public exposure before patch release is functionally a zero-day.

Customer-data DLP focuses on patterns the data itself contains (SSNs, card numbers). Source-code DLP has to also reason about what the code knows or implements — which is much harder for a regex-based system to detect.

The four common code-leakage paths through AI tools

1. Pasting code into a chat to "fix this for me"

The most common pattern in 2024–2026: a developer copies a function, file, or stack trace into ChatGPT, Claude, or Gemini and asks for help. The block may include API keys in test fixtures, customer-data examples in test cases, comments referencing internal architecture, or a license header that itself reveals proprietary status.

Why it slips past existing controls. Most code-aware DLP runs on commits, pull requests, or push events — not on arbitrary keystrokes inside a browser tab.

What to do. Redact at the source. A local-first tool such as Sonomos detects sensitive entities — including secret-pattern formats (AWS keys, Stripe keys, JWTs), customer identifiers, and explicit PROPRIETARY markers — and replaces them with reversible tokens in the browser, before the request leaves the device.

2. AI coding assistants reading your repo

Copilot, Codeium, Cursor, Cline, Continue, and similar tools analyze the active file plus relevant context — sometimes the whole repo. Each provider's defaults differ:

• GitHub Copilot Business / Enterprise — Code suggestions are not retained for training; "Code referencing" is opt-in for organizations.
• GitHub Copilot Free / Individual — Telemetry and code snippets may be retained per the consumer privacy settings; check the current product page.
• Amazon Q Developer (Pro) — Code suggestions not used for training; enterprise features include indemnification and reference tracker.
• Cursor / Windsurf / Cline (Anthropic-or-OpenAI-backed) — Vary by configuration; review settings and provider DPA before enabling on private repos.

What to do. Audit the assistant's settings per repo. Disable repo-wide context for sensitive repos, or run such repos with a self-hosted assistant.

3. Implicit secrets in prompts and stack traces

Even careful developers paste stack traces, log snippets, environment-dump output, or error messages into AI tools — content that often contains tokens, hostnames, IP ranges, or path names that reveal infrastructure. A stack trace from a production incident is rarely sanitized in the rush to get help.

What to do. Train the team to redact production output before pasting. Better, intercept it at the browser layer with the same local-first tool used for code. Make the safe path the easy path.

4. Prompt injection via dependency / fetched-content

When a coding agent reads files from outside the user's own repo — a fetched dependency, a README from a remote project, a copied-in third-party SDK — it may ingest content authored by an attacker. Indirect prompt injection in code agents has been demonstrated repeatedly in 2024–2025: attacker-controlled comments instruct the agent to leak environment variables, modify generated code, or run unintended shell commands. (See our prompt injection explainer for the broader threat model.)

What to do. Limit what the agent can do — not just what it can read. Apply egress allow-listing to the agent runtime, validate generated code with a deterministic linter / SAST step before execution, and require human confirmation for high-impact tool calls.

What providers actually do with your code in 2026

| Provider / product | Used for training? | Retention | DPA / enterprise terms | | --- | --- | --- | --- | | OpenAI ChatGPT Free / Plus | Yes, unless opted out | History on by default | Consumer terms | | OpenAI ChatGPT Enterprise / Edu / API | No | Configurable; ZDR available | Enterprise DPA | | Anthropic Claude (claude.ai consumer) | No, by default | History on by default | Consumer terms | | Anthropic Claude for Work / API | No | Configurable; enterprise DPA | Enterprise DPA | | GitHub Copilot Business / Enterprise | No | Snippets not retained | Enterprise DPA + audit logs | | Amazon Q Developer Pro | No | Configurable | AWS DPA / BAA where applicable | | Google Gemini (consumer) | May review prompts | Configurable | Consumer terms | | Google Gemini (Workspace / Vertex) | No | Configurable | Enterprise terms | | Microsoft 365 Copilot (Commercial Data Protection) | No | Per M365 terms | Enterprise DPA |

These positions move; verify against the current vendor product pages before relying on them. For private repos, the difference between consumer and enterprise tier is the difference between "your code may train future models" and "your code is processed under contractual confidentiality."

What "private context" really means

Several AI coding tools advertise "your code stays private." Read what is actually being claimed:

• No training use. Most enterprise tiers commit to this. Important but limited — it does not prevent transit, retention for abuse monitoring, or human review on flagged content.
• Zero retention. Stronger: prompts and completions are not stored beyond the duration of the request. Available on enterprise tiers from major vendors; usually requires an explicit configuration.
• Local execution. The strongest claim: model inference happens on the user's device. Available from a small number of products (e.g., self-hosted deployments, some IDE-based small models). Latency and capability trade-offs are real.

For most teams, the practical sweet spot is enterprise tier + zero retention + local-first redaction at the browser layer — strong contractual posture for the agent's primary work, plus a deterministic technical control to prevent the most-sensitive bits from ever leaving the device.

A workflow that scales without slowing developers

Below is a pattern several engineering organizations have adopted in 2025–2026:

1. Tool selection. Approve specific AI coding products under enterprise contracts (GitHub Copilot Enterprise, Claude for Work, Cursor with enterprise terms, etc.). Block consumer endpoints at the network and identity layers for engineering accounts.
2. Repo classification. Tag each repo with a sensitivity level. High-sensitivity repos get stricter assistant settings (no fetch outside repo, no telemetry beyond the necessary minimum, mandatory secret scanning on every commit and every push).
3. Browser-level redaction for chat-style AI. Deploy a local-first tool (Sonomos and similar) on every developer device. The tool detects secrets (cloud-provider keys, JWTs, private keys, internal hostnames, customer data fixtures) and replaces them with reversible tokens before any chat-style AI sees them.
4. Pre-commit + pre-push secret scanning. Use a tool like detect-secrets, gitleaks, or trufflehog configured for your secret patterns. Make the scan blocking — the cost of an accidentally committed key is days of rotation.
5. Indirect-injection defense for agentic tools. For agents that fetch and act, restrict the action surface. Allow-list outbound destinations from the agent runtime. Require human confirmation for sensitive tools (file delete, git push, deploy, secret read).
6. Auditing and review. Sample a percentage of AI-assisted PRs each sprint for security review, especially around authentication, authorization, and crypto. AI code is often syntactically clean and subtly wrong.
7. Training and honeypots. Run periodic fire drills: planted "fake secret" files in low-traffic repos to verify your secret scanning catches them. Train new hires on the AI policy as part of onboarding, not optional reading.

Examples: what is and is not safe

| Scenario | Safe? | Why | | --- | --- | --- | | Pasting a 50-line function from a private repo into ChatGPT Plus to "rewrite for clarity" | Risky | Consumer tier; code may be retained or used for training | | Same function, with secrets and customer-data fixtures redacted by a local-first tool, into Claude for Work | Acceptable | Reversible tokens preserve utility; raw secrets never leave device | | Asking Copilot Enterprise for completions inside a private repo | Acceptable | Enterprise terms; not used for training | | Pasting a stack trace containing an unredacted JWT into any AI chat | Not acceptable | Secret may grant immediate production access | | Asking a code agent to "use this repo and follow its instructions to refactor" | Risky | Indirect injection vector via file content; restrict action surface | | Building a private LLM grounded on internal APIs, deployed in your VPC | Acceptable with controls | Strong privacy posture; still requires output review |

Frequently asked questions

Does GitHub Copilot Enterprise see all my code?

Copilot processes the active file and contextual snippets to produce suggestions. With Business and Enterprise tiers, those prompts are not used for training and are not retained beyond the request. The tool does not maintain a long-lived index of your repo unless you have specifically enabled features that do so (e.g., Copilot Enterprise Knowledge Bases). Read the product page for the current architecture.

Are AI completions copyrightable, and do they create license risk?

This is an evolving area. Several providers offer indemnification for code generated under their enterprise tiers (Microsoft for Copilot, Amazon for Q Developer, GitHub for Enterprise). The protections come with conditions — typically that the customer has filtering enabled and follows reasonable processes. Read the indemnification clauses; they are not all alike.

Is it safe to use AI for security-relevant code (auth, crypto, SAST)?

AI tools can help draft and review such code, but treat their output as a junior engineer's first pass. Security-relevant changes should go through human review by someone qualified, plus deterministic tooling (linters, SAST, dependency analysis). The 2024–2025 stream of AI-assisted vulnerabilities in production systems shows the failure mode is real.

What's the difference between secret scanning and code redaction?

Secret scanning checks files at commit / push time and blocks known patterns. Code redaction (browser-layer) catches secrets before they leave the device — including ad-hoc paste-into-chat scenarios that secret scanning never sees. Use both; they cover different points in the lifecycle.

How do indemnification clauses interact with redaction?

Indemnification typically requires the customer to follow reasonable security practices. A redaction layer is increasingly cited as exactly the kind of "reasonable measure" expected of a customer. Combining a vendor that indemnifies output with a redaction layer that controls input is the strongest current posture.

Are there standards for AI-coding-tool governance?

Several emerging frameworks help: ISO/IEC 42001 (AI management systems), NIST AI RMF, OWASP Top 10 for LLM Applications, and SLSA (supply-chain integrity) intersect with AI-coding-tool decisions. These are starting points; tailor to the specific products and risk you accept.

A short checklist for engineering and security leaders

• Inventory every AI coding tool actually in use (sanctioned and not).
• Choose enterprise tiers with DPAs and no-training commitments; block consumer endpoints at the network and identity layers.
• Deploy local-first redaction at the browser for chat-style AI tools.
• Make secret scanning blocking at pre-commit and pre-push.
• Classify repos by sensitivity; configure assistants per class.
• Restrict the action surface and outbound network of agentic tools.
• Sample AI-assisted PRs for security review each sprint, focused on auth, authz, and crypto.
• Add prompt-injection scenarios to red-team exercises.
• Update the engineering handbook so the AI policy is on day one, not in a quarterly compliance reminder.

The bottom line

In 2026, AI coding assistants are net positive for almost every engineering team — but the threat surface has moved. Source code, secrets, and architectural detail can leak through routes that traditional DLP and secret scanning never watch. The defense is the same defense in depth that has always worked: enterprise contracts, local-first redaction at the input layer, strict action surfaces for agents, and human review for security-relevant code. Get those right and AI is a productivity multiplier. Skip them and you are training someone else's model on your codebase, one paste at a time.