Defensive field guide

How to reduce AI security vulnerabilities

A source-linked operating guide to seven high-impact failure paths in AI applications, from prompt injection and poisoned data to excessive agency.

Treat model input, retrieved context and generated output as untrusted. Keep permissions, secrets and irreversible actions in deterministic controls outside the model.

For teams designing, reviewing or operating AI applications with retrieval, tools or sensitive data.

Open the full research desk

7defensive checkpoints

12authoritative sources

24 Aug 2026Evidence checked

21 Aug 2026Content reviewed

Application

Prompt injection containment

Keep model instructions from becoming an authorization boundary.1 2 3

Failure path

Untrusted user content competes with application instructions and can redirect the model toward unauthorized disclosure or actions.1 2 3

Prevent

Separate instructions from data, enforce permissions in deterministic code, minimize connected capabilities and require approval for consequential actions.1 2 3

Detect

Log instruction conflicts, tool requests, denied operations and policy-filter outcomes without retaining sensitive prompt bodies.1 2 3

Validate

Run direct, multilingual and obfuscated adversarial cases against every privileged path and assert that server-side policy still denies access.1 2 3

Known limits

Input filtering and system prompts reduce exposure but do not reliably eliminate prompt injection.1 2 3

Operational sequence

Treat the model as an untrusted planner: validate its proposed action against user identity, resource scope and an explicit allowlist before execution.1 2 3

Evidence for this checkpoint

Model

Model and dependency supply chain

Make every model, adapter, dataset and runtime artifact traceable.1 2 3

Failure path

Compromised repositories, unsafe serialization, mutable tags or opaque providers can introduce malicious code, backdoors or unexpected behavior.1 2 3

Prevent

Pin immutable versions, verify signatures and hashes, prefer safe formats, inventory provenance and sandbox untrusted artifacts before promotion.1 2 3

Detect

Monitor checksum drift, dependency advisories, unexpected files, capability changes and deviations from a signed release manifest.1 2 3

Validate

Reproduce the build, scan artifacts, compare behavior to a trusted baseline and rehearse rollback before production admission.1 2 3

Known limits

A clean signature proves artifact identity, not that the model is safe or that its training data is trustworthy.1 2 3

Operational sequence

Acquire into quarantine, verify provenance and integrity, run security and behavior gates, sign the approved bundle and deploy by digest.1 2 3

Evidence for this checkpoint

Data

Training and fine-tuning data poisoning

Protect the data path that changes model behavior.1 2 3

Failure path

An attacker alters a small but influential subset of training or feedback data to create targeted errors, bias or hidden triggers.1 2 3

Prevent

Track lineage, restrict write access, separate contributors, deduplicate data, review high-influence samples and preserve immutable snapshots.1 2 3

Detect

Look for source concentration, label anomalies, trigger-linked behavior, distribution shifts and training runs that diverge from controls.1 2 3

Validate

Compare clean-room and candidate models on held-out slices, influence-focused tests and known trigger families before release.1 2 3

Known limits

Poisoning can mimic normal long-tail data, and no single statistical test establishes that a corpus is clean.1 2 3

Operational sequence

Ingest with provenance, quarantine anomalies, train reproducibly, compare against a signed baseline and require two-person approval for promotion.1 2 3

Evidence for this checkpoint

Application

Improper model-output handling

Treat generated output as untrusted input to every downstream system.1 2 3

Failure path

Generated markup, code, queries or tool arguments can trigger injection, unsafe rendering or unauthorized execution when consumed directly.1 2 3

Prevent

Use typed schemas, escaping, parameterized interfaces, allowlists and sandboxing; never execute raw model text.1 2 3

Detect

Record parser failures, rejected fields, dangerous sink attempts and mismatches between requested and generated operations.1 2 3

Validate

Fuzz every output-to-sink adapter with malformed structures and verify encoding at the final rendering or execution boundary.1 2 3

Known limits

Schema validity does not establish business authorization or semantic safety.1 2 3

Operational sequence

Generate a typed proposal, parse strictly, authorize values, preview high-impact changes, execute through a narrow adapter and audit the result.1 2 3

Evidence for this checkpoint

Agent

Excessive agency and tool privilege

Reduce what an agent can do, not only what it is told to do.1 2 3

Failure path

An agent with unnecessary functions, broad credentials or unchecked autonomy can turn model error or manipulation into real-world impact.1 2 3

Prevent

Give each task a scoped identity, narrow function set, bounded resources and explicit approval for irreversible or external actions.1 2 3

Detect

Alert on new tool sequences, privilege denials, repeated retries, destination changes and actions outside the user's active intent.1 2 3

Validate

Run scenario tests for compromised tools, hallucinated parameters and malicious context while checking that policy blocks the final action.1 2 3

Known limits

Human approval can become a rubber stamp if reviewers lack context or receive too many prompts.1 2 3

Operational sequence

Plan in read-only mode, resolve exact resources, display the proposed effect, obtain risk-based approval and execute with a short-lived credential.1 2 3

Evidence for this checkpoint

Application

System prompts, secrets and trust boundaries

A hidden prompt is configuration, not a secret vault or policy engine.1 2 3

Failure path

System instructions may be reconstructed or exposed, revealing confidential data or controls that fail once known.1 2 3

Prevent

Keep credentials and sensitive business data outside prompts, enforce controls in code and design instructions to remain safe if disclosed.1 2 3

Detect

Use canary markers, scan responses for protected configuration fragments and investigate repeated extraction-style conversations.1 2 3

Validate

Assume the full prompt is public during threat modeling and verify that no data access or authorization decision changes.1 2 3

Known limits

Refusal instructions can reduce accidental disclosure but cannot guarantee prompt confidentiality.1 2 3

Operational sequence

Store secrets in a vault, pass only scoped results, inspect output for leaks and rotate any credential that crosses the model boundary.1 2 3

Evidence for this checkpoint

Application

Unbounded consumption and denial of service

Bound tokens, tools, time and spend at every level.1 2 3

Failure path

Long contexts, recursive agents, repeated tool calls or many parallel requests can exhaust compute, quota or budget.1 2 3

Prevent

Apply per-user and global quotas, token and step budgets, timeouts, concurrency caps, caching and circuit breakers.1 2 3

Detect

Monitor cost per task, recursion depth, repeated near-identical calls, queue saturation and abnormal completion lengths.1 2 3

Validate

Load-test worst-case contexts and tool loops, then confirm graceful cancellation and bounded financial exposure.1 2 3

Known limits

Hard limits protect capacity but can interrupt legitimate long-running work and require product-specific tuning.1 2 3

Operational sequence

Estimate a task budget, reserve capacity, meter each step, stop at thresholds and return a resumable partial result instead of retrying blindly.1 2 3

Evidence for this checkpoint

Continue the research

Search all 21 defensive dossiers, compare hallucination-control methods and inspect every source record.

Open the full research desk