Architecture & solutions

AI security & threats

AI-specific threats — prompt injection and prompt poisoning, RAG and training-data poisoning, raw-data corruption, agent memory poisoning, compromised tools, model supply chain, exfiltration, privilege escalation, cost exhaustion — with controls.

Threats

Direct prompt injection & jailbreak: Input filter, system-prompt isolation, refusal tests, tool allow-list
Prompt poisoning (indirect injection in emails, documents, web pages, tickets): Treat retrieved content as data not instructions, content sanitising, provenance tags, human approval of irreversible actions
RAG / knowledge-base poisoning: Only curated, owner-approved sources; signed ingestion; change review; anomaly detection on new chunks
Training / fine-tuning data poisoning and backdoors: Curated datasets with lineage, reviewer sampling, held-out evaluation and red team before release, adapter signing
Raw data corruption (source systems, extracts, object storage): Checksums and hashes on ingestion, immutable raw zone (WORM), reconciliation with systems of record, agents have no write access to raw data
Agent memory poisoning: Scoped, expiring memory; no cross-user memory; memory writes logged and reviewable
Tool output manipulation / compromised MCP server: Signed and registered MCP servers only, mTLS, response schema validation, deterministic validators
Model supply chain (tampered weights, malicious packages): Weights from approved registries, hash verification, SBOM, image signing, scanning
Data exfiltration through the model or tools: DLP, egress allow-list, data-class routing, output scanning, real-time audit
Privilege escalation by an agent: Least privilege, ABAC decided by the policy engine, agents cannot change access or policies
Denial of wallet (cost exhaustion) and runaway loops: Step, token and cost budgets per run, rate limits, kill switch

AIHA Office

AI security & threats

Threats

More in this demo