A prompt injection test here, a red team exercise there, a retrieval audit somewhere else — these only become a real security program once they're wired into a pipeline that runs continuously and maps cleanly onto the layers of your actual AI system. Here's that architecture.
Scope note: for the phase-by-phase checklist of what to verify before, during, and after deployment, see LLM Security Checklist for Production →. This guide is the underlying architecture those checklist items map onto.
Most organizations start with isolated checks: someone runs a prompt injection test before launch, someone else schedules an occasional red team exercise. These checks are valuable, but disconnected from each other they leave gaps — a retrieval pipeline that was audited once at launch but never re-checked after the knowledge base changed, an agent whose permissions were reviewed in isolation from how it actually behaves at runtime. A pipeline is what closes those gaps: the same five layers, tested the same way, continuously, with each layer's output feeding the next.
Prompt filtering, injection detection, and semantic analysis applied to everything entering the system — user messages, uploaded files, API parameters.
Deep dive: Prompt Injection Attacks →
Adversarial testing, jailbreak resistance validation, and policy enforcement checks against the model itself, independent of what's feeding it.
Deep dive: AI Red Teaming →
Document validation, embedding integrity checks, and poisoning detection for any RAG pipeline or vector database the system reads from.
Deep dive: RAG Security →
Workflow restrictions, tool permission audits, and execution-boundary testing for anything with the authority to take real actions.
Deep dive: AI Agent Attack Simulation →
Observability, anomaly detection, and behavioral monitoring once the system is live — the layer that catches what pre-deployment testing structurally can't.
Deep dive: AI Behavior Monitoring →
The pipeline isn't five separate checklists run in parallel — each layer's findings inform the next. A weakness found in input security (Layer 1) often points to a specific retrieval source that needs auditing (Layer 3). A runtime anomaly (Layer 5) frequently traces back to an agent permission that was never tightened (Layer 4). Treating these as one connected system, rather than five independent audits, is what turns periodic testing into an actual security program.
Traditional input validation checks for malformed structured data — SQL fragments, oversized payloads, script tags. It's pattern matching against a known-bad list. AI input validation has to evaluate natural language for adversarial intent, and the exact same words can be a legitimate request or an attack depending entirely on context. This is why Layer 1 in an AI pipeline needs semantic analysis, not just pattern filtering — the input validation testing methodology genuinely is different, not just relabeled.
The HexTyx AI Security Assessment runs structured tests across all five layers against your live system, not just one layer at a time.
Each layer gets checked once, but findings never get cross-referenced, so a Layer 5 anomaly never traces back to its Layer 4 root cause.
Traditional pen testing doesn't touch semantic attacks, prompt manipulation, or autonomous reasoning risk — it covers a fraction of the five layers.
Retrieval sources change, agents get new permissions, models get updated — a pipeline tested once at launch is stale within weeks.