LLM pentesting needs to cover more than the model's text output. A proper test scopes the model, its plugins, and its data sources. Then, it works through known risk categories, including prompt injection, data leakage, and tool abuse.
Shipping an AI feature moves faster than most security processes were built to handle. A chatbot goes from prototype to production in weeks, sometimes with a connected tool or two bolted on along the way.
Security testing catches up later, if it catches up at all.
We built our AI and LLM security testing around that reality. This checklist walks through what a proper LLM pentest covers, organised the same way we scope engagements:
- Define what's in play
- Test against known risk categories
Scope This Before You Test Anything
A checklist is only useful once the scope is defined. Before AI security testing starts, get clear on:
- Which model version or versions are in scope. A model behind an API you don't control can behave differently from one release to the next, so make sure you know what you're testing against.
- Which integrations are connected. APIs, plugins, retrieval sources, and any tools the model can call all expand what an attacker can reach through it.
- Which risks are most important for this specific feature? A customer-facing chatbot with access to account data has a different priority list than an internal coding assistant.
Testing without this groundwork produces noise, findings that don't map to anything you can act on. Scope first, checklist second.
The LLM Pentesting Checklist
We test against two overlapping sets of risk categories: one for the model and its immediate interface and one for anything acting with autonomy on top of it.
Risks Tied to the Model Itself
These map to what our methodology calls out under the OWASP framework for LLM applications.
- Prompt injection, both direct (a user typing a malicious instruction) and indirect (a malicious instruction hidden in an uploaded document, a connected service, an API response, or a plugin output the model reads).
- Data leakage, where the model reveals information it shouldn't, whether that's system prompt content, training data artefacts, or another user's data.
- Training data poisoning occurs when someone tampers with the data a model was trained or fine-tuned on, influencing its behaviour later.
- Insecure plugin or tool integration, where a connected tool trusts the model's output more than it should.
- Overreliance, where a product design assumes the model's output is always correct and skips the checks that would catch it when it isn't.
- Insecure output handling, where the model's raw output gets passed downstream (into a database query, a rendered page, or a shell command) without the same validation you'd apply to any other untrusted input.
- Excessive agency, where a model or agent has more permissions or autonomy than the task in front of it needs.
- Privilege escalation, where a lower-permission interaction with the model leads to an action it shouldn't have been able to take.
Risks Tied to Agent Behaviour
When a system goes beyond answering questions and starts taking autonomous action, a further set of risks arises. Our AI risk approach currently covers:
- Goal injection, where an attacker manipulates what the agent is trying to achieve, not just what it says.
- Task hijacking, where a legitimate task gets redirected mid-execution toward something the attacker wants instead.
- Memory poisoning, where an agent's stored context or memory is tampered with to influence future decisions.
- Overpermissioned tools, where an agent has broader tool access than the task requires, widening the attack surface for a successful assault.
- Autonomous misalignment, where an agent's actions drift from its intended purpose over a longer session without a human catching it in the moment.
Agent security is newer, faster-moving ground than the established LLM risk categories above, so treat this as what we currently test for rather than a fixed standard.
Ask us to confirm the current framework we're mapping against when you scope a test.
Book Your Free Call
The Manual Layer That Makes This Work
A scripted test suite runs the same set of prompts against a model every time. It's useful for catching known, repeatable issues, but it stops there.
A human red-teamer adapts the attack across several turns based on how the model responds. They layer in obfuscation or shift the framing mid-conversation the way a real attacker would, rather than the way a fixed script does. That difference is where the more serious findings tend to show up: indirect prompt injection and multi-turn manipulation that a single scripted prompt would never surface.
For teams that want technique-level detail, MITRE ATLAS catalogues named adversarial techniques against AI systems, including LLM prompt injection, plugin compromise, and data leakage. We map our findings against both the MITRE ATLAS and the OWASP framework.
What an LLM Security Test Deliverable Looks Like
Depending on what access you can provide, testing runs as black-box, grey-box, or white-box. From there, a proper LLM pentest deliverable includes:
- Threat modelling of the AI components in scope, not just the model but everything it's connected to.
- Custom test harnesses and red-team prompts built for your specific integration, not a generic prompt list.
- Bias and fairness probing where it's relevant to how the system is used.
- Findings mapped back to the OWASP risk categories above and to MITRE ATLAS, so the report connects to frameworks your team can reference elsewhere.
- Proof-of-concept exploits for anything exploitable, with reproduction steps, not just a description of the theoretical risk.
Want to see a real system example? Check out our lightweight threat-model breakdown of an AI-Agent platform.
Shipping an AI Feature Soon?
The earlier a test happens relative to launch, the more options you have for fixing what the LLM security test finds.
Scope Your LLM Pentest
Common Questions About LLM Pentesting
What's the Difference Between the Model Risks and the Agent Risks, and Do I Need Both Covered?
- If your system only answers questions or generates content, the model-focused risks (prompt injection, data leakage, and the rest) are your main exposure.
- As soon as it can take action on its own (call a tool, update a record, or send a message), the agent-focused risks also apply.
Most production AI features today do both, so most tests need to cover both.
What Counts as "Data Leakage" if the Model Never Had Direct Database Access?
That's specifically the assumption this risk category exists to test. A model doesn't need a database connection to leak something. Training data, a document pulled in at query time, its own system prompt, or anything that has ever touched the model is a potential leak path.
This risk potential is why AI data leakage testing has to look well past whether the model can query a database directly.
Does LLM Pentesting Cover the Agent or Tool Layer, or Just the Model's Text Output?
Both when there's an agent or tool layer to test. Pentesting only the model's text output misses everything an attacker could do by manipulating which tools get called and how. Once a model has real permissions, that's often the more damaging attack path.
How Do You Scope an LLM Pentest Before the System Is Finished Being Built?
By testing what's stable rather than waiting for everything to be finished. The core model integration, the plugins already connected, and the data sources already wired in can all be scoped and tested even while other parts of the product are still in flux.
Who Reads the Report: Engineers or Leadership?
Both. But they usually read the AI security testing reports from different sections. Findings and reproduction steps are written for the engineers who'll fix them.
Alongside that, the report can carry a short summary of overall exposure and priority for anyone who needs the business risk picture without wading through proof-of-concept detail.
Want This Mapped to Your Specific Integration?
Generic prompt lists don't test what your specific setup is exposed to. Tell us what the model connects to and what it can do, and we'll scope a custom LLM pentest around that.
Talk to Us About Your AI System
Want to Read More?
- Avoid hefty fines and make sure your new AI features aren't violating regulatory frameworks by reading our summary of the EU AI Act.
- Learn how attackers are using AI to scale their attacks in our post discussing AI vs human hackers in cybercrime.
- If your AI agent interacts with physical hardware, ensure the endpoints are secure by reading our IoT pentesting guide.
