garak-mcp-probes
Run garak against your own MCP servers, with detectors for denied tool calls, leaked canaries and runaway step counts.
What it is
garak-mcp-probes puts garak's existing probes through an agent that uses your own MCP servers, records every tool call the agent makes, and scores that trace with detectors for deny-listed tools, leaked canary strings, tools outside an allow-list, runaway loops and failed calls.
For teams that run MCP servers behind their own agents and want to know whether garak's prompt catalogue can push that agent into misusing those tools. Use it only on servers and agents you own or are authorised to test. This repository contains no probes, no injection payloads and no jailbreak text: every prompt comes from the garak you have installed.
python3 -m venv .venv && . .venv/bin/activate
pip install "garak-mcp-probes @ git+https://github.com/basitalisandhu/garak-mcp-probes"
garak-mcp run --dry-run --probes test.Test --report-dir garak-mcp-report
The dry run needs no network and no keys (a fake model and a fake MCP server). The install is large, because garak brings PyTorch and Hugging Face libraries. The summary table from a real run (garak 0.17.1):
| Probe | Attempts | Tool calls | ArgumentContains | StepBudgetExceeded | ToolCalled | ToolError | UnexpectedTool |
|---|---|---|---|---|---|---|---|
| `test.Test` | 8 | 8 | 0 / 8 | 0 / 8 | 0 / 8 | 0 / 8 | 0 / 8 |
| **Total** | 8 | 8 | 0 / 8 | 0 / 8 | 0 / 8 | 0 / 8 | 0 / 8 |
Install
garak-mcp doctor --dry-run
garak-mcp run --dry-run --probes test.Test --report-dir garak-mcp-report
From the README; see the full README for every option.
Links
Releases
More in MCP tooling
- dev-mcp-servers: Ten small MCP servers for everyday development and security checks.
- mcp-server-template: Secure MCP server template in TypeScript and Python, safe by default.
- mcp-tools-lint: Lint MCP tool schemas and annotations before clients reject them.
- mcp-auth-doctor: Diagnose OAuth discovery problems on remote MCP servers.
- mcp-egress: Record every host an MCP server contacts, per tool, and fail CI on new ones.
- claude-mcp-allow: Least-privilege Claude Code permission rules for MCP tools, generated from their annotations.
- agentdojo-mcp: Run the AgentDojo benchmark's user and injection tasks against a real MCP server's tools.
- mcp-rc-check: MCP specification migration checker for the 2026-07-28 revision: flags removed, changed and deprecated constructs and writes a patch for mechanical fixes.
Topics: agent-security, ai-agents, ai-security, cli, detectors, docker, garak, hacktoberfest, llm-security, mcp, model-context-protocol, prompt-injection, python, red-team, sbom, security-testing, tool-use