One command, one audit
A CI pipeline should not need a separate mcptest invocation for tests, then another for compliance, another for the security scan, another for conformance, and another for model compatibility. Each extra command is another step to wire, another exit code to reconcile, and another place for the pipeline to drift out of sync with what you actually test locally.
mcptest run folds all of it into one command. A suite that declares capability blocks runs each as a lane alongside the tool and agent tests, and the run returns one report and one exit code that fails if any lane fails.
Run it
examples/full-audit/full-audit.yml declares seven concerns against the built-in mock, all offline and key-free:
mcptest run examples/full-audit/full-audit.yml| Block | Lane | Gates on |
|---|---|---|
tools: | behavioral tests | a failing assertion |
compliance: | the compliance rubric checks | a failed check |
security: { scan: true } | the deterministic tool-surface scan | a finding at or above High |
conformance: | the SEP protocol probes | a failed MUST |
model_compat: | a baseline-versus-candidate diff | a breaking change (drift is advisory) |
coverage: | tool-surface coverage instrumentation | a declared dimension below its minimum (exit 6) |
performance: | a p95 latency budget over per-test durations | nothing (advisory: a p95 over budget is highlighted, not gated) |
The tool/agent tests run first, then each declared lane runs after the drive and folds its rows into the same report. A breaking result in any lane fails the run; non-breaking model drift is reported but does not gate.
One command, not five
Before, the equivalent coverage meant a pipeline step per concern:
mcptest run suite.yml
mcptest compliance run --from-suite suite.yml
mcptest security tools.json
mcptest conformance run --server-command "..."
mcptest model-compat diff --baseline a.json --candidate b.jsonNow it is one line, and the single exit code already reconciles every lane:
mcptest run examples/full-audit/full-audit.ymlThe standalone subcommands still exist for a focused run of one concern. The lanes only fire when the suite declares their block, so a suite that declares none behaves exactly as before.