mcptest docs GitHub

Project layout

This page is the canonical map of the mcptest workspace: which crate owns what, how they depend on each other, how the language SDKs are laid out, and how to build the project locally. The repo README.md is the one-page overview; this page is the depth.

The Cargo audit at architecture/workspace-layout.md carries the published-vs-internal boundary and the historical record of which crates split off when. Treat that as the source of truth when the two pages disagree; this page is the human-facing index.

Workspace

Top-level layout:

mcptest/
  AGENTS.md            # contributor rulebook (CLAUDE.md is a symlink)
  Cargo.toml           # workspace manifest
  README.md            # one-page user intro
  rust-toolchain.toml  # pinned stable channel
  rustfmt.toml         # format rules
  clippy.toml          # lint thresholds
  schemas/             # JSON Schema for the YAML config + cassette format
  scripts/             # check.sh gate, helper scripts
  docs/                # this directory
  crates/              # Rust workspace members (below)
  sdks/                # language SDKs (Python, TypeScript, Go, Rust, .NET, JVM)
  examples/            # runnable YAML test suites
  conformance-corpus/  # vendored MCP SEP corpus (mcptest conformance reads)

Rust crates

Every crate lives under crates/. Public-API crates are published to crates.io; binary-only crates ship via the mcptest artifact.

CratePathRolePublic?
mcptestcrates/mcptest/The CLI binary + a thin library wrapper so integration tests exercise the parser without spawning a subprocess.Published; the binary is the user-facing artifact.
mcptest-corecrates/mcptest-core/Matchers, MCP protocol client, transports, runner, cache eligibility, coverage, reporters, lint, redaction, cassette, compliance, composition DAG, conformance corpus loader. Transport-free except the upload reporter.Published as a library.
mcptest-configcrates/mcptest-config/YAML loader with JSON Schema validation, .env parser, variable resolution, ${VAR} interpolation.Published as a library.
mcptest-agentcrates/mcptest-agent/Agent test driver, conversation trace, cassette format.Published as a library.
mcptest-scorer-corecrates/mcptest-scorer-core/Scorer framework: the ScorerAdapter trait, registry, and the exec external-scorer substrate.Published as a library.
mcptest-mcp-servercrates/mcptest-mcp-server/Library that drives the mcptest mcp-server subcommand (stdio MCP server exposing the engine to local agents).Published as a library.

Dependency graph:

mcptest (bin + lib)
  -> mcptest-core
  -> mcptest-config
       -> mcptest-core
  -> mcptest-agent
       -> mcptest-core
  -> mcptest-scorer-core
  -> mcptest-mcp-server
       -> mcptest-core
mcptest-core
  -> (no internal deps)

mcptest-core has no internal dependencies on purpose: every other crate composes on top of it, and a future SDK FFI host can pull just the core matchers without committing to YAML loading or the CLI.

Language SDKs

SDKPathDistributionTest integration
Pythonsdks/python/pip install mcptestpytest plugin
TypeScriptsdks/typescript/@mcptest/sdk on npmvitest, jest, mocha, node:test adapters
Gosdks/go/go get github.com/soapbucket/mcptest/sdks/got.Run subtests
Rustsdks/rust/mcptest-runner crate (proc-macro)emits #[test] items
.NETsdks/dotnet/Mcptest.Sdk NuGetxUnit [ClassData] discoverer
JVMsdks/jvm/com.soapbucket.mcptest:mcptest-junit5 Maven artifactJUnit 5 @TestFactory

Each SDK reuses the same YAML config + cassette schema, so a suite authored against any one of them runs unchanged against the Rust CLI.

Build from source

The workspace builds with cargo against the pinned stable channel in rust-toolchain.toml. No build script tricks, no native deps beyond what rustls already pulls.

# Release binary
cargo build --release
./target/release/mcptest --help

# Run the full local gate (fmt + clippy + doc + build + test). The
# same script CI runs.
./scripts/check.sh

scripts/check.sh is the single source of truth for "is the tree green." It runs:

  1. cargo fmt --all -- --check
  2. ./scripts/check-module-size.sh (production-module size guard)
  3. The em-dash lint (AGENTS.md hard rule)
  4. cargo clippy --workspace --all-targets --all-features -- -D warnings
  5. RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps
  6. cargo build --workspace --all-targets
  7. cargo test --workspace

Any step that fails fails the gate. A ticket is not Done until the gate is green locally.

Documentation

Markdown docs live under docs/. The published site at https://mcptest.sh/docs/ builds from this directory via mdbook. Headers in docs/SUMMARY.md drive the sidebar; new docs land there to be linkable.

# Locally preview the documentation site.
mdbook serve docs/
# Or just read raw Markdown.

The doc gate (cargo doc --workspace --no-deps -D warnings) catches broken intra-doc links + missing pub doc comments. Every public Rust item carries a /// comment; cargo doc enforces it.

Conformance corpus

crates/mcptest-core/seps/ holds the vendored MCP SEP corpus mcptest conformance reads. The corpus is also baked into the binary at compile time via include_dir! so a cargo install mcptest user gets the same SEPs the source clone has. Maintainers refresh the in-repo copy via scripts/refresh-conformance-corpus.sh.

Where to file changes

ChangeLand in
Matcher logic, transport, runner internalsmcptest-core
YAML field, .env parsing, interpolationmcptest-config
New CLI subcommand or flagmcptest (cli/args/, cli/handlers/)
Documentationdocs/ (and re-link from SUMMARY.md)
User-facing examplesexamples/ (and re-link from examples/README.md)
JSON Schema fieldschemas/v1.json (with a docstring; reviewer checks IDE autocomplete works)

The contributor rulebook in ../AGENTS.md carries the harder rules (no em-dashes, 850-line module cap, all code has tests, the check gate before "done").