mcptest docs GitHub

Scenario 3: snapshot tests

The snapshot matcher records a tool's response on the first run and asserts equality against the recorded copy on every subsequent run. It is the right tool for "the response should look like this" when the response is too large or too nested to inline in YAML.

The challenge with snapshots in any test runner is volatile fields: timestamps, UUIDs, generated IDs, retry counters. A naive snapshot fails on the second run because one field changed. mcptest handles this for you: the snapshot cassette is normalized when it is recorded, so you do not hand-list the volatile fields.

The YAML

Save this as tests/snapshots.yml:

# yaml-language-server: $schema=https://mcptest.sh/schema/v1.json

servers:
  local:
    command: ["./target/debug/my-mcp-server"]

tools:
  - name: "lookup returns a stable shape"
    server: local
    tool: "lookup_record"
    args:
      id: "rec_abc123"
    expect:
      - target: "result"
        matcher:
          snapshot: "lookup-stable-shape"

The snapshot matcher takes a string: the snapshot file basename. The recording lives next to the YAML by convention. The first run writes it; every subsequent run reads it and compares.

You do not list volatile fields by hand. When mcptest records a snapshot it runs a normalization pass that rewrites the values that change run to run: timestamps, UUIDs, request IDs, IP addresses, and epoch seconds. A recording from today and one from next week diff cleanly because the normalized values match. See docs/cassettes.md for the normalization details.

Workflow

# first run: writes the snapshot file
mcptest run tests/snapshots.yml

# review the recorded snapshot, then commit it
git add tests/snapshots/lookup-stable-shape.json
git commit -m "Capture lookup-stable-shape snapshot"

# subsequent runs: assert against the committed file
mcptest run tests/snapshots.yml

If the response shape changes intentionally (you added a field, removed one, restructured a section), regenerate the snapshot:

mcptest run --update-snapshots tests/snapshots.yml

The --update-snapshots flag (short form -u) rewrites every snapshot the run touches. Review the diff carefully before committing; an unexpected change is the whole point of using snapshots.

Expected output

First run:

  PASS  lookup returns a stable shape         (snapshot written)

Subsequent runs:

  PASS  lookup returns a stable shape         (snapshot matched)

On failure:

  FAIL  lookup returns a stable shape         (snapshot mismatch)
    --- expected (committed)
    +++ actual (this run)
    @@ -3,4 +3,4 @@
       "metadata": {
         "version": 2,
    -    "region": "us-east-1"
    +    "region": "eu-west-1"
       },

The diff shows what changed. The volatile fields the normalization pass rewrote (timestamps, UUIDs, request IDs, IPs, epoch seconds) read the same on both sides, so they cannot mask a real change.

When not to use snapshots

Snapshots are great for "the response should look exactly like this" and miserable for "the response should contain this text somewhere." If you only care about a couple of fields, reach for contains or exact against narrowly-targeted JSONPaths instead. Snapshots tend to over-test, which means they fail on legitimate changes you do not actually care about.

A good rule of thumb: if you would not commit the snapshot file to review, do not write it. Use contains or schema instead.

See also