The suite file

A mcptest.yaml suite is the contract for your MCP server. It names the servers under test, lists the checks to run against them, and says what a correct result looks like. mcptest run reads the suite, executes every check, and folds the results into one report and one exit code.

This page is the on-ramp. It walks through the shape of a suite once, end to end. For the full field list (every block, every option) see the YAML test format reference.

A minimal suite

# yaml-language-server: $schema=https://mcptest.sh/schema/v1.json

servers:
  api:
    url: https://test.mcptest.sh/mcp

tools:
  - name: greet returns a greeting
    server: api
    tool: greet
    args:
      name: mcptest
    expect:
      - target: result.content[0].text
        matcher:
          exact: "Hello, mcptest!"

Run it:

mcptest run mcptest.yaml

That is a complete, runnable suite. The first line tells your editor where the schema lives so it can complete and validate fields as you type. The rest is three parts: the servers, the tests, and the matchers.

Declaring servers

The servers: block is an object map, one entry per server. The key is a name you choose; the value says how to reach it. A server is either a subprocess (command:) or a URL (url:):

servers:
  local:
    command: ["node", "./my-server.js"]
  hosted:
    url: https://test.mcptest.sh/mcp

Every test names which server it runs against with a server: field, even when the suite declares only one. That is what lets a single suite test several servers at once. See Multi-server suites for the multi-server story.

Declaring tests

A suite tests three MCP primitives, each in its own block:

tools: calls a tool and asserts on the result.
resources: reads a resource and asserts on its contents.
prompts: renders a prompt and asserts on the messages.

Each entry has a name: (what the report prints), the server: it runs against, the thing to invoke, and an expect: list of assertions:

tools:
  - name: search finds the package
    server: api
    tool: search
    args:
      query: mcptest
    expect:
      - target: result.content[0].text
        matcher:
          contains: mcptest

resources:
  - name: readme resource is markdown
    server: api
    resource: "docs://readme"
    expect:
      - target: result.contents[0].mimeType
        matcher:
          exact: "text/markdown"

prompts:
  - name: summary prompt has a system message
    server: api
    prompt: summarize
    args:
      topic: deployment
    expect:
      - target: result.messages[0].role
        matcher:
          exact: system

Asserting with matchers

An expect: entry has two parts. The target: is a dotted path into the result, and the matcher: says what the value at that path must satisfy. The common matchers:

Matcher	Passes when
`exact`	the value equals the given value exactly
`contains`	a string value contains the given substring
`regex`	a string value matches the given pattern
`schema`	the value validates against the given JSON Schema

expect:
  - target: result.isError
    matcher:
      exact: false
  - target: result.content[0].text
    matcher:
      regex: "^Hello"

A test passes when every assertion passes. The full matcher set (numeric comparisons, set membership, embedding similarity, model-graded judgment, and the weighted-score model) is in the matchers reference.

Where to go next

This page covers the spine. From here:

YAML test format reference: every block and field in one place.
Multi-server suites: name several servers and route tests across them.
Compositions and pipelines: chain tool calls into a DAG when one tool's output feeds the next.
Fixtures and isolation: seed and tear down state around a run.