mcptest docs GitHub

Auto-stub generation

The mcptest-core crate ships a small library that turns a tools/list response from an MCP server into a starter YAML test file. The flow lives in crates/mcptest-core/src/stub_gen/.

Why

A fresh mcptest project against an unfamiliar server should not start from a blank file. The server already advertises every tool it supports, including each input schema, so the runner can synthesize a useful default: one happy-path call per tool, plus a couple of obvious negative cases. The developer then opens the file and tightens the assertions to match the real behavior.

What gets generated

For every tool in the catalog the library emits a YAML document containing:

Sample values follow a fixed table:

JSON Schema typeHappy-path valueType-mismatch value
string"sample"12345
number1"not-a-number"
integer1"not-a-number"
booleantrue"true"
array[]"not-an-array"
object{}[]
enumfirst value(uses base type)

When a property has no type field the generator falls back to "sample".

Programmatic use

use mcptest_core::stub_gen::{generate, ToolStub};
use serde_json::json;

let catalog = json!({
    "tools": [{
        "name": "echo",
        "inputSchema": {
            "type": "object",
            "properties": {"msg": {"type": "string"}},
            "required": ["msg"]
        }
    }]
});

let stubs: Vec<ToolStub> = generate(&catalog).expect("well-formed catalog");
for stub in stubs {
    std::fs::write(format!("tests/{}.yaml", stub.tool_name), stub.yaml).unwrap();
}

generate returns Err(StubGenError::NoToolsArray) when the catalog has no tools array, and Err(StubGenError::MissingName) when a tool entry has no name. A well-formed empty catalog ({"tools": []}) returns an empty Vec, not an error.

Future CLI integration

The runner-aware command mcptest init --auto-stubs will compose this library with the live transport: connect to the server, fetch tools/list, hand the JSON to generate, and write each ToolStub to disk under a per-tool file. That wiring is deliberately out of scope and is planned for a future release so the library can be reviewed and tested without an async runtime in the loop.

Edits to the generated YAML are expected and encouraged. The generator's job is to give the developer a running start, not a final test suite.