Release process

This document explains how to cut a new release of mcptest. The heavy lifting (cross-compiling for five targets, signing, packaging, attaching artifacts to a GitHub Release) is handled by .github/workflows/release.yml on push of a v*.*.* tag. Your job locally is to bump versions, update the changelog, tag, and push.

Targets we ship

Each artifact is a stripped release binary packaged with LICENSE, NOTICE, and README.md. Linux and macOS ship .tar.gz. Windows ships .zip. Every archive has a companion <name>.sha256 file.

Code signing

The workflow signs macOS binaries with codesign plus notarization, and Windows binaries with signtool. Signing is gated on the presence of repo secrets so the workflow keeps working in forks and on contributor branches:

• macOS skips signing when any of APPLE_DEVELOPER_ID_CERT, APPLE_DEVELOPER_ID_SIGNING_IDENTITY, APPLE_DEVELOPER_ID_APP_PASSWORD, APPLE_DEVELOPER_ID_TEAM_ID, or APPLE_DEVELOPER_ID_APPLE_ID is absent.
• Windows skips signing when WINDOWS_PFX_BASE64 is absent.

When signing is skipped the workflow prints a ::warning:: and continues. The unsigned artifact is still uploaded and still has a SHA256 sidecar.

Cutting a release

The workflow is driven by cargo-release and git-cliff. Both are dev tools, installed via cargo install, not workspace dependencies. Local configs live at release.toml (cargo-release) and cliff.toml (git-cliff).

One-time setup (per machine)

cargo install cargo-release --locked --version ^0.25
cargo install git-cliff --locked --version ^2

Routine minor release (0.1.0 to 0.2.0)

1. Verify the gate passes. Releases never ship with a red gate:

   ./scripts/check.sh

2. Regenerate CHANGELOG.md from Conventional Commits. git-cliff walks the commit log since the last tag and writes grouped bullets under feat, fix, perf, docs, ci, chore, revert, and security headings:

   git cliff --unreleased --prepend CHANGELOG.md

   Inspect the diff. Hand-edit any bullets that read poorly.

3. Bump the workspace version with cargo-release. This updates Cargo.toml, refreshes Cargo.lock, and creates a release commit locally. The --no-publish --no-push --no-tag flags keep the run side-effect-free so you can inspect the result:

   cargo release minor --workspace --no-publish --no-push --no-tag

   Review the commit, then add the tag:

   git tag v0.2.0

4. Push the commit and tag. The release workflow fires on the tag, not the branch, but both should ship together:

   git push origin main
   git push origin v0.2.0

5. Watch the run. Two workflows trigger off the tag:

   • release.yml: cross-compiles for five targets, signs platform-conditionally, and creates the GitHub Release with git-cliff release notes attached.
   • publish-crates.yml: publishes mcptest-config, then mcptest-core, then mcptest to crates.io via the Trusted Publishers OIDC flow, then runs cargo install mcptest --version <tag> --locked to verify the binary lands cleanly.

Pre-release (release candidate)

Use cargo release rc for tags like v1.0.0-rc.1. Pre-release tags route to the GitHub pre-release channel automatically (the publish-release job in release.yml sets prerelease: true when the tag matches *-rc.*, *-alpha.*, or *-beta.*):

cargo release rc --workspace --no-publish --no-push --no-tag
git tag v1.0.0-rc.1
git push origin main
git push origin v1.0.0-rc.1

The publish-crates.yml workflow also publishes pre-release versions to crates.io, where they install with cargo install mcptest --version 1.0.0-rc.1. Users on the stable cargo install mcptest path never see them, because cargo defers to the latest non-pre-release by default.

Promoting an RC to a stable release

When v1.0.0-rc.N is ready to promote, repeat the routine minor flow with the target stable version:

cargo release 1.0.0 --workspace --no-publish --no-push --no-tag
git tag v1.0.0
git push origin main
git push origin v1.0.0

Patch release (bugfix)

cargo release patch --workspace --no-publish --no-push --no-tag
git tag v0.2.1
git push origin main
git push origin v0.2.1

Trusted Publisher setup (one-time, per crate)

The publish-crates.yml workflow uses crates.io's Trusted Publishers OIDC flow, so we ship no long-lived CARGO_REGISTRY_TOKEN in repo secrets. The exchange is brokered by rust-lang/crates-io-auth-action@v1. Configure once per crate on the crates.io side:

1. Sign in to crates.io as a crate owner.
2. Open the crate's settings page (Manage -> Trusted Publishers).

3. Add a GitHub Actions Trusted Publisher with:

   • Repository owner: soapbucket
   • Repository name: mcptest
   • Workflow filename: publish-crates.yml
   • Environment: (leave blank; we do not use protected environments)
4. Repeat for mcptest-core, mcptest-config, and mcptest.

After registration, every tag push that triggers publish-crates.yml can mint a short-lived token without any secret in the repo. The token expires when the job ends.

If a crate is brand new (never published), the first publish still requires a manual upload by an owner so crates.io knows who you are. Reserve the name first with cargo publish --dry-run to confirm metadata is valid, then do the initial cargo publish locally with a personal API token. Register the Trusted Publisher immediately after.

Local sanity check before tagging

Run the gate locally so CI does not catch a regression after the tag is already public:

./scripts/check.sh

If you want a dry run of the cross-compile step for Linux arm64:

cargo install cross --locked
cross build --release --target aarch64-unknown-linux-gnu -p mcptest

Pinning GitHub Actions to commit SHAs

Third-party actions used in .github/workflows/*.yml should be referenced by full 40-character commit SHA, not by tag or branch. Tags can be re-pointed by the action author at any time, so a pin like @v4 is not reproducible. A pin like @a1b2c3d4...0123 # v4.2.2 is.

Until the first release we annotate every uses: line with a TODO: replace tag with commit SHA before first release comment. Resolve them with the following procedure for each uses: owner/repo@tag:

1. Find the tag's commit SHA via the GitHub API:

   gh api repos/<owner>/<repo>/git/refs/tags/<tag> \
     --jq '.object.sha, .object.type'

   If object.type is tag (an annotated tag) follow the indirection:

   gh api repos/<owner>/<repo>/git/tags/<sha-from-step-1> \
     --jq '.object.sha'

   The resulting 40-character SHA is the pin target.

2. Rewrite the workflow line:

   - uses: actions/checkout@a1b2c3d4e5f60718293a4b5c6d7e8f9012345678  # v4.2.2

3. Drop the TODO comment from that line.

Dependabot then keeps the SHAs current. The github-actions ecosystem entry in .github/dependabot.yml opens weekly PRs that bump the SHA and update the trailing # vX.Y.Z comment, so reviewers can verify each bump in the diff.

Release notes

The publish-release job in release.yml runs git cliff --current --strip header and pipes the result into the GitHub Release body. The generator config lives at cliff.toml at the repo root. Commits not matching a Conventional Commit prefix (feat:, fix:, perf:, docs:, ci:, chore:, revert:, security:) are dropped. Keep commit messages tidy and the release notes will follow.

download.mcptest.sh mirror

download.mcptest.sh is the public mirror that scripts/install.sh reads from. It serves three classes of file, all keyed off the release tag:

The release workflow attaches everything to the GitHub Release. The mirror sync from GitHub Releases to download.mcptest.sh is one step removed from this repo (it lives in the soapbucket-infra setup that owns the CDN). The expectation is: every tagged release becomes reachable at download.mcptest.sh/<tag>/ within a few minutes of the GitHub Release being published. If the mirror lags, contributors can still install from the GitHub Release URL directly.

The install.sh byte-served at the root of download.mcptest.sh is identical to the file checked in at scripts/install.sh. A reader can audit either before piping into a shell.

Schema publishing

The .github/workflows/publish-schema.yml workflow ships every public JSON Schema under schemas/ to mcptest.sh/schema/* via GitHub Pages. It runs on every push to main and on every release tag (v*.*.*), so a tagged release always re-publishes the schema in lockstep with the binary.

What it publishes:

Aliases:

• https://mcptest.sh/schema/latest.json returns a 301 redirect to v1.json. The redirect is configured in schemas/_headers style and applied by Cloudflare Pages once we migrate; GitHub Pages serves the file unchanged for now (no harm, no effect).

Cache headers (Cloudflare Pages format, see schemas/_headers):

• Cache-Control: public, max-age=3600. One hour is short enough that a schema fix reaches IDEs the same afternoon and long enough that yaml-language-server polling does not hammer the CDN.
• Content-Type: application/schema+json per the IANA registration.
• Access-Control-Allow-Origin: * so browser-based schema explorers can fetch the file cross-origin.

Validation: the workflow runs python3 -c "import json; json.load(...)" on each schema before deploy as a syntax sanity check. The runtime loader (the jsonschema crate) is the source of truth for structural correctness.

Smoke test: after deploy the workflow curls each published URL and pipes the body through jq -e '."$id"'. A missing $id fails the run, so a misconfigured deploy never goes silent.

When you ship a new schema version (for example schemas/v2.json), extend the workflow's "Stage publish tree" step to copy the new file and add a smoke-test line for its URL. Do not delete the previous version: old YAML configs still reference the old $id.