Blog · 8 November 2026 · 8 min read · for developers

Make your coding agent verify every deploy: a CLAUDE.md block, a CI step and a definition of done

A green deploy proves that the files went up. It says nothing about whether the site works for a visitor on a phone, whether Google is allowed to index it, or whether .env went up with the files. Agents are good at finishing the task they were given, and “deploy” ends when the upload ends. This post gives you three pieces to paste so that the task ends one step later: after the live site has been checked. Everything below runs against the public URL only – no SDK, no repository access, and the free plan needs no account.

Why “deployed” is not “done”

The tests ran against a local build. The deploy ran against a server. The visitor and the crawler see a third thing: the live site behind a CDN, with the production config, the production robots file and whatever the hosting panel added on top. The failures that slip through are boring and expensive – and they are the same ones every time: a noindex left over from staging, a title that still says “Vite App”, a form whose fields have no labels, a 404 page with no link out, a cart that cannot reach the checkout, a missing Content-Security-Policy, a .git directory served to the world. The earlier post explains why agent-built sites break in exactly these places. This one is about making the check automatic.

A definition of done for a live site

An agent follows a definition of done far more reliably than a wish like “make sure it works”. The table below is the one we use. Each line maps to finding ids that the Webskan audit returns, so the agent can verify a line by id instead of by opinion. The severity is what the audit assigns; the gate in the CI step below is “no high”.

Page-specific findings carry the page in the id after @, for example ux.form.unlabelled_fields@/contact/ or seo.title.missing@/. That is what makes a re-check exact: the same id either comes back or it does not.

Piece 1: the CLAUDE.md block

Put this in the project's CLAUDE.md (Claude Code), AGENTS.md (Codex and most other agents) or a Cursor rule. It assumes the Webskan MCP server is connected – one command: claude mcp add --transport http webskan https://webskan.com/mcp, or the equivalent config for your client. The block does three things: it moves the finish line after the audit, it tells the agent what to gate on, and it forbids the two mistakes agents make with audits (chasing a score, and re-running in a loop when a limit is hit).

## After every deploy

- When the site is live, run the Webskan audit before reporting the task as done:
  `audit_website` with the production (or public preview) URL and `pages: 10`,
  then `get_audit` every 30 s until `status` is `done`. Public URLs only –
  never localhost, a private IP or a tunnel that needs a header.
- Definition of done: `summary.high` is 0. Fix `medium` findings in the same
  change when they touch files you changed; otherwise list them in the hand-off
  note with their ids.
- Fix by id, not by score. Each finding has `id`, `pages` and `detail`. Map the id
  to the template or config in this repo, change it, and keep a list
  `id -> what changed`. Do not "fix" `perf.*` findings by chasing the Lighthouse
  score; fix the concrete cause named in `detail` (image size, blocking script,
  server response time).
- After the fix is deployed, call `recheck_website` with the first `auditId`,
  poll `get_audit` with the new id, and report `diff.fixed`, `diff.open`,
  `diff.new` and `diff.unverified`. Lighthouse differences of up to 5 points
  are noise – never report them as a fix or a regression.
- Never start a second audit of the same host while one is running. If a limit
  error comes back, say when to retry instead of retrying.
- Put the `auditId` in the hand-off note.

If you install the Claude Code plugin instead, the /webskan:site-check skill carries a longer version of the same rules; the block above is still worth having in the repo, because it is the project saying “this is part of done”, not a tool saying “I exist”.

Piece 2: the CI step (curl and jq, no SDK)

The MCP endpoint is plain JSON-RPC over HTTP, so a CI job does not need an MCP client. The script below starts an audit, polls until it is finished, prints the high and medium findings with their ids, and exits with 1 when there is any high-severity finding (so the job fails) or 2 when the audit itself could not run (unreachable site, limit hit, timeout – which you probably want to see, but not to confuse with a failing site). Lines that start with data: are the server's Streamable HTTP framing; the sed strips it.

#!/usr/bin/env bash
# Post-deploy audit with Webskan over plain HTTP (JSON-RPC).
# Usage: webskan-audit.sh https://example.com [pages]
# Optional: WEBSKAN_KEY (Developer plan key) for CI runners that share IP addresses.
set -euo pipefail
URL="$1"; PAGES="${2:-10}"; MCP="https://webskan.com/mcp"

call() {  # call <tool> <json-arguments>  -> structuredContent of the tool result
  curl -sS "$MCP" ${WEBSKAN_KEY:+-H "Authorization: Bearer $WEBSKAN_KEY"} \
    -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
    -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"$1\",\"arguments\":$2}}" \
    | sed -n 's/^data: //p; /^{/p' | jq -s '.[0].result.structuredContent'
}

START=$(call audit_website "{\"url\":\"$URL\",\"pages\":$PAGES}")
if [ "$(jq -r '.error // empty' <<<"$START")" ]; then
  echo "webskan: $(jq -r .error <<<"$START")" >&2; exit 2
fi
ID=$(jq -r .auditId <<<"$START"); echo "audit $ID queued for $URL ($PAGES pages)"

for _ in $(seq 1 40); do                      # up to 20 minutes
  sleep 30
  RES=$(call get_audit "{\"auditId\":\"$ID\"}")
  STATUS=$(jq -r .status <<<"$RES")
  case "$STATUS" in
    done)  break ;;
    error) echo "webskan: audit failed: $(jq -r '.error // .message // "unknown"' <<<"$RES")" >&2; exit 2 ;;
    *)     echo "  $STATUS (eta $(jq -r '.etaSeconds // "?"' <<<"$RES") s)" ;;
  esac
done
[ "${STATUS:-}" = done ] || { echo "webskan: timed out" >&2; exit 2; }

jq -r '"pages: \(.pagesCrawled)  high: \(.summary.high)  medium: \(.summary.medium)  low: \(.summary.low)  perf(mobile): \(.lighthouse.mobile.scores.performance // "-")"' <<<"$RES"
jq -r '.findings[] | select(.severity=="high" or .severity=="medium") | "[\(.severity)] \(.id): \(.title)"' <<<"$RES"
echo "auditId: $ID (keep it for recheck_website)"
[ "$(jq -r .summary.high <<<"$RES")" = 0 ]

Wire it in after the deploy step. With GitHub Actions:

  audit:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Post-deploy audit (fails on high-severity findings)
        env:
          WEBSKAN_KEY: ${{ secrets.WEBSKAN_KEY }}   # optional, see "Limits"
        run: bash scripts/webskan-audit.sh https://www.example.com 10

The output of a run against a small site looks like this (our own sample site, three pages):

audit c03ce1221fea1e55 queued for https://kamilobstawski.com (3 pages)
  running (eta 48 s)
  running (eta 48 s)
pages: 3  high: 0  medium: 3  low: 5  perf(mobile): 87
[medium] perf.score.mobile: Mobile performance score: 87/100
[medium] perf.tbt.mobile: Page is slow to respond to taps (TBT 497 ms)
[medium] ux.form.unlabelled_fields@/pl/kontakt/: Form fields without labels: /pl/kontakt/
auditId: c03ce1221fea1e55 (keep it for recheck_website)

Three medium findings, no high: the job passes and the ids go into the issue tracker. If you want the diff after the fix as well, store the auditId as a job output and call recheck_website with it in the next run – the result then carries diff.fixed, diff.open, diff.new and diff.unverified, which reads well as a pull-request comment.

What to gate on, and what not to

• Gate on high findings. They are the ones a visitor or a crawler is hitting right now: a page that errors, a site that blocks all robots, a missing title on the home page, mixed content, a certificate about to expire, a broken mobile menu. Zero is a reasonable bar for a production deploy.
• Do not gate on the Lighthouse score. It moves by a few points between runs on the same site, more between environments. Gate on the concrete metric findings (perf.lcp.mobile above 4 s is high; between 2.5 and 4 s it is medium) and let the score be information.
• Treat medium as the sprint's backlog, not the deploy's blocker. Unlabelled form fields and a missing CSP should be fixed, but failing the deploy for them trains people to disable the check.
• Treat unverified honestly. In a re-check, findings from a page that did not respond land in diff.unverified. They are not fixed; they are unknown. Say so in the report.
• Do not loop on limits. If the tool returns isError: true with a limit message, the right move is to wait or to use a key, not to retry every minute. An agent that retries will burn the next window too.

Limits, previews and shared IP addresses

The free plan is counted per IP address and per website host: 2 whole-site audits per IP per 24 hours, 1 audit per host per 24 hours, re-checks included; scan_website is 10 per hour per IP. That is fine for a developer's machine. It is not fine for a hosted CI runner, because hosted runners come out of shared address pools – the two free audits for “your” IP may already have been used by somebody else's job, and a second audit of the same host within 24 hours returns the existing auditId instead of a fresh run. For CI, use a Developer plan key: 30 audits per 24 hours per key and no one-audit-per-host window, passed as a bearer header (the script above picks it up from WEBSKAN_KEY).

The audit runs from our machines, so the URL has to be public. Localhost, private ranges and .local names are rejected; a preview behind a login or a header-authenticated tunnel will be audited as a login page. Deploy a public preview, or audit production immediately after the deploy – which is what the CI job above does.

What we store: audit results for 30 days and scan results for 7, under the auditId. We never see the repository; every change is the agent's to make and to explain.

Sources and tools

• Model Context Protocol, Transports – Streamable HTTP – why a plain curl with Accept: application/json, text/event-stream works and where the data: lines come from
• Claude Code, Manage Claude's memory (CLAUDE.md); AGENTS.md – the equivalent file read by Codex and other agents
• GitHub Docs, Using secrets in GitHub Actions – where WEBSKAN_KEY goes
• Webskan, MCP server for developers (tools, limits, client configs), Audit the site your agent just shipped and Exposed .git, .env, SQL dumps and backups on live sites