2026 OpenClaw Cluster
Multi-AZ Gateways, Probes & Webhook Digests

Why monolithic gateways fight multi-AZ reality

This HowTo assumes three or more dedicated Mac nodes on clustervps: two regional gateways plus at least one worker or notifier. When every automation path pins to the same host, you inherit correlated outages, noisy paging, and brittle token stories that are hard to rehearse.

• Thundering herds: Webhook providers retry in parallel and your gateway drops TLS before OpenClaw can dequeue work.
• Probe drift: Each load balancer invents its own curl script, so half the fleet thinks it is healthy while agents cannot reach upstream APIs.
• Secret coupling: One long-lived bearer token on a shared keychain means rotation becomes a weekend event instead of a fifteen minute drill.

Split versus single gateway: quick matrix

Pair the matrix with the Mac plan catalog so each AZ owns predictable RAM and SSD headroom. You are not buying vanity redundancy—you are buying the ability to drain one node while collaborators stay on the public homepage flows without guessing passwords.

HowTo: split clustervps gateways per availability zone

1. Freeze scope. List every inbound hostname, mTLS profile, and static IP that OpenClaw terminates today. Nothing moves until the inventory matches DNS.
2. Clone configuration, not state. Copy launch daemons and environment files to the second Mac, but keep SQLite or local queues empty until cutover.
3. Shift DNS weights. Start with ten percent traffic on the new gateway, watch error budgets for thirty minutes, then ramp in twenty percent steps.
4. Document drain switches. Each operator should know the single command that marks a gateway read-only without touching workers.
5. Run paired chaos drills. Stop one gateway service during business hours while the other holds webhook backlog—no surprises during real incidents.

Keep SSH bastion paths identical across nodes so your help center screenshots stay truthful. If you need another bare-metal seat, the purchase form stays reachable without forcing console logins first.

HowTo: merge health probes into one honest signal

Load balancers should call exactly one composite endpoint per gateway. Inside that handler, chain lightweight checks: disk pressure, launchd job heartbeats, outbound TLS to your webhook partner, and certificate expiry. Return JSON with separate booleans so operators can see which sub-system failed without opening five tabs.

#!/usr/bin/env bash
set -euo pipefail
/usr/bin/curl -fsS --max-time 4 https://hooks.partner.test/ping >/dev/null
/usr/sbin/diskutil apfs list | /usr/bin/grep -q "Container"
/usr/bin/printf '{"disk":true,"webhook":true,"queue_depth":0}\n'

Publish the same probe path on every gateway so automation from other Mac nodes can curl peer hosts during maintenance. When a probe fails, attach the gateway AZ label so downstream OpenClaw schedulers can migrate sessions quickly.

HowTo: broadcast webhook failure digests, not noise

Assign a notifier node whose only job is to aggregate failures. Gateways should emit structured events to a local Redis or file tail; the notifier batches five minute windows, deduplicates HTTP status codes, and pushes a single Slack or email summary. Collaborators on other Mac workers subscribe to that digest channel instead of watching raw logs.

• Envelope metadata: Include correlation IDs, AZ, and retry count so on-call engineers replay precisely one failed delivery.
• Back-pressure: If digest publishing itself fails, fall back to a voicemail-style counter on disk so nothing silently disappears.
• Human language: Summaries should read like airline status boards—short clauses, no stack traces unless severity is critical.

HowTo: rotate tokens with rolling validation

Mint shadow credentials in your secret store, inject them alongside legacy tokens on a canary gateway, and validate outbound webhooks for one full business day. Promote the shadow secret to every gateway using a configuration revision tag, then revoke the old token only after success metrics stay flat for two polling intervals.

Workers that call the gateway should read tokens from a short-lived file that updates atomically via rename, preventing half-written secrets during rotation. Log every rotation event with actor and ticket ID so security reviews stay painless.

Schedule a calendar reminder every ninety days, but shorten the interval whenever webhooks traverse the public internet or whenever a vendor publishes a cross-tenant incident. Pair reminders with a lightweight spreadsheet that tracks which Mac node last confirmed success so nobody assumes a silent rotation succeeded.

Multi-node collaboration checklist

• Runbook parity: Both gateways share the same systemd or launchd unit names.
• Time sync: sntp drift under two seconds across all Mac nodes.
• Webhook replay queue: Bounded disk usage with explicit high-water alerts.
• Probe SLO: Composite endpoint answers in under two hundred milliseconds at p95.
• Digest latency: Failure summaries arrive before executive escalations.
• Token overlap: At least twelve hours where old and new secrets both work.
• Operator SSH: Jump paths documented beside help articles.
• Capacity review: Quarterly revisit of plans after traffic growth.

FAQ: keeping OpenClaw boring on clustervps

Do I need separate TLS certificates per AZ? Yes, terminate locally so users stay on-region paths, but keep issuance automated from the same CA profile so renewal playbooks stay identical.

What if merged probes hide partial failures? Surface degraded mode: HTTP 200 with "status":"degraded" triggers yellow dashboards while traffic still flows.

Can interns rehearse rotation? Absolutely—shadow tokens plus canary gateways make drills safe, and every participant should practice without touching production queues.