Production delivery and rollback

Fast publishing needs a fast way back. If a new banner breaks a page, returning to a known-good release should take a channel update, not a rebuild or a GitHub repair.

This is the proposed hosted delivery architecture for the next major. The roadmap tracks its implementation. The server-side loader is described in How hosted releases stay current.

Two separate paths

The control plane owns editing, GitHub access, release preparation and promotion. The data plane serves already prepared content. Production servers never call GitHub, including on a cache miss, a cold start or an error.

R2 with Cloudflare's CDN, or S3 with CloudFront, can implement this shape. Provider details stay behind a stable, versioned asset contract. An optional small authorization gateway checks site credentials before serving private assets; it reads storage only, has no GitHub credentials, and deploys independently of the editing API. A content hash identifies bytes; it is not permission to read them. Private responses must not enter a shared cache before authorization.

Immutable snapshots and mutable channels

A revision is the hash of the content map, computed by the same canonical serialization in the CLI, ingestion worker and SDK. Define and version that serialization in shared code; hashing raw files or insertion-order-dependent maps would make identical content produce different revisions. Store each full snapshot under its revision and never overwrite it.

The delivery contract has two resources, scoped to a site:

/sites/acme/revisions/<revision>.json
/sites/acme/channels/production.json

A snapshot is the SDK's existing { revision, blocks } shape. The proposed channel manifest is small:

{
  "format": 1,
  "generation": 184,
  "revision": "sha256-opaque-content-hash",
  "snapshot": "/sites/acme/revisions/sha256-opaque-content-hash.json"
}

The snapshot path stays on the configured delivery origin and within the site namespace. The SDK refuses unknown formats and unexpected paths. Other environments get their own channels. The manifest's generation orders promotions; the revision identifies content. Neither a content hash nor a Git commit hash expresses publication order.

Give immutable assets a long cache lifetime. Give channel manifests an explicit, short cache lifetime and an ETag. Configure caching for JSON explicitly, and avoid caching missing revision assets. Object-store consistency does not make CDN caches immediately current. Include the manifest cache lifetime, the SDK's poll interval and any rendered-page cache in the propagation budget.

Prepare before promoting

1. Accept and authenticate a GitHub push event, durably enqueue it and deduplicate deliveries. Also reconcile repository heads periodically: an event can be missed.
2. Resolve the production commit and assemble its content. Preparation doesn't check content: route conflicts and fit with the schema are deco check's job in CI, and ingestion does not run site code.
3. Upload the snapshot, verify its bytes and record the release as ready. Preserve its source commit in release history.
4. Only then update the channel manifest. A failed preparation leaves the previous release live; it never points readers at an unfinished object.

Use one durable promotion coordinator per site and channel. Every publish and rollback goes through it, with an expected generation. Jobs carry the generation against which they were requested; an older job cannot overwrite a newer promotion. Serialize the actual manifest writes too: checking a database generation and then independently uploading a manifest is not an atomic operation. Recover an interrupted write by reconciling the manifest against the coordinator's durable desired state.

GitHub push events may arrive out of order. Reconcile the production head in the control plane rather than treating arrival order as commit order. No reader asks GitHub to rebuild a missing snapshot. A missing or unavailable asset is an error, and the SDK retains its last good content or build fallback.

Expose preparation and promotion as separate statuses. "Saved" means committed; "published" means promoted for delivery. Neither means every server has refreshed yet.

Fast rollback

Rollback selects a retained, previously prepared revision that fits the deployed code. The coordinator advances the generation while pointing at the older immutable asset:

publish:  generation 184 → revision B
rollback: generation 185 → revision A

No snapshot rebuild, application deploy or GitHub API request is required for the delivery change. Record who requested it, why, the previous revision and the target. Keep snapshots and their referenced uploads for the documented rollback window; garbage collection must preserve channel targets, pinned revisions and active preview grants.

An older in-flight job cannot undo generation 185. Rollback also places the channel in an explicit hold: new Git commits may prepare releases, but automatic promotion stays paused until an operator resumes it or deliberately publishes. Otherwise a webhook replay or reconciliation could immediately restore the release being rolled back.

The repository still contains its current production content. Offer a separate Git revert to reconcile it later; that operation can wait for GitHub to recover. Draft synchronization continues against repository production, and the editor must show when delivered content differs from Git. A code rollback is separate and must select content compatible with the older build.

Rollback uses the same propagation path as publishing. An active process sees it on its next background check plus the manifest cache delay; an idle Worker checks after a subsequent response. A cold process first serves its build fallback. There is no promise that every visitor changes at once. Notify the rendered-page caching integration on a generation change, including when the content revision is older, so HTML caches cannot hide the rollback indefinitely.

Exact draft previews

A draft is delivered as an immutable overlay, not a complete release snapshot. Its version fixes the replacements and deletions; it does not fix the inherited production content. There is no baseRevision in the draft asset or pointer. The base is whatever production snapshot the server already has when this client first loads. There is no separate kind of preview server: a server renders a draft the same way it renders production, with the draft's overlay on top.

{
  "format": 1,
  "set": { "HomeHero": "sha256-changed-block-hash" },
  "delete": ["OldPromotion"]
}

Store the manifest at /sites/<site>/drafts/<overlay-version>.json and changed JSON blocks at /sites/<site>/draft-blocks/<block-hash>.json. Each manifest contains the complete cumulative set of draft overrides and tombstones, not a patch that requires replaying earlier draft versions. Generate disjoint set and delete sets. The overlay version hashes the canonical manifest; block hashes identify the canonical changed JSON. Production releases still use complete { revision, blocks } assets.

The SDK downloads the small manifest and only referenced block blobs missing from its bounded, site-scoped cache. A new save can reuse previously downloaded blobs. It never fetches a production revision just to align a draft, and never crawls GitHub. A cold server uses its normal production fallback, loading that fallback by its existing mechanism if necessary; there is no extra draft base fetch or release-refresh barrier.

Capture that local production snapshot once for each draft client. Look up draft replacements first, treat tombstones as absent, and inherit all other blocks from the captured production map. Enumeration unions the names and filters deletions. Use a read-through view or persistent map sharing unchanged objects instead of deep-copying the full release. Do not mutate the production snapshot when applying an overlay or resolving its entries.

A subsequent request may inherit a newer local release, and another server may inherit a different one. That is intentional: a draft link identifies exact draft changes, not a reproducible full-state preview or a promise to match publication exactly. Rollback of production also changes inherited preview content on later requests. In-flight clients keep their captured base and overlay.

Cache overlay assets by site and overlay version, and changed blobs by site and block hash. If caching composed views or rendered previews, key them by both the captured local production revision and overlay version, plus the normal access/request scope. An overlay version alone cannot identify the effective content. The draft client's opaque revision identifies this pair; it is not the release's full-map hash. Do not hash or serialize the entire merged map simply to construct that identity.

A signed grant scopes access to site, draft overlay version and expiry. Enforce the expiry on every read, including warm servers: an authorized manifest response is cacheable only privately and only until its grant expires (Cache-Control: private, max-age=<seconds left>), and the SDK keeps an authorized manifest no longer than that; a composed draft is reused for at most a minute before that check runs again. Changed blobs stay immutable; each read of one is authorized through a live manifest. Authorize changed-blob reads against membership in that authorized manifest; knowing a block hash alone grants no access. Never substitute newer branch changes for an older overlay version. Missing, invalid or unauthorized overlay assets fail the draft client as documented; they do not silently display published content.

Saving and preview readiness are separate: a save is committed before its overlay and referenced blobs necessarily exist. Upload changed blobs first, then the immutable manifest; show "preparing preview" until both are ready. Control-plane preparation uses saved write bodies and cached Git blob IDs, reads bodies only for changed files, and does not reconstruct a complete draft snapshot. The production commit incorporated by draft synchronization remains internal Git metadata, not a preview base constraint. The same file-level rule applies during synchronization and publication: an edited draft file wins in full; production changes in that same file are intentionally not merged by property. Untouched files inherit production.

Publishing, rollback, preview grants and draft lifecycle remain control-plane routes outside the content protocol. Publishing prepares a complete production snapshot; draft overlays do not change release or rollback semantics.

Bound preparation memory

Build assets in a worker with bounded concurrency and a memory budget, independently of the editing API. Stream downloads and snapshot output to temporary storage or the object store instead of holding every file body and the final serialized map simultaneously. Enforce per-entry and aggregate limits during reads, including for existing repository files; compressed byte size is not a bound on parsed JSON memory.

The SDK still needs a full parsed snapshot for synchronous resolution. Bound accepted snapshot sizes separately, retain only a bounded set of versions, and account for the old revision, new revision, fallback and in-flight readers during a swap. Streaming preparation does not remove that runtime limit.

Studio implementation

The Studio implementation handoff maps this design to existing Fast Preview modules and specifies persisted state, operation contracts, sequencing, initial limits, recovery scenarios and rollout phases.