Ideas

Deploy the Set, Not the Module: Review Before I Build

A federated frontend platform where the unit of deployment is the whole validated set of remotes rather than the one that changed — the ledger of what that buys and what it costs, published while changing my mind is still cheap.

By Vorpalwolf

Design

micro-frontends · deployment · rollback · platform engineering

The deployable unit is the whole validated setA directed graph. On the left, five inputs — a host version, three remotes named at content hashes, and a singleton version — all converge on a single node labelled manifest, one immutable set generated on merge. An arrow leads from that node to the application users open. Along the bottom, four previous sets sit in a row, all still addressable; an orange pointer arrow indexes the newest of them, and a dashed arc shows that rolling back moves that one pointer to an earlier set rather than rebuilding anything.host 4.2.0checkout @9f2c1asearch @1ab7d0profile @c40e83react 18.3.1 · singletonMANIFESTOne immutable setone app, one environmentgenerated on mergenever hand-writtenApplicationwhat users openEvery previous set stays addressableset 3f80set b214set 55acset 7c1aPOINTERrollback moves the pointer, nothing rebuilds

What I'm about to build, and the bet underneath it

I am building a platform for federated micro-frontends — one web application assembled at runtime out of pieces that several teams deploy separately — and I have made one decision that every other decision hangs off. The thing that gets deployed is the whole validated set of pieces, not the piece that changed. Every merge generates an immutable list naming the exact version of every piece; that list goes through a canary; production advances by moving one pointer to it. No piece is ever published straight into production on its own.

Nothing here is built yet. That is the reason I am publishing it now rather than in six months: the ledger below has real costs on it, some of which I cannot price without operating the thing, and the cheapest time to find out I am wrong is before the artifact catalog exists.

The bet is falsifiable in one line. If a fleet running the ordinary way — each module published to its own fixed URL, picked up by whoever loads next — does not accumulate incidents from module combinations that were never tested together, then I am wrong and everything below is overhead. I would like to know that from someone who has run one.

The words in that claim

Skip this if you already speak Module Federation. Everything in this essay is built out of ten terms.

TermWhat it actually means
HostThe application a user opens. A normal web app, except some of its screens are not in its own code.
Remote (or module)A chunk of UI built and deployed by a different team, which the host downloads and runs while the page is already open.
Module FederationThe build-tool feature that makes that possible: one app imports code from another app at runtime instead of at build time.
SingletonA library that must exist exactly once on the page. React is the usual one — two copies of React in a page and hooks break.
Content hashA short string derived from a file's bytes. Same bytes, same hash; one byte different, different hash. It names one exact version of a file, permanently.
ManifestA list naming the host version and the content hash of every remote it should load. The subject of this essay.
CanarySending a slice of live traffic to a new version and comparing its error rate against the old version's at the same moment.
Blue/greenRunning two complete versions side by side and moving traffic between them, rather than upgrading in place.
Template and builderThe boilerplate a new module starts from, and the pipeline that builds it. Change the template, and the builder rebuilds.
ReconciliationSweeping every existing artifact in the fleet to bring it up to the current template and builder.

The platform has five kinds of artifact: libraries (ordinary packages, any language), modules (UI remotes, served from a CDN), applications (the hosts users open), services (backend functions), and templates (the boilerplate the other four are scaffolded from). Modules, applications and services each have a builder behind them, and a builder change reconciles every artifact of that type.

Figure 1

Five artifact types, three builders, and one runtime relationshipThree templates feed three builders, which build modules, applications and services. A library card sits apart with no builder above it and no deployment target below it, annotated as upgraded by hand. Modules deploy to a CDN, applications to static hosting, services to Lambda or Kubernetes. One highlighted arrow runs from the module card into the application card, labelled "loaded at runtime". A dashed line beneath the artifact row is labelled as reconciliation sweeping every artifact of that type.TemplatesBuildersa template changefires its builderArtifactsmodule templatemodule builderapp templateapp builderservice templateservice builderLibraryany languageModuleremoteApplicationhostServicebackendno builder;upgraded by handloaded at runtimereconciliation sweeps every artifact of that typeDeployment targetsCDN / object storagestatic hostingLambda or K8s
Five artifact types and the machinery behind them. Three have builders; a library has none and is upgraded by hand. The one relationship the rest of the essay is about is a module being loaded into a host at runtime.

Why I'd be reasonable to do the opposite

The ordinary setup is one URL per module per environment — https://cdn.example.com/prod/checkout/remoteEntry.js. Merge to production, overwrite that file, and every host that loads afterwards picks up the new code. No host rebuild. No coordination. No release train.

That is not laziness. It is the entire product. The reason a team accepts the micro-frontend tax — runtime loading, version skew, a shared-dependency configuration nobody enjoys, an extra class of deployment artifact — is to buy one property: team A ships when team A is ready. A mutable pointer is the shortest path to that property. Publish the file and you are live.

Four things it genuinely solves:

  • The host never rebuilds. If a remote change forces a host rebuild, you have a monolith with extra network hops and worse failure modes.
  • Fix-forward is immediate. A hotfix is a publish. Seconds, not a pipeline.
  • It costs nothing to start. No catalog, no consumer graph, no manifest generator. It works on day one with a bucket and a CDN.
  • Ownership is legible. The team that owns the module owns the URL. Nobody has to ask who can change what.

I am giving up the second and third of those deliberately, and I want that on the record before I explain why. The next section is the case that made me willing to.

Figure 2

A host fetching three remotes and resolving one shared copyFour columns. A browser opens a host application whose shell is drawn around three empty dashed slots. The host fetches three remote entry files from a CDN, and each arrives to fill its slot, drawn solid. A highlighted bus then runs from all three filled slots into a single shared-scope card holding one copy of React and one copy of the router, annotated that a singleton must exist exactly once on the page.BrowserHost applicationCDNShared scopeshell: nav, layout, authslot: searchsearchslot: checkoutcheckoutslot: accountaccountdashed: the slot before its remotesearch/remoteEntry.jscheckout/remoteEntry.jsaccount/remoteEntry.jsone copy onlyreact 18.2.0router 1.4.0a singleton must existexactly once on the page12341 — the user opens the page. 2 — the host fetches the three remotes. 3 — each remote fills its slot.4 — every remote resolves its singletons against the one shared copy.
Three separately deployed remotes land inside one open page, and all of them resolve their singletons against a single shared copy. That convergence is the mechanism every failure below runs through.

Why I'm not doing that

Two green merges, one red production

Say checkout and search are remotes inside the same host, and both use a shared router that holds state. It has to be a singleton: two routers on one page and navigation stops working. Federation resolves one copy at runtime out of the versions the remotes offer, by a tie-break rule I am not going to paraphrase from memory — the argument below only needs that one copy wins and some remote gets a version it was not built against.

  • 10:00. checkout merges. Its pipeline built the host against whatever search was current when the pipeline started. Green.
  • 10:30. search merges, having widened its router requirement from ^1 to ^2. Its pipeline built the host against the checkout that was current when its pipeline started — possibly the 09:00 one. Green.
  • 10:31. A user loads the page. search loads first and puts router 2.x into the shared scope. checkout, built and tested against 1.x, gets 2.x. Its context provider mounts against an API that changed shape. Half of checkout renders empty.

Two pipelines were green. Nobody deployed anything untested. The pair that broke had simply never existed anywhere except production, and it existed there ninety seconds after the second merge.

I want to be precise about the status of that example: it is a mechanism I am confident in, not an incident I am reporting. Anything holding module-scoped state has this property — the framework itself, a router, a store, a design-system theme context. What I do not have is the frequency. I am arguing from the mechanism because I have no survey and no published incident data telling me how often cross-remote incompatibility actually bites, and the frequency is precisely what decides whether I should build this at all.

Figure 3

Two green pipelines producing an untested combination in productionA timeline marked 10:00, 10:30, 10:31 and 10:45. Checkout merges at 10:00 having been tested against the 09:00 search; search merges at 10:30 having been tested against the 09:00 checkout. Both pipelines pass. At 10:31 production holds checkout, which expects router 1.x, beside search, which supplies router 2.x — a pair that never existed anywhere else. Errors spike at 10:45.untested pair serving users10:0010:3010:3110:45checkoutmerge · router ^1CI built the host against search @ 09:00searchmerge · router widened to ^2CI built the host against checkout @ 09:00productioncheckout — built against router 1.xsearch — supplies router 2.xthis pair never existed anywhere elseerrors spike
Two green pipelines, thirty-one minutes apart, each tested against a version of the other that was already stale. The pair that reaches production at 10:31 never existed anywhere else.

The combination space is larger than anyone is testing

Numbers below describe an assumed fleet, not a measured one. Four host applications. Twelve modules. Five remotes in an average host. Each remote ships about three times a week.

Within one week, a host with five remotes at three versions each has 3⁵ = 243 reachable combinations. Staging validated perhaps five of them — the ones that happened to be current when a suite ran. The other 238 are not forbidden. They are just untested, and a user lands on one whenever two deploys interleave.

Testing the matrix head-on does not rescue this. Full-matrix integration testing grows as modules × applications, and it still does not cover version skew within a module's own history. You cannot run 243 suites per application per week, and if you could, they would be stale before they finished.

Pinning collapses the space rather than climbing it. If each application resolves exactly one named set per environment, then there is exactly one combination per application per environment, and end-to-end testing is one suite per application. Four suites, not a matrix.

That is the part I did not expect when I worked through this design, and it is the reason I stopped treating manifests as a rollback feature. Pinning the set is what makes testing finite. Every other benefit is downstream of that.

Figure 4

243 reachable combinations against four pinned onesThe upper panel is a field of 243 small outlined squares, one per reachable combination of five remotes at three versions each, with five of them filled to mark the combinations a staging suite actually ran. The lower panel shows four applications, each with a single filled square beneath it: one pinned combination per application per environment, and four end-to-end suites in total.Mutable pointers — one host, five remotes, three versions eachreachable combinations: 243tested by a staging suite: 5the other 238 are not forbidden — only untestedPinned manifests — one named set per application per environmentapp Aapp Bapp Capp Dcombinations per environment: 1 each · end-to-end suites: 4
Five tested combinations in a field of 243, against four applications with one pinned combination each. The squares are drawn at the same size in both panels so the areas compare directly.

Rollback has no target

10:45. Errors spike. You revert checkout's pointer to yesterday's file.

But search also moved, at 10:30. You are now in a third combination, which also never ran anywhere. Reverting a mutable pointer can state what you are reverting from. It cannot state what you are reverting to. During an incident that distinction is the whole job.

Rolling back the host does not help either, because rolling back a host does not roll back its remotes. There is no single artifact whose reversion returns the page to a known state. The only coherent rollback target is a set — which means the set has to have been named on the way in, before anyone needed it.

Figure 5

A mutable pointer beside an addressable manifest historyOn the left, one CDN URL points at whatever bytes are currently at that path, with two earlier versions drawn crossed out and unreachable. On the right, a manifest names the host, two remotes and the singleton version the set was resolved against, and sits in a row of previous manifests with a marker beneath one of them showing where production currently points.Mutable pointercdn.example.com/prod/checkout/remoteEntry.jswhatever bytesare there right nowoverwrittenoverwrittenNames one file. It cannot name a set,and it cannot name a previous state.Manifestmanifest 7f3a9chost@ 4b1e07search@ a90c22checkout@ 55de81router1.4.0 (singleton)7f3a9ce21b409c0d15production points hereNames the whole set. Every previous set is stilladdressable, so rollback is moving this arrow.
A mutable pointer names one file and overwrites its own history. A manifest names the whole set, and every previous set stays addressable, so rollback is moving one arrow along a row.

A canary needs two populations that hold still

The naive version of automated rollback compares a metric before the deploy with the same metric after it. That cannot distinguish a bad release from a downstream dependency degrading at the same moment, and its usual response to the second case is to roll back a healthy version while the real problem keeps running.

The working version runs old and new concurrently and compares the two populations. Kayenta does this by construction: it stands up a baseline alongside the canary and judges the difference between them. Argo Rollouts can, by querying stable and canary metrics over the same window and comparing the two. Flagger is the counter-example worth knowing — its built-in checks score the canary against fixed thresholds rather than against a concurrent baseline, which inherits exactly the problem in the paragraph above.

Concurrent comparison requires both arms to stay put for the length of the comparison. Under mutable pointers, the old arm drifts the moment any other module publishes — both populations are moving, and the difference between them stops meaning anything. Blue/green needs the same property in a stronger form: two complete version sets addressable at the same time. A mutable pointer names one thing by construction, so it cannot express blue/green at all.


What I'm building

Register, stated before the design: all of this is designed and none of it is built. No number in this section is measured, and where I have guessed I say so.

A manifest is an immutable, content-addressed document describing one application in one environment. It names three things:

  • the host version
  • every remote at a content hash
  • the singleton versions the set was resolved against

Manifests are generated on merge and never hand-written. Promotion runs like this:

  1. A module merges. The artifact is published — and not activated.
  2. The consumer graph says which applications use it. A candidate manifest is generated for each.
  3. Each candidate runs its own application's end-to-end suite.
  4. Each candidate runs a canary against the manifest currently live for that application.
  5. Pass: the pointer flips. Fail: the pointer does not move, and the previous set was never disturbed.

Automatic downstream propagation survives all of this. The graph query, the generation, the canary and the flip are all machine-driven. What changed is that the thing propagating is a validated set instead of a lone file.

Figure 6

A merge fanning out into one validated manifest per applicationA module merges and its artifact is published but not activated. The consumer graph names the applications that use it, and the pipeline fans out into one candidate manifest per application. Each candidate runs that application's end-to-end suite and then a canary against the manifest currently live. Application A passes and its pointer flips; application B fails, its pointer does not move, and the version is quarantined pending human clearance.module merges to prodartifact publishedpublished, not activatedartifact catalog +consumer graphwhich applications consume this?one candidate manifest per consuming applicationcandidate manifest — app Aapp A end-to-end suitecanary vs the live manifestcandidate manifest — app Bapp B end-to-end suitecanary vs the live manifestpointer flips to the new manifestpointer does not move; the set is undisturbedversion quarantined; human clearance to re-promoteone merge, live for Aand pending for B
One merge fans out into a candidate manifest per consuming application, each validated independently. Application A's pointer flips; application B's canary fails and its previous set is never disturbed.

Singleton bumps get blue/green rather than a maintenance window. Moving React, or a router, or anything else holding module-scoped state, has to be atomic across the host and every remote. Serve two manifests at once: new sessions resolve to v2, in-flight sessions finish on v1, nothing mixes, and rollback is a pointer flip. A maintenance window buys the same atomicity with downtime, and it is incompatible with canary analysis because there is no live traffic to measure. It also creates a scheduling bottleneck: bumps get batched, batches get large, and large batches are the risky kind. That is the road that ends with a dependency version nobody can upgrade.

Figure 7

Two manifests served concurrently through a router upgradeA time axis with a flip line in the middle. Manifest v1, holding router 1.x across the host and both remotes, runs from the left and tapers away after the flip as in-flight sessions finish. Manifest v2, holding router 2.x, starts at the flip line and runs to the right. In the overlap both sets are served and no session holds both. A traffic strip below runs at constant height with no gap, beside a smaller inset of a maintenance window that has a gap cut out of it.flipmanifest v1 · router 1.xhost 1.xsearch 1.xcheckout 1.xin-flight sessionsfinish hereboth sets served at once — no session ever holds bothmanifest v2 · router 2.xhost 2.xsearch 2.xcheckout 2.xtraffic — no downtimemaintenance window, for comparison
Two complete sets served at once through a singleton bump. In-flight sessions drain on v1, new sessions start on v2, no session ever holds both, and the traffic strip has no hole in it.

Auto-rollback defaults differ by artifact type. On by default for modules and applications; off by default for services. A controller cannot safely roll back a service that has already migrated a schema forward, and that has to be declared in artifact config upfront rather than discovered during an incident.

The rollback unit is the manifest pointer — one atomic operation, because nothing smaller is a coherent target.

Ordinary libraries stay outside all of it. They do not auto-propagate and do not trigger deployments; the owner of the consuming artifact bumps the dependency deliberately. Automate the pull request, never the merge. That is autonomy, and it is also how a fleet ends up with a CVE unpatched for a quarter, so it needs a drift inventory and a patch deadline bolted to it.


The ledger

Every row on the left is paid for by the row beside it. This is the honest version, and it is the part I most want argued with.

What pinning the set buysWhat it costs
Testing becomes finite: one combination per application per environment, four end-to-end suites instead of a matrix.Two pieces of infrastructure have to exist first — an artifact catalog and a consumer graph derived from build metadata. Weeks of work that ship no user-visible feature.
A rollback target that actually exists. Reverting names a set that ran and passed.Old remotes cannot be deleted when the pointer flips, because in-flight sessions still reference them. Retention has to exceed the longest session, and a reaped manifest is a rollback you cannot perform.
A canary that means something, because both arms hold still for the length of the comparison.Time to production goes up. “Publish and you are live” becomes publish, generate, test, canary, flip — and fix-forward loses its speed exactly when speed matters most.
Zero-downtime singleton bumps. Two complete sets served concurrently, sessions never mixed.Every production merge in the fleet blocks on the manifest generator. That is a new service and a new class of outage, where nothing ships anywhere.
The controller reverts the right thing, because per-remote labels say which remote is at fault.A browser error beacon, an error boundary per remote, and client-side artifact / version / manifest_id labels — all of which must exist before auto-rollback is switched on, or it covers backend services only.
Blast radius is per application. A failing canary for one app does not block the flip for another.The same module is then live for application A and pending for application B. Two truths at once is harder to reason about than one, and every dashboard has to say which.
Authorship stays independent. Teams merge on their own schedule, into their own repositories, without asking anyone.Activation becomes coordinated. That is a genuine loss of autonomy and I am not going to describe it as anything else.

Reading that table honestly: it is not obviously positive. Four of the seven costs are permanent operating burdens rather than one-time build costs, and one of them — the generator on the critical path — trades a distributed failure mode for a centralised one. I think the trade is right for a fleet whose remotes share a runtime. I have no operating data proving it, which is the whole reason this page exists.

And the boundary condition, stated plainly, because it is the fastest way for a reader to work out whether this applies to them: if your remotes share nothing, mutable pointers are correct and everything here is overhead. No framework singleton, no shared store, no cross-remote props, no design-system context holding state, and the failure in the first section cannot occur. I expect most fleets sit on the sharing side of that line — reasoning, not observation. Not sharing costs a framework copy per remote on every page load, and gives up composition across remote boundaries: no shared context, no design-system provider, no passing a component from one remote to another. That bill is immediate and visible, where version skew is deferred and probabilistic, and the tooling defaults point the same way. Teams that isolate deliberately exist and are not making a mistake — I doubt they are the majority.

Figure 8

Host-level error attribution against per-remote attributionAbove, a host holding four remotes emits one error metric labelled only with the application name, so the rollback controller reverts the whole host and moves four artifacts to fix one. Below, each remote sits in its own error boundary and emits its own labelled series — three flat, one rising for checkout — so the controller identifies the culprit and flips the manifest pointer back to the last good set.Host-level attributionstorefront (host)searchcheckoutaccountreviewsapp_errors_total{app='storefront'}rollback controllerrolls back the host — four artifacts move, three of them healthyPer-remote attributionsearchcheckoutaccountreviewserrors_total{artifact='checkout',version='55de81', manifest_id='7f3a9c'}rollback controllermanifest pointerflips the pointer back to the last good set — one culprit named, one whole set reverted
Host-level error metrics can only name the host, so the controller moves four artifacts to fix one. Per-remote labels name the culprit — and the thing that still moves is the whole set.

What I'm building first, and in what order

The order is a dependency chain, not a priority ranking. Everything above the line it sits on is inert without it.

  1. Artifact catalog — template version, builder version, dependency set and current manifest for every artifact. Nothing else can be built until this knows what exists.
  2. Consumer graph, derived from build metadata. Without it the generator cannot answer “which applications use this module”, which is step 2 of promotion.
  3. Build once, promote the artifact. Rebuilding per environment means the thing tested is not the thing shipped, and every guarantee below is void.
  4. Manifest generation and the pointer flip. The first point at which any of this is visible to a developer.
  5. Concurrent manifest serving, which is what makes blue/green possible.
  6. Browser error beacon, per-remote error boundaries, client-side labels. Observability before automation, not after.
  7. Canary comparison and auto-rollback, which is the first step that requires everything above it to already work.

I expect to find out that step 1 is bigger than it looks and step 4 is smaller. That is a guess, not a plan.

Figure 9

Seven layers of platform work, stacked in dependency orderA stack built from the bottom up: artifact catalog, consumer graph, build once and promote the artifact, manifest generation and the pointer flip, concurrent manifest serving, then browser beacon with per-remote error boundaries and client-side labels, and finally canary comparison with auto-rollback. A highlighted line separates the last layer from the rest, marked "observability before automation". The bottom three layers are annotated as shipping no user-visible feature.buildorder1. Artifact catalogtemplate, builder, dependencies, manifest2. Consumer graphderived from build metadata3. Build once, promote the artifact4. Manifest generation + pointer flip5. Concurrent manifest serving6. Beacon, error boundaries, client labels7. Canary comparison + auto-rollbackinert without 1 and 2;first thing a developer seesblue/green is impossiblebelow this linewithout 6 this coversbackend services onlyobservability before automationlayers 1 to 3 ship no user-visible feature
The build order as a stack rather than a backlog: each course rests on the one below it, and the one rule the ordering exists to enforce is that observability lands before automation.

The objection I expect, and my honest response

Written the way someone holding it would write it:

You have rebuilt the coordinated release and kept the runtime complexity. The only reason to accept runtime composition, version skew, shared-scope configuration and a second class of deployment artifact is that my team ships when we are ready. Under your design my module merges and then waits — for a graph query, for a manifest, for someone else's end-to-end suite, for a canary window sized by someone else's traffic. If a consuming application's suite is flaky, my deploy is stuck behind a test I do not own and cannot fix. You have also added one service that every production deploy in the fleet blocks on: when it is down, nothing ships anywhere, which is a worse day than the incompatibility you set out to prevent. And you kept a human approval gate on top of all of it. At that point a monolith with a good test suite is simpler, and I would rather run one.

Three responses, one of which is a concession.

On coupling: the objection is partly right and I should not soften it. What stays independent is authorship. What becomes coordinated is activation. My position is that the autonomy being surrendered is the autonomy to activate a combination nobody tested, which is not autonomy I want the platform to offer. But “you took something away” is a correct description, not a misunderstanding, and if I am going to build this I should say so to the teams it affects before they discover it in a release.

On the flaky suite: mostly conceded. Manifests are per application, so application B's failing canary does not block application A's flip. That contains the blast radius. It does not fix “my module is stuck behind a suite I do not own.” Doing that properly needs an owner-controlled bypass with an audit trail, which I have designed and not built, and which is the exact mechanism that decays into rubber-stamping.

On the critical-path service: that is a fair hit and I do not have a satisfying answer. Mitigations exist — queue merges instead of failing them, keep the last known good manifest servable, treat generator downtime as a deploy freeze rather than an outage. None of them makes the dependency go away. I have no data saying a manifest generator's availability beats the incident rate it prevents, and that comparison is the single number I most want and least have. If someone hands it to me and it points the other way, I do not build step 4.


What would stop me building it

I would rather learn any of these now than after the catalog exists. Any one is enough:

  • A fleet of ten or more remotes across four or more hosts, running mutable pointers with genuinely shared singletons, reporting under one cross-remote incompatibility incident per quarter across a full year. That sinks the premise, not just the design, and I stop at step 0.
  • Module Federation, or a successor, shipping real per-remote singleton isolation, so a version mismatch degrades one remote instead of breaking the page. If atomicity stops being required, blue/green goes and most of the manifest argument goes with it.
  • Merge-to-production time under manifests measured at more than four times the mutable-pointer baseline with no measurable drop in incident rate over two quarters. That is the number I will measure on myself once steps 1 through 4 exist, and I will publish it whichever way it lands.
  • A coherent per-remote rollback primitive: one remote reverted inside a live host, without touching the others and without landing in a combination nobody tested. If that exists, the manifest is not the only possible unit of change and I would want the simpler one.

Open questions, and what I'm asking you

This is the part I actually published for. In rough order of how much your answer would change what I build:

  1. Have you run a federated fleet on mutable pointers with shared singletons? How often did cross-remote incompatibility actually bite — once a week, once a quarter, never? That single number decides whether any of this is worth building, and I do not have it.
  2. Have you built something in this shape? If so, which manifest fields did you need that are not in my three, and what broke first? I am expecting to find that the schema is wrong and I would rather find out now.
  3. What is your merge-to-production time, under whichever scheme you run? I need a baseline to measure step 4 against, and I would rather not use my own optimistic guess.
  4. Where does the ledger lie? If one of those seven rows is understating a cost or overstating a gain, say which. The row I trust least is the last one — I may be underrating how much teams value activating on their own schedule.
  5. Reconciliation against a repository with unmerged work. A builder update sweeps the fleet and meets a repo with an open branch on it. Commit onto main, open a pull request and walk away, or skip and record drift? I have no answer I like, and the wrong one silently breaks somebody's in-progress work.
  6. Contract testing on a federated edge. Pact is the reference for consumer-driven contracts between services, but the thing being checked here is exposed-module names, prop shapes, event contracts and singleton ranges — closer to a type-signature check than an HTTP interaction. If tooling for that already exists, I would rather use it than write it.
  7. Canary thresholds on a low-traffic application. Two requests and one error is a 50% error rate, and the least-used app in a fleet will flap constantly under a static threshold. Multi-window burn-rate alerting plus a floor on sample size is the plan; the actual numbers are unset and I would take a starting point from anyone who has tuned one.

Figure 10

A builder change fanned out against the same change rolled in wavesOne commit to a builder. Fanned out, it mutates twenty repositories simultaneously and four of them break with the partial state unknowable. Rolled in four waves of five, the first wave is clean, the second carries a failure, and a halt on failure rate stops the sweep so the third and fourth waves never start. A note records the unresolved case: one repository in the halted wave has unmerged work on it.One commitbuilder v3Fan-out, no wavesbuilder v3twenty repos mutated at once;partial state unknowableWaved rolloutwave 1wave 2wave 3wave 4halt on failure ratenever startedand one repo in wave 2 has unmerged work on it.Commit? Open a pull request? Skip and record drift?
A builder change is a fleet mutation, not a build. Fanned out, twenty repositories move at once and partial state is unknowable; waved, the sweep can stop — and one repository in the halted wave still has unmerged work on it.

Two more I am carrying and have not solved: manifest retention is “longer than your longest session”, which is a policy rather than a number and depends on whether you have dashboard users who never reload. And per-deploy human approval scoped by ownership registry survives on paper, but its usual failure is not rubber-stamping — it is an on-call engineer at 03:00 with no approval rights for the artifact that is on fire.

None of this is open source yet, because none of it exists yet; when there is a repository to point at, this section will name it and its license. Until then the contribution I want is the review. If you have run one of these fleets, or built the thing I am describing, or think the ledger is wrong, tell me — the whole point of publishing before building is that changing my mind is still cheap.

Suggestions wanted

Send the approach I haven't tried

This is a working position, not a finished one. If you have built the hardware or run the systems it argues about, the useful reply is the constraint I left out, the number you would set differently, or the method that beat this one in practice. Every suggestion gets an answer, and the ones that hold change the essay in place with a dated note.

[email protected]