Sequence allocation

Some numbers are a shared resource that git cannot see. Decision records named 0001-*.md, 0002-*.md. Issues written as ## 58. headings in one long file. Migration numbers, RFC numbers, schema versions – anything where “the next one” is a scalar that two people can reach for at the same moment.

This is the one collision class that every other control in this repository is blind to, and it is the reason scripts/coord/alloc.ps1 and scripts/hooks/seq_check.py exist. Decision records are used as the worked example throughout – see examples/sequence-adr/ – but nothing about the mechanism is specific to them. What a sequence is lives entirely in ccx.config.json.

The defect

Two sessions each look for the next free number. Both compute the same answer – correctly, from their own point of view – and both use it. They create differently named artifacts: 0004-alpha.md and 0004-beta.md, or two ## 58. headings sixteen hundred lines apart in the same file.

Git merges both cleanly. There is no textual conflict, because the two sessions never touched the same bytes.

Measured on the repo this tooling was developed in, this fired three separate times. Each time the symptom was recorded as “numbers churn, recompute before merging” and a workaround was written down. It is not churn. It is a concurrency defect, and the workaround is the bug.

Rule. Never compute the next free number by scanning for a maximum and adding one. Allocate it atomically, and enforce the allocation at commit time. When a symptom keeps recurring and the remedy keeps being “redo it by hand”, ask whether you are looking at a race.

The two halves

Neither half is sufficient alone.

The allocator is a test-and-set, not a read-modify-write. It claims the number by exclusively creating <state-root>/alloc/<kind>/<number>.json with FileMode::CreateNew and FileShare::None; if a sibling session got there first, the create throws IOException and the loop moves to the next number. That throw is the mutual exclusion.

A shared list you read, edit and write back is not an alternative. Measured on the repo this tooling was developed in: eight concurrent PowerShell writers to one shared file lost four writes, with no error raised anywhere. Eight concurrent allocator processes against the exclusive-create scheme produced eight distinct numbers and zero collisions.

The registry lives beside the shared object store – <git-common-dir>/<prefix>-coord/alloc, resolved by Get-CcxStateRoot in scripts/coord/_common.ps1 and by state_root() in scripts/hooks/_ccxconfig.py. Every linked worktree of a clone sees the same allocations; a different clone gets its own automatically; and nothing there can be swept into a commit by git add -A.

Numbers are never reclaimed. An abandoned branch holds its number forever and the sequence develops holes. That is deliberate: holes are free, collisions are not.

Configuring a sequence

One key in ccx.config.json. Omit sequences entirely and both halves are inert – the allocator refuses with a message naming the file to edit, and the gate returns 0 without a word.

{
  "sequences": {
    "adr": {
      "dir": "docs/adr",
      "filePattern": "^docs/adr/(\\d{4})-[^/]+\\.md$",
      "pad": 4,
      "indexFile": "docs/adr/README.md",
      "indexRowPattern": "^\\|\\s*\\[(\\d{4})\\]"
    }
  }
}

Both scripts validate this before touching the registry, and both name the file and the key in every message. indexFile and indexRowPattern must be given together or not at all: half a configuration silently drops a whole term from the floor, and a floor that is silently too low is the exact failure the allocator exists to prevent.

indexRowPattern is compiled multiline, and that was once a silent hole

alloc.ps1 builds it with RegexOptions::Multiline; seq_check.py compiles it with re.M. Without that flag ^ anchors to the start of the string, not of each line.

This mattered because the two terms that use the pattern feed it differently. The all-refs term feeds it one line at a time, where it matched and looked correct. The working-tree term feeds it the whole file as one string, where ^ could never match past the first line. Measured on the repo this tooling was developed in: without Multiline the working-tree term found none of the index’s rows. So the term that exists to catch a number written but committed nowhere had been finding nothing since the day it was written. The all-refs term hid it, by covering every number that had been committed somewhere – which is every case except the one that term is for.

Rule. When two terms of the same computation feed one matcher different shapes of input, the stricter shape is the one to test. A term that is subsumed by a broader term in the common case will not announce that it has stopped working.

Using it

# take a number
pwsh -NoProfile -File scripts/coord/alloc.ps1 -Kind adr -Title "Worktree gate"

# inspect the floor without spending anything
pwsh -NoProfile -File scripts/coord/alloc.ps1 -Kind adr -ShowFloor

# what does this worktree currently hold?
pwsh -NoProfile -File scripts/coord/alloc.ps1 -List

-Kind may be omitted when exactly one sequence is configured. With two or more it is required, and the error lists the configured names – the parameter is deliberately not a ValidateSet, so the repo does not carry two lists of kinds that have to agree.

-Title is required for a real allocation. It is recorded in the claim so a sibling session running -List can see what the number is for.

A successful allocation prints the number, the directory to put it in, the pattern the path must match, a suggested filename, and a reminder to add the index row in the same commit. The claim file records number, kind, title, branch, worktree, claimed, as UTF-8 with no BOM – the Python gate reads it with encoding="utf-8" and a BOM makes json.loads raise.

The floor, and why it ratchets

The floor is the maximum over four terms, then ratcheted against a persisted high-water mark:

Every ref, not just the trunk. Reading only the published branch is precisely what re-issues numbers that already exist on refs the published branch does not carry. They are invisible to the sweep, so the allocator hands them out as free. The collision surfaces later as two differently-named files that merged clean. A number that exists on any ref is taken.

Term 2 is batched for a reason. A git show per ref spawns one process each; measured on the repo this tooling was developed in, that cost roughly 34 seconds on Windows where two git cat-file processes did the same work in about 3. Most refs share the same blob, so de-duplicating by object id collapses several hundred specs into far fewer reads.

The ratchet

Every term above is derived from refs a routine cleanup can remove. The sweep is only as good as the refs this clone happens to hold. Measured on the repo this tooling was developed in: the floor computed over all refs was materially higher than the floor computed over origin’s refs and local heads alone, because numbers lived on remote-tracking refs for a remote that git remote -v no longer listed. Drop those and the floor silently reverts to a lower value, and the allocator resumes issuing numbers that are already in use – no error, no signal.

So the floor is persisted to <state-root>/alloc/<kind>/.floor-highwater and may rise but never fall. When the computed floor comes in below the mark, alloc.ps1 prints a loud NOTE naming both numbers and telling you to re-fetch before trusting any number-space reasoning in that clone.

The ratchet is a backstop, not a substitute for the refs: it keeps the allocator from re-issuing, but the history those refs pointed at is still gone.

Allocation is a one-way door, so it ships a read-only probe

Numbers are never reclaimed, so before -ShowFloor existed the only way to find out what the floor could see was to spend a number on the question. That made the floor’s own correctness the one property nobody re-tested – which is how it went an entire release reading two refs while its own header promised all of them.

-ShowFloor prints the kind, the resolved trunk, the floor (with the computed value and the high-water mark shown separately), the paths it swept, the number it would issue next, and the watermark path – and allocates nothing.

Two details make it trustworthy:

• It names its sources, not just the number. “Which paths did this sweep actually read” is the question every silent-narrowing bug turns on, and a bare integer cannot answer it. A floor looks identical whether it swept one path or two.
• It cannot corrupt what it reads. Get-Floor -Peek skips the high-water write. The first run of -ShowFloor against a deliberately planted number ratcheted that clone to a fabricated floor no later run could undo. An inspection that moves the thing it inspects is not an inspection.

One computation, two callers: -ShowFloor and a real allocation differ only by -Peek, so they cannot report different numbers. They used to – -ShowFloor returned before a guard every real allocation ran, and printed a next number the tool would then refuse to issue. Anything added later that can change the outcome belongs inside Get-Floor or above both branches, never in one of them.

Rule. Any irreversible allocator needs a dry run that reports its own inputs, and the dry run must run the same code path as the real thing.

The gate

scripts/hooks/seq_check.py runs at pre-commit, and again in CI with --ci.

Why a git hook, and why the shared hooks directory

.git/hooks lives in the common git directory, which every linked worktree shares. One file there:

• reaches every worktree the instant it is written – no branch, no merge, no propagation lag;
• survives a branch switch in any of them, because it sits outside every working tree;
• and sees every write route, because it inspects the staged tree rather than a tool call.

That last property is the one that matters here. A PreToolUse gate reads tool arguments, so it is blind to a shell redirect, Set-Content, python -c, a heredoc, an editor, or a subagent. The commit hook sees all of them, because by then the bytes are in the index.

What it checks

Per configured sequence:

1. An added file carrying number N must not reuse an N that already exists on trunk – unless the index row for N names the new file as a declared companion. (One number, one row, two files is legal; the row itself names the companion. Only an undeclared reuse is a collision. The companion is matched with and without its extension, since an index row conventionally links the stem.)
2. An added number must have been allocated to this worktree. Local only – see the mode asymmetry below.
3. An added number must have a row in the sequence’s indexFile.
4. The index must not gain a duplicate row for one number.

What it deliberately does not check

Anything about numbers already on trunk. Rule 3 applies only to files this change adds; rule 4 only to duplicates this change introduces (duplicates already on the base are subtracted out).

Rule. A gate that fails on pre-existing debt is a gate that gets uninstalled, and it takes the real protection with it when it goes.

For the same reason it reads the staged tree (git show :path), never the working tree. A gate reading the working tree blocks every unrelated commit the moment you have an untracked work-in-progress file in your checkout.

And it is stdlib only, with no project import. Most worktrees have no virtualenv and no project install; a gate that skips because an import failed is worse than no gate, because it still looks installed. The one shared import is its sibling _ccxconfig.py, and a failure to find it exits non-zero with an explicit message rather than degrading to silence.

Two rules that keep it honest

Both live in _ccxconfig.git() and both were paid for:

• encoding= is required, not cosmetic. text=True alone decodes with the locale default, which is cp1252 on a stock Windows box. Index and ledger files are routinely UTF-8, so the decode raised inside subprocess’s reader thread, proc.stdout came back None, and the caller died on findall(None) – blocking every commit that touched exactly the files the gate guards.
• A non-zero git exit raises. A bad ref, a missing path or an unfetched base must never read as “the file is empty”, because an empty index parses as “no numbers taken” – the false clean a collision gate must never emit. When the git wrapper swallowed non-zero exits, the added-files list came back [] and the gate reported PASS on every run where it could not see.

The one legitimate “absent from that ref” case gets its own explicit probe, object_exists(), rather than a broad except that would also hide a genuinely broken ref.

Whatever it could not check, it prints – pass or fail, on stderr. An unresolvable trunk means the already-taken-on-trunk rule did not run, and it says so. A skip that prints nothing is byte-identical to a clean run.

Wiring the pre-commit hook

Nothing in this repository installs it for you. scripts/coord/install-git-hooks.ps1 installs the claim gate (commit-msg) and the push guard (pre-push), and never writes pre-commit, at all. Two tools cannot both own that one file. A hook framework that finds a foreign hook there may rename it and invoke it from its own shim, and that chain has failed on Windows and blocked every commit in a repository until the shim was removed.

So the installer does the next best thing: whenever sequences is configured, it prints in yellow that the sequence gate is not installed by it and that until you wire one, nothing at commit time stops two sessions using the same number. bin/ccx-doctor.ps1 goes further and checks: it reports the control as OFF, with the reason, rather than omitting it. An absent gate looks exactly like one that passed.

Wire it into whatever hook framework you already use:

# in your existing pre-commit hook, or as its own file if you own that slot
python scripts/hooks/seq_check.py || exit 1

Verify by receipt, not by presence:

pwsh -NoProfile -File bin/ccx-doctor.ps1

The doctor reports whether any pre-commit in the resolved hooks directory invokes seq_check, and separately fires a read-only floor probe at the allocator – -ShowFloor, which never spends a number – so a broken allocator is caught without corrupting the sequence to find out.

Modes are not symmetric, and saying so is the point

--ci re-runs the same rules against a freshly fetched trunk, which is what catches the stale-base collision: each branch is internally consistent, and the duplicate only exists once both have merged. It re-runs every rule but one.

Rule 2 – allocation ownership – cannot run in CI. It reads a per-clone registry inside the git directory and compares a worktree path; a runner clones fresh and has neither, so the check would return False for every item and nothing could ever merge. So ownership is enforced locally and never in CI.

An earlier version ran the CI half of that rule anyway: it computed a set and discarded it, which made it structurally incapable of failing while reading, in source, exactly like coverage.

Rule. If a rule cannot run in a mode, name it as not running. Never leave it in place looking like coverage. A green CI on a numbered pull request is not evidence that the number was allocated to anybody.

The residual, stated plainly: after a --no-verify commit, a number belonging to another session’s unmerged branch can be taken with nothing objecting. The corruption then surfaces late – but loudly and recoverably – when the second of the two merges.

CI mode also insists on a resolvable base. Locally, an unresolvable trunk downgrades to a printed note; in CI it raises, because there the base comparison is the job and a base that does not resolve is a workflow misconfiguration.

If you wire the CI leg

Two things to get right, neither of which is obvious:

• Do not gate the step on a “code changed” path filter. A pull request that only adds a decision record is a docs-only change, so a code == 'true' condition makes the governance step skip on exactly the pull requests it exists to police. Path filters written for test suites invert the intent of a docs-governance check.
• Ride it inside an already-required job rather than adding a brand-new required context. A newly required check wedges every pull request opened before it existed.

Use a two-dot diff (base HEAD), not three-dot. On a pull request the checkout is typically the merge commit, so HEAD already contains base and three-dot buys nothing – while costing everything. It resolves a merge base, the checkout is shallow, and two truncated histories routinely fail to reach their common ancestor. Deepening to fix that is itself a race. A two-dot diff compares two trees: no ancestry, no depth, nothing to race. And the three-dot failure was silent – see the raise-on-non-zero rule above.

Ownership is only as real as your isolation

Rule 2 keys ownership on the worktree that holds the claim: owns() compares the claim’s worktree field, folded through fold_path(), against the current repo root.

That only discriminates because each session gets its own worktree. Measured on the repo this tooling was developed in, the ownership rule was a no-op before worktree isolation was enforced. Every co-tenant session authored in the same shared primary checkout, so every one of them mapped to the same key. The check could not separate exactly the sessions it was written to separate.

Rule. Check that your ownership key actually distinguishes the actors in practice, not merely in principle. Number allocation and worktree isolation are a pair – the first is meaningless without the second making the key real.

The folding is shared deliberately: fold_path() in _ccxconfig.py and ConvertTo-CcxComparablePath in _common.ps1 must agree character for character, because each side compares paths against records the other side wrote. If they fold differently, ownership silently stops matching and the gate either refuses everything or grants everything.

Two lessons from a guard that had to be removed

Both concern a rule that once sat in the allocator and is deliberately not in the shipped code. They are worth knowing because the shape recurs.

Two different maximums got conflated, and the allocator bricked on correct input. A guard meant to detect one band of a partitioned sequence encroaching on another read the floor – the maximum over everything swept. The first legitimate entry filed in the upper band therefore made every allocation in the repository throw a refusal. There were two measurements, not one: the floor answers “what must I not re-issue?” and must include every number from every band; the per-band maximum answers “how much runway does this band have?” and must not. The guard was not detecting a breach. It was detecting the partition being used exactly as designed.

Rule. Name each measurement by the question it answers, then check which one every consumer reads. A guard that fires on correct input will be disabled, and it takes the real protection with it.

A branch that cannot fire reads as protection and is worse than none. The obvious repair was to keep the refusal arm and make it unreachable. But once an entry exists at a number in the shared band, it is indistinguishable in the published files from a legitimate entry at the same number – both are just a number. A refusal arm would have to fire on correct input or never fire at all. Detecting a real breach needed an input the repository did not have.

Rule. Remove a branch that cannot fire; do not leave it dormant. Replace it with something the data can actually support – a warning at a threshold measured on the band where the other band’s numbers cannot distort it – and say in the docs what is no longer detected.

Limits

Stated plainly, because each one is a hole somebody will otherwise assume is covered.

Files