Rewriting chunk generation: the slices
The decision and its arithmetic are in ADR 0002. This file is the execution order. Each slice is sized for one agent, is independently committable, and leaves the tree working and the parity dashboard green.
Project-wide roadmap status and the current milestone are maintained in the data-driven vanilla plan; this file tracks the chunk-generation execution details.
Before any slice
Read ADR 0002, then java-chunk-generation-pipeline.md sections 1, 2, 6 and 7. Those sections are the specification. The ADR is the decision about which parts of it to copy and which one to deviate from.
Section 6 of worldgen-defects.md is superseded. Its 875 ms for structures is a one-dimensional walk artifact and its wall-clock ladder was taken on a loaded host. ADR 0002's "Why the current shape costs 1572 ms" has the second measurement pass. Take counts and microbenchmarks from either; take wall-clock totals from neither until someone retakes them on an idle machine.
Get the real data on disk and the environment variables into the shell:
pnpm --filter @jsprismarine/bedrock-data run fetch:sources
pnpm --filter @jsprismarine/bedrock-data run datagen:java
node scripts/real-data-env.mts # prints the assignments, fails loudly if the cache is shortThe four checks every slice runs
Name them by these labels; later slices refer to them.
| Label | Command | What it holds |
|---|---|---|
| corpus | pnpm --filter @jsprismarine/parity run test then pnpm --filter @jsprismarine/parity run dashboard | worldgen.terrain, worldgen.corpus, worldgen.structures and worldgen.determinism stay green, and no other row turns red or defects |
| generator | pnpm vitest run packages/prismarine/src/world/generators | the gated suite, real structures in real chunks |
| sampler | pnpm --filter @jsprismarine/worldgen exec vitest run src/chunk/golden.test.ts | six whole chunks, sha256, bit for bit |
| types | pnpm run typecheck | 43 of 43, test files included |
Also run pnpm run lint. Add a Changeset for every package a slice changes, choosing the bump from the versioning policy.
A slice that changes generated blocks says so in its own section and says which digests to re-capture. Every other slice must leave corpus, generator and sampler byte-identical. Do not re-capture a digest in a slice that does not name one.
The corpus this plan verifies against
The generation corpus landed in packages/parity while this plan was being written, and it is the net every slice below is caught by. Read it before slice 1: packages/parity/src/generation/{plan,capture,golden,diff,write}.ts, packages/parity/scripts/capture-corpus.ts, the golden at packages/parity/goldens/worldgen-corpus.json, and the three families in packages/parity/src/families.ts: worldgen.corpus, worldgen.structures, worldgen.determinism.
What it already gives you: per chunk, digests of layer 0 and layer 1, twenty-four per-section digests so a change is located within sixteen blocks of y, the biome cells, the heightmap, the block entities, the entity spawns, a block census, and a handful of chunks kept whole so firstBlockDifference names the coordinate and both state names. Three seeds, two tiers (terrain skips decoration and costs a quarter as much, full runs everything), and a structure placement scan. Provenance is in the file, including the Java data version, so a stale golden reports itself.
Its capture protocol is the thing to understand before slice 7. CAPTURE_PROTOCOL in capture.ts says each chunk is generated by a generator that has produced nothing else and is read before any other chunk is generated, because today's generator caches neighbor terrain and feature output per seed. That protocol is the only one that is currently a function of the seed and the coordinates. After slice 7 every order is, so the protocol sentence stops being a constraint and becomes a description of one case.
Two things the corpus does not cover, and one of them is a design requirement:
- Worker count. Nothing varies a pool size, because no pool exists. Slice 11 adds the sweep.
- Features belonging to their biome. A digest catches a change; it does not catch a change in the wrong direction. The count of
placeFeaturecalls per chunk should differ between a plains chunk and a desert chunk, and no chunk should hold logs of more than two species. The census in the golden can answer the second one today.
Do not mistake the older worldgen.terrain row for coverage of any of this. It compares nine columns out of ChunkSampler against a Java column dump and never touches DataDrivenGenerator, so carving, decoration and structures cannot move it. The other byte-level guard is packages/worldgen/src/chunk/golden.test.ts, and it covers ChunkSampler.fill() alone.
Slice 1: finish the corpus rows and prove they bite
What changes. The three declared families get their test files, if the in-flight work has not already written them, and the corpus gains the two things it does not cover.
Check first. git status --short packages/parity and ls packages/parity/src/worldgen.*.test.ts. If worldgen.corpus.test.ts, worldgen.structures.test.ts and worldgen.determinism.test.ts exist, this slice is only the additions below. Do not rebuild what is there.
Files. packages/parity/src/worldgen.{corpus,structures,determinism}.test.ts, packages/parity/src/generation/plan.ts.
Shape. The top-level describe title must be the family id or the row lands in (other). Declare the data need through REFERENCES.generationCorpus and REFERENCES.javaWorldgenData with describeFamily, not an ad-hoc skipIf, so a machine without the fetch cache reads not measured with a reason rather than passing silently.
The determinism row. Generate the corpus chunks in at least three orders, row major, reversed and a seeded shuffle, and assert the digests agree. Today they will not: the same chunk generated first and generated last differs at 169 of 98304 positions. Land the row with an allow-list entry of kind defect in packages/parity/src/allowlist.ts, reason "chunk content depends on generation order", reference this plan. The row then reads defects rather than red, which is what the dashboard's statusOf is for. Slice 7 removes the entry.
The two additions. A per-chunk placeFeature count that differs between a plains chunk and a desert chunk, and a log-species assertion over the census the golden already holds.
Prove it correct. Two consecutive runs agree. Flip one block in toChunk, watch worldgen.corpus go red and name the coordinate, then revert.
Prove no output change. This slice records behavior and changes none. generator, sampler and types unchanged.
Slice 2: ban the two nondeterminism hazards by lint
What changes. A lint rule that rejects Math.random and String.prototype.localeCompare under packages/worldgen, packages/features, packages/structures and packages/prismarine/src/world/generators.
Files. biome.json at the root, or a per-package override if the Biome config is split.
Why. Math.random is seeded independently per V8 isolate, measured as seven different sequences across a main thread and six workers, and the difference is silent. localeCompare orders minecraft:Oak before minecraft:_oak where a plain comparison does the reverse, which changes a palette index and therefore a feature seed.
Depends on. Nothing.
Prove it correct. Add a line calling Math.random() in one of those packages, see pnpm run lint fail, remove it. Then confirm the rule finds nothing today, or fix what it finds.
Prove no output change. No runtime code changes. corpus, generator, sampler, types unchanged.
Slice 3: stop assembling structures that the biome will reject
What changes. StructureStartSource.buildStart tests the biome before calling the builder instead of after, so a candidate that cannot exist where it landed is never assembled.
Why this is slice 3 and not slice 13. It is 87% of all structure layout time, measured. Over a 102 by 102 chunk area at seed 1234, 256 candidate starts were laid out and 234 were rejected by the biome check, and those 234 cost 85.5 of the 98.1 seconds. A rejected village costs 3083 ms, a rejected trial chamber 2145 ms, a rejected desert pyramid 796 ms. The cost is jigsaw assembly, not terrain sampling. Nothing else in this plan is a single line change worth a factor of seven.
Files. packages/prismarine/src/world/generators/datadriven/javaStructureStarts.ts (buildStart at :200, biomeAllows at :241).
This needs a parity ruling before any code. Java also checks the biome after findGenerationPoint, so moving the check in front of the builder is a deviation. Settle it this way. Java's check reads the biome at the generation point; a pre-check can only read the biome somewhere else, so the two agree only when the generation point sits in a biome the earlier position already allowed. Measure, over the same 102 by 102 area, how far each kept start's generation point is from its start chunk, and how many starts a coarse pre-check over the start chunk's own biome quarts would wrongly reject. If that number is zero, make the pre-check exact by testing the union of biomes the start chunk's quarts hold and keeping the exact check at :225 as it is; a candidate then has to fail both to be dropped, and no kept start can be lost. If it is not zero, report the number and stop, because a cheaper world is not worth a different world.
Depends on. Slice 1, so the corpus can prove no start moved.
Prove it correct. generator green, all 12 structure placements in javaStructures.test.ts in the same chunks. worldgen.structures green against the golden, which holds the placement scan.
Prove no output change. All four checks byte-identical. Every rejected candidate is a structure that was already not in the world, so a correct version of this change is invisible in every digest. If a digest moves, the pre-check is rejecting something Java keeps.
Report. Layout milliseconds per chunk, before and after, on both a 40-chunk walk and a 2-D fill, because the two differ by a factor of 33.
Slice 4: the status model
What changes. A new package @jsprismarine/chunkgen holding ChunkStatus, ChunkStep, ChunkDependencies and ChunkPyramid. Data and arithmetic only. Nothing imports it yet.
Files. packages/chunkgen/ with the same shape as packages/features: package.json ("type": "module", sideEffects: false, the vite build && node ../../scripts/emit-dts.mts build script, the dev exports pointing at src/), vite.config.ts, tsconfig.json, tsconfig.dts.json, tsconfig.test.json, vitest.config.ts, README.md, src/{index,ChunkStatus,ChunkStep,ChunkPyramid}.ts and their tests.
The content. All ten of Java's statuses, each with an index and a parent, and isOrAfter as an integer comparison. ChunkStep.Builder.addRequirement(status, radius) and buildAccumulatedDependencies reproduced from java-chunk-generation-pipeline.md section 2, including the addRequirement table exactly as the bytecode has it. blockStateWriteRadius is 0 for terrain, 1 for features, -1 everywhere else.
Keep initialize_light, light and spawn even though nothing runs in them yet, as ADR 0002 says. Collapsing them would shift every accumulated radius by one and turn the step table from a transcription into a derivation nobody can check against the jar.
Depends on. Nothing.
Prove it correct. Unit tests that reproduce the direct-dependency and accumulated-dependency tables from section 2 of the research note cell by cell, and the inverted radii: structure_starts at 11, structure_references and biomes at 3, terrain at 2, features at 1. The Java cross-check is that the accumulated list for full has length 12, which is what makes SAFETY_MARGIN_CHUNKS equal 90.
Prove no output change. Nothing imports the package. corpus, generator, sampler unchanged. types must still be 43 of 43 plus the new package's two projects.
Slice 5: the proto chunk
What changes. A ProtoChunk in @jsprismarine/chunkgen: the intermediate chunk state, stored as flat typed arrays so it can cross a thread boundary in a transfer list.
Files. packages/chunkgen/src/ProtoChunk.ts and its test. No change to the generator.
The content. A block palette as string[] of Java state keys plus a Uint16Array of indices in the generator's existing order, four heightmaps as Int32Array, a biome palette and index array, the block entity and entity spawn requests, and the persisted status. Two methods carry the depth: transfer() returns { payload, transferList } with every buffer in the list, and ProtoChunk.adopt(payload) rebuilds one on the other side with no copying.
Model the storage on GenerationRegion's, which is already a Uint16Array plus a palette plus an Int32Array of column heights (GenerationRegion.ts:148 to :153). This slice does not change GenerationRegion; slice 7 makes it a view onto a ProtoChunk.
Why flat. Measured round trip for one 24-section column: 18 microseconds transferred, 76 cloned, 1370 as a plain number[]. A per-section object graph costs the main thread 298 microseconds per chunk against 20. Nothing that crosses the boundary may be a class instance, a Map or an array of objects.
Depends on. Slice 4, for the status type.
Prove it correct. Round trip a populated proto chunk through structuredClone with a transfer list and assert the rebuilt one is equal block for block, and that the source buffers are detached.
Prove no output change. Nothing imports it. All four checks unchanged.
Slice 6: the holder and the scheduler, driving terrain
What changes. terrainFor and terrainChunks are replaced by a holder map and a scheduler. generateChunk asks the scheduler for the 3 by 3 neighborhood at terrain instead of calling terrainFor nine times, and then runs today's decoration and patch code unchanged on top of it. terrain is the only status with a body in this slice; the rest pass through. Slice 7 moves the feature pass and the promotion onto the ladder.
Keeping the change to one status is the point. The holder, the reference counting and the layer loop are the parts most likely to hide a bug, and this slice can prove them with byte-identical output because the feature pass it feeds has not moved.
Files. packages/chunkgen/src/{ChunkHolder,ChunkHolderMap,GenerationScheduler,StepTasks}.ts and tests; packages/prismarine/src/world/generators/datadriven/DataDrivenGenerator.ts.
The content.
ChunkHolder holds one Promise<ProtoChunk> slot per status. holder.advance(status) returns the existing promise when there is one and otherwise creates it from the step task. That is Java's acquireStatusBump without the compare-and-exchange, because the event loop makes the map insert atomic.
GenerationScheduler runs Java's layer loop: for a target status, walk the square of radius accumulatedRadiusOf(status) for each status from empty upward, call advance on every holder in it, and await the first promise that is not already settled. Holders already past a status cost nothing.
StepTasks is the seam: one function per status, taking the holder's chunk position and a view of its declared neighborhood, returning the proto chunk. The only body this slice supplies is today's DataDrivenGenerator.buildTerrain, moved behind it unchanged.
ChunkHolderMap replaces terrainChunks. Retention is by reference count, not by LRU: a generation task acquires a reference on every holder inside its radius and releases them when it finishes, and a holder with no references and a full chunk already handed out is dropped. GENERATION_CACHE_SIZE and trimCache go away. This matters because the LRU at 128 silently dropped cross-chunk writes past 128 chunks, and preload-radius: 5 is 121.
Keep serializeGeneration for now. It becomes a no-op in slice 11 and removing it here would change feature ordering before slice 7 makes that safe.
Depends on. Slices 4 and 5.
Prove it correct. New unit tests in packages/chunkgen against a fake StepTasks: a status body runs once for ten concurrent callers; a holder refuses a read outside its declared window; and the layer loop walks the slice-4 pyramid's widths for a full target, which are 11, 11, 3, 3, 2, 1, 1, 0, 0, 0.
Prove no output change. All four checks byte-identical. This is the slice where a scheduling bug hides, so also generate the corpus twice in one process and once per process and compare.
Watch for. The 16 by 16 column loop in buildTerrain yields the event loop on a 4 ms budget (DataDrivenGenerator.ts:347). Keep that yield. A status body that never yields blocks the loop for its whole duration, and that is the defect the yield exists for.
Slice 7: features as a status, and the patch merge
What changes. The feature pass becomes its own status and the promotion to a Bedrock Chunk becomes the full body. A feature's writes outside its own chunk are recorded as patches on the target holders instead of being replayed into terrain tiles and already-emitted chunks. A chunk reaches full only once it and its eight neighbors have passed features, which the pyramid already says. generateChunk becomes one call: advance this holder to full.
This slice changes generated blocks. It is the only one that does. Re-capture the golden with pnpm --filter @jsprismarine/parity run capture:corpus in this commit, remove the order-independence allow-list entry, and rewrite CAPTURE_PROTOCOL in packages/parity/src/generation/capture.ts. The old sentence says a chunk is generated by a generator that has produced nothing else, because that was the only protocol that was a function of the seed and the coordinates. After this slice every order is, and generating one chunk alone now drives nine feature passes, so the sentence has to say what actually happened.
Expect the golden to move in one direction: a chunk gains the blocks its eight neighbors' feature passes owe it. Under the old protocol a lone chunk never received them, which is why trees were cut off at borders.
Files. packages/chunkgen/src/{FeaturePatch,ProtoChunk}.ts; packages/prismarine/src/world/generators/datadriven/DataDrivenGenerator.ts (delete applyFeaturePatch, applyPatchToChunk, featurePatches, generatedChunks); packages/prismarine/src/world/generators/datadriven/GenerationRegion.ts (copyCenterFrom becomes a read-only view over the neighbors' proto chunks rather than a copy).
The rule, exactly. A chunk's feature pass reads its own live proto chunk and the terrain-status proto chunk of each of its eight neighbors. Writes into its own chunk apply immediately, so features see each other the way Java's do. Writes outside its own chunk append to a FeaturePatch keyed by target chunk. When a holder is promoted to full, its blocks are its own post-features state, then the eight inbound patches applied in a fixed neighbor order: (dx, dz) ascending with dx fastest, that is (-1,-1), (0,-1), (1,-1), (-1,0), (1,0), (-1,1), (0,1), (1,1). Write that order down in the code, in a comment, as the reason the output is a function of position.
Heightmaps update as the chunk's own writes land and again as each patch merges, in the same order.
Read ADR 0002's section "The one divergence from Java we accept" before writing any of this. The divergence is deliberate and bounded, and the reason the merge order is fixed rather than arrival-ordered is the whole point of the slice.
Depends on. Slices 1 and 6.
Prove it correct. The order-independence check from slice 1 goes green: row major, reversed and shuffled orders produce identical digests. The neighbor chunk data tests in GenerationRegion.test.ts still pass. A new test asserts a tree straddling a border is whole in both chunks.
Prove the change is the one intended. packages/parity/src/generation/diff.ts already names the first differing block by coordinate for the chunks the golden keeps whole, and the per-section digests put every other change inside a sixteen-block band of y. Use them. Expect the changes concentrated within one block of a chunk border and expect roughly the 169-per-chunk order-dependent set the defects note measured, not more. Record the count in the commit message. If blocks change in the middle of a chunk, something else broke.
Prove nothing else changed. sampler unchanged, because nothing here touches ChunkSampler. generator green: all 27 gated tests under generators/, including the 12 structure placements. types 43 of 43.
Slice 8: structure_starts and structure_references as statuses
What changes. A structure start is laid out once, in the chunk it starts in, at a radius-0 status. The radius-8 reference scan reads starts that already exist rather than building them. beardPieces stops calling startsFor once per terrain tile.
Files. packages/prismarine/src/world/generators/datadriven/javaStructureStarts.ts (split startsFor into startsIn(chunkX, chunkZ) at radius 0 and referencesFor(chunkX, chunkZ) reading the holders), DataDrivenGenerator.ts (beardPieces and decorate read the holder's references).
The content. structure_starts asks every structure set whether this chunk is one of its, and lays out the starts that say yes. It writes no blocks. The result lives on the holder, which replaces the 8192-entry starts map that clears wholesale.
candidateChunksIn (javaStructurePlacement.ts:83) answers "which chunks in this window does the placement grid pick", which is the wrong shape for a radius-0 status. Add the single-chunk form, Java's StructurePlacement.isStructureChunk(x, z), and derive the window form from it or keep both; candidateChunksIn is still what structure_references wants. structure_references walks the radius-8 window and collects the starts whose box reaches into this chunk, all of which exist because the pyramid drove the neighborhood to structure_starts first. Piece placement stays where it is, inside features, clipped to the chunk by chunkWriteBox.
Depends on. Slices 6 and 7. Measured, this is worth about 8 ms per chunk on its own, so land it for the shape rather than the speed.
Prove it correct. The 12 structure tests in javaStructures.test.ts still place a village, a shipwreck, a stronghold and the other nine in the chunks the seeded placement chose. structureStartsAt returns the same starts for the same positions as before.
Prove no output change. All four checks byte-identical, digests included. A start is already a pure function of the seed and its chunk, so moving where it is computed must change nothing.
Report. The candidate scan time per chunk before and after. It was about 11 ms at four scans per chunk, because buried_treasures and mineshafts have spacing 1 and each scan does 289 region draws for each of them.
Slice 9: hand a chunk over by its palette, not block by block
What changes. toChunk stops translating 98304 blocks and translates the chunk palette instead, then installs the index arrays into BlockStorage directly. recordBiomes stops writing 98304 biome entries one at a time.
Files. packages/prismarine/src/world/generators/datadriven/DataDrivenGenerator.ts (toChunk at :650, recordBiomes at :704), packages/prismarine/src/world/chunk/BlockStorage.ts and SubChunk.ts if they need a constructor that adopts an index array and a palette.
Why. This is a precondition for the worker pool, not an optimization for its own sake. toChunk is the work that has to happen on the main thread after a worker hands a chunk back, and at 100 chunks per second anything per block saturates the loop. Per chunk, per-block translation is 98304 Map lookups; per-palette translation is a few hundred.
Depends on. Slice 6.
Prove it correct. ChunkFastPath.test.ts, ChunkNetworkPayload.test.ts and BlockStorage.test.ts pass unchanged.
Prove no output change. All four checks byte-identical. The digest is computed from runtime ids, so a palette ordering difference would show up there.
Report. Main-thread milliseconds per chunk inside toChunk, before and after.
Slice 10: the registry bundle in shared memory
What changes. loadJavaWorldgenRegistries gains a second source: one UTF-8 JSON bundle rather than 1514 files. A builder concatenates the documents once, and a reader parses them from a buffer.
Files. packages/prismarine/src/world/generators/datadriven/javaWorldgenData.ts (loadJavaWorldgenRegistries at :173, findJavaWorldgenSources at :110), plus a new registryBundle.ts beside it.
The content. buildRegistryBundle(sources): ArrayBuffer writes a small header of (registry, id, byteOffset, byteLength) entries and then the raw document bytes. readRegistryBundle(buffer): JavaWorldgenRegistries decodes with TextDecoder and JSON.parse per document. The buffer may be an ArrayBuffer or a SharedArrayBuffer; the reader must not care.
Why this shape. Measured, eight workers reaching parsed registries: 75 ms from one shared 2.66 MiB bundle, 540 ms reading the files each, 289 ms through workerData, and the workerData cost lands on the spawning thread. Splitting the bundle is worth it later if a worker does not need all of it: without biome_parameters it is 1.2 MiB and 9 ms.
Depends on. Nothing. Can land in parallel with slices 4 to 9.
Prove it correct. A test that builds the bundle from the fetch cache, reads it back, and asserts the result is deep-equal to loadJavaWorldgenRegistries reading the files, registry by registry and document by document.
Prove no output change. Production still reads files in this slice. All four checks unchanged.
Slice 11: the worker pool, running terrain
What changes. terrain status bodies run in a piscina pool. features, the patch merge and the Bedrock translation stay on the main thread.
Files. packages/chunkgen/src/{GenerationPool,generationWorker}.ts, packages/chunkgen/package.json (piscina), packages/prismarine/src/world/generators/datadriven/DataDrivenGenerator.ts (pass a pool to the scheduler), and a worker-side pipeline entry in packages/prismarine/src/world/generators/datadriven/.
The content. The pool is generic. It takes a module specifier and a SharedArrayBuffer of registry bytes in workerData, and the worker dynamically imports that module to get its StepTasks. That keeps @jsprismarine/chunkgen free of any dependency on the server, and it is the seam the tests use with a fake pipeline module.
A worker builds one seed's pipeline once and keeps it, because RandomState costs 55 to 134 ms and the 7594-entry climate biome source 145 to 345 ms. Pin a worker to a world.
A terrain task takes (chunkX, chunkZ) and returns a transferred ProtoChunk payload. Nothing else crosses the boundary.
ChunkScheduler.tick has to ask for several chunks at once in this slice, or the pool is never more than one deep from the per-player path. It currently awaits recipient.deliver(x, z) one chunk at a time inside a 5 ms budget (ChunkScheduler.ts:83). Keep the budget, keep the round-robin between players, and let it have as many requests outstanding as the pool has workers. World.preload already batches 32 through Promise.all (World.ts:759) and needs nothing.
Build setup. Each package's vite.config.ts globs every non-test file under src/ into its own entry and preserveModules is on, so a new worker file needs no config change. worker_threads is already externalized in the root vite.config.ts. Two things to get right: every package is "type": "module", so the worker is an ESM worker; and the dev exports point at raw .ts, so under Vitest the worker URL must resolve to something Node can execute.
Depends on. Slices 5, 6, 9 and 10.
Prove it correct. The pool-size sweep this slice adds to worldgen.determinism, with digests identical at inline, 1, 2, 4 and 6 workers. The order-independence check still green at every pool size. Add a test that a worker throwing propagates as a rejected generateChunk rather than a hang.
Prove no output change. All four checks byte-identical at every pool size.
Report. Chunks per second at pool sizes 1, 2, 4, 6 and 8, best of three, and the main-thread CPU share while the pool is saturated. Expect the peak at five or six and a decline past that; that is this machine's shape, not a bug.
Slice 12: the rest of the pure statuses into the pool
What changes. biomes, structure_starts, structure_references and features run in the pool too. The main thread keeps the holders, the scheduler, the patch merge and the palette translation.
Files. packages/chunkgen/src/GenerationPool.ts, the worker-side pipeline entry, and the StepTasks wiring in DataDrivenGenerator.ts.
The content. A features task takes the center chunk's proto chunk and the eight neighbors' terrain proto chunks, and returns the center's updated state plus the eight patches. That is nine 192 KiB payloads in and one plus eight small patches out. At 100 chunks per second it costs the main thread about 4% of a core, which the measurements say is affordable. The serial alternative is not. A feature pass left on the main thread caps throughput at one over its own cost.
The neighbors' proto chunks must be copied rather than transferred, because the main thread keeps them. Copy cost is 42 microseconds per column.
Depends on. Slice 11.
Prove it correct. Same sweep as slice 11, plus the order-independence check, plus generator green.
Prove no output change. All four checks byte-identical at every pool size. The patch merge order is the thing that makes this true; if a digest moves, suspect the merge order before suspecting the workers.
Report. Chunks per second, and the split between worker CPU and main-thread CPU.
Slice 13: make the structure layout that survives slice 3 cheap
What changes. The two remaining structure costs, both measured, in either order.
Memoize jigsawBlocksOf. packages/structures/src/jigsaw.ts:329 rescans every block of a template looking for jigsaw blocks, and shuffledJigsaws at :460 calls it inside a loop over parent jigsaws, candidate elements and rotations with no cache. It is 20.9% self time in a CPU profile of a window containing a trial chamber. A template's jigsaw blocks are a function of the template and the rotation, so a Map keyed by both is the whole fix.
Give baseHeight a cheap column. Done. StructureLevel.baseHeight asked columnAt for a whole 384-block column and scanned it downward, which was 384 final_density evaluations, 384 aquifer calls and 768 allocations for one height. Measured, that was 50 column misses per chunk, 6% of the chunk's density samples and 16% of layout wall time; a desert pyramid alone asks for 441 columns. ChunkSampler.columnTop walks down and returns at the first block the predicate accepts, which is what baseHeight uses.
Aquifers stay on, against what this plan said before. NoiseBasedChunkGenerator.iterateNoiseColumn in the pinned 26.3 jar builds its NoiseChunk with the dimension's own fluid picker and the settings' aquifers block; the aquifersEnabled: false that older versions passed is gone. Turning them off would have moved 28 of 11,520 sampled columns by up to 20 blocks and dropped a cold ocean ruin 28 blocks through the sea floor.
While there, the column and sampler caches clear wholesale, columns at 16384 entries and samplers at 256 (javaStructureLevel.ts:57 and :72). Retain by chunk through the holder instead, and release with the holder.
Files. packages/structures/src/jigsaw.ts, packages/worldgen/src/chunk/ChunkSampler.ts, packages/prismarine/src/world/generators/datadriven/javaStructureLevel.ts.
Depends on. Slices 3 and 8.
Prove it correct. sampler byte-identical. The early exit must return the same height, not an approximation of it. The structures corpus in packages/structures/src/corpus.test.ts and the 12 placements in javaStructures.test.ts are unchanged.
Prove no output change. All four checks byte-identical, digests included. A memo and an early exit that change a block have a bug in them, not a trade-off.
Report. Layout milliseconds per chunk on both the 40-chunk walk and the 2-D fill, against slice 3's numbers.
Slice 14: make the terrain fill cheap
What changes. The per-block cost of buildTerrain. 227 ms for 98304 blocks is 2.3 microseconds each, and 55% of all CPU samples are inside density and Perlin evaluation. By this point terrain is about 90% of the per-chunk cost, so this slice and slice 15 are the only ones left that move the number. The target is 2.5 times.
Index the beard pieces by column first. createBeardifier (DataDrivenGenerator.ts:327, javaStructureBeard.ts:108) is the right operand of the top-level add in the overworld's final_density, so it runs on every one of the 98304 samples and scans the whole piece list each time. Measured per tile: 0.5 ms at no pieces, 11.3 ms at 10, 32.5 ms at 40, 125 ms at 76 under beard_thin and 220 ms at 76 under encapsulate. That is the largest single item in this slice for any chunk near a structure, and the pieces are fixed for the whole tile, so bucketing them by block column once per tile turns a list scan per sample into a short list lookup.
Then, in this order, measuring each. Skip the air above the column's surface rather than evaluating final_density for every block from y=320 downward. Iterate in the order the interpolated nodes' cell lattice wants, so the direct-mapped CornerCache never evicts within a chunk. Give Aquifer.computeSubstance the same treatment, since it is the second item after the noise. Keep every per-chunk buffer a typed array sized once.
Files. packages/worldgen/src/chunk/ChunkSampler.ts, packages/worldgen/src/chunk/Aquifer.ts, packages/worldgen/src/density/SampleContext.ts, packages/prismarine/src/world/generators/datadriven/javaStructureBeard.ts.
Depends on. Slice 11, so the gain is measured where it lands.
Prove no output change. sampler is the whole test here: six whole chunks, 98304 blocks each, bit for bit. Run it after every change, not at the end. Then all four checks.
Report. Milliseconds per chunk for buildTerrain alone, before and after each change, for a chunk with no structure near it and a chunk inside an ancient city's beard. One number for both hides the thing this slice is fixing.
Slice 15: the WebAssembly terrain kernel - abandoned
Do not start this. It was built, measured bit-identical against the JavaScript, merged, and then reverted. The reason is in ADR 0002 under what it measured once it was built: the premise under the whole slice - that terrain is about 90% of a chunk once the ladder has landed - is wrong. Terrain is 71 of 239 milliseconds. Six point four times on 30% of a chunk is about 1.25 times overall, for a second implementation of the density graph, the noise and the aquifer that would have had to stay bit-identical to the first forever.
What the measurement did find is that the decoration pass is 161 of those 239 milliseconds, which is why slice 12 - every pure status on a worker - is the slice that pays, and why the pool now runs the whole ladder rather than terrain alone.
Slice 16: the pool's lifecycle, and Ctrl+C
What changes. The server owns the pool, built at world load and terminated at shutdown. And the two defects that make a blocked server unkillable, which are independent of everything above and can land whenever someone has the time.
Files. packages/prismarine/src/world/WorldManager.ts and World.ts for the lifecycle; packages/prismarine/src/Console.ts and packages/server/src/Server.ts for the two defects.
The two defects. Console.ts calls process.stdin.setRawMode(true), and Node's tty documentation says Ctrl+C no longer raises SIGINT in that mode, so the keypress path is the only one that runs and it has no force-exit escape. And Server.shutdown opens with if (this.stopping) return;, so a second Ctrl+C does nothing at all. Give the keypress handler the same forceExitTriggered escape the SIGINT listener has.
Depends on. Slice 11 for the lifecycle half. The Ctrl+C half depends on nothing.
Prove it correct. A worker interrupted by terminate() stops in 2 to 24 ms. The server exits on a second Ctrl+C during a preload.
Prove no output change. All four checks unchanged.
The order, and what it buys at each point
Two columns, because structure layout costs 33 times more on a moving front than on a region fill. A fill is what a preload does; a front is a player walking in one direction, where every step drags a fresh column of chunks into the radius-11 structure window.
| After slice | What is true | Fill chunks/s | Front chunks/s |
|---|---|---|---|
| 1, 2 | Output is pinned and the two nondeterminism hazards are banned | 0.6 | 0.6 |
| 3 | A structure is rejected before it is assembled, not after | 1.0 | 1.0 |
| 6 | Each chunk's terrain is computed once | about 4.0 | about 3.5 |
| 7 | Output is a function of the seed and position, order independent | about 4.0 | about 3.5 |
| 8 | Each structure start is laid out once, at a radius-0 status | about 4.1 | about 3.6 |
| 9, 10, 11 | Terrain runs on six workers | about 19 | about 17 |
| 12 | Every pure status runs on six workers | about 19 | about 17 |
| 13 | jigsawBlocksOf is memoized and baseHeight exits early | about 20 | about 19 |
| 14 | The terrain fill is 2.5 times faster | about 42 | about 31 |
Rows 1 to 8 are arithmetic over measured parts. Rows 9 to 14 apply the measured 4.7 times and one target that is not a measurement: 2.5 times on the terrain fill.
These projections have since been measured and row 9 to 11 was wrong. A pool over terrain alone measured 1.29 times, not 4.7, because terrain is 30% of a chunk rather than the 90% this table assumes and the other 70% stayed on the server thread. With the whole ladder in the worker the measurement is 124.7 ms a chunk against 427.8 on one thread, which is 8.0 chunks per second. Slice 15 is abandoned; see above.
Risks, worst first
1. Moving the biome check in front of the builder is a deviation from Java's order. It is 87% of structure layout time and the single largest lever in this plan, and Java checks the biome at the generation point after assembling, which a pre-check cannot do. If a start's generation point can sit in a biome the start chunk's own quarts do not hold, a pre-check drops a structure Java keeps, and the world is quietly different in a way no digest in the corpus would show unless that exact chunk is in it. Slice 3's instruction is to measure how often that happens and to stop and report rather than ship if the answer is not zero. This is first because it is the only risk where the failure is a wrong world rather than a slow one.
2. The patch merge changes chunk borders by an unmeasured amount. The 169-positions-per-chunk figure that sizes it was taken before the biome filter fix cut the feature count per chunk. If the real divergence is larger, or lands somewhere a player sees (a lake cut in half, a village path stopping at a border), the parity claim gets harder to defend and slice 7 has to fall back on a positional order over feature passes, whose nine-deep colour chain stalls every teleport into new terrain. Slice 7's instruction to count and locate the changed positions is the early warning.
3. Terrain is now the whole problem and it has to get 2.5 to 4.5 times faster. After slices 3 to 11 it is about 90% of the per-chunk cost. 227 ms for 98304 blocks is where the target lives, and nothing has profiled that fill at the level slices 14 and 15 need. The hot path is also exactly where a reassociated float expression hides, so the two slices that have to deliver the factor are the two most likely to change a block.
4. The beardifier makes terrain cost several times more near a structure. createBeardifier is the right operand of the top-level add in the overworld's final_density (DataDrivenGenerator.ts:327), so every density sample runs the loop at javaStructureBeard.ts:108 over every nearby piece. Measured per 98304 samples, which is one terrain tile: 0.5 ms at no pieces, 32.5 ms at 40, 220 ms at 76 under encapsulate, which is an ancient city. At three new tiles per chunk that is about 660 ms per chunk inside an ancient city's beard against 1.5 ms away from everything, so the 227 ms the terrain rows assume is an average over a very wide spread, and the expensive end is where players go. Slice 6 takes it from three tiles to one; slice 14 has to index the pieces by column.
5. 4.7 times is this machine's ceiling, not a portable one. It was measured on an M1 Pro at load average 36 to 158, so it is a floor for that machine and a fiction for a two-core VPS. The design has to run with zero workers and produce the same bytes, which slice 11's pool-size sweep enforces. The chunks-per-second table does not travel; say so wherever it is quoted.
6. The worker may drag the server into its module graph. toChunk and javaGeneratedContent.ts reach BlockRuntimeIds, BlockEntityRegistry and bedrockLootPathOf, all of which live in packages/prismarine. Slice 9 keeps translation on the main thread so the worker should not need them, but the worker's pipeline entry still lives in that package and ESM evaluates whole subgraphs. Measure worker startup time and RSS in slice 11; if it is far above the 40 ms the research note predicts for a generator import, the subgraph is the reason.
7. The corpus landed while this plan was being written, and it is still moving. Run git status --short packages/parity before slice 1 touches anything there. The three family ids, the golden and packages/parity/src/generation/ are in the tree; the three row test files may not be. Writing a second corpus beside the one that exists is the failure mode here, and it costs a merge conflict in a committed golden that nobody can resolve by hand.
8. A new package is a published product. @jsprismarine/chunkgen needs a README, a Changeset, entries in the build and typecheck project lists, and from its first release its interface is a compatibility commitment. Slice 4 is where that is cheapest to get right and most tempting to skip.
9. serializeGeneration is load-bearing until slice 7. It is the only thing that makes today's feature ordering reproducible inside one process. Removing it earlier makes the corpus intermittent rather than wrong, which is the worst failure mode to debug.
10. WebAssembly is a second copy of the hot path. This is the risk that decided it: see slice 15, which was built and then reverted. Every later change to a density function has to land twice, in the same floating point association order, forever. That is why slice 15 is last and conditional rather than an obvious win.
11. Memory. Six workers at about 80 MiB each plus the retained holder working set is roughly 500 MiB. That is fine on 16 GiB and unknown on a small host, and the reference-counted retention replaces an LRU that at least had a hard bound.