Compaction and collection

Writes append. A table written in many small batches ends up as many small segments, and every update or delete leaves the blocks it superseded on disk until something reclaims them. Two background jobs handle both, and both are stepped and resumable — a browser tab can be backgrounded, throttled, or closed mid-job.

Compaction

Merges small segments into larger ones, which is what keeps a scan from paying per-segment overhead after a long run of small writes.

await db.compactTable("orders");

It runs automatically by default. To drive it yourself — a slice per idle callback, so it never holds the main thread:

const db = new MinnowDatabase(store, { autoCompact: false });

requestIdleCallback(async function step() {
  const progress = await db.compactTableStep("orders", { maxBlocks: 64 });
  if (progress.result === null) requestIdleCallback(step);
});

Each step processes at most maxBlocks output blocks and checkpoints; progress.result stays null until the job publishes.

Deletes and updates before compaction

Compaction is not what makes a mutated table readable at speed. A query applies the table's deltas over its appended data directly, so a table that has been deleted from or updated answers from the same scan every other table gets, plus the cost of the deltas themselves. What compaction adds is returning the table to a plain append — the deltas stop being re-read on every query, and the storage they occupy is freed.

One limit worth knowing: merging a keyed table plans the merge in memory, and a large table can need more than the default 32 MiB budget. Raise it with memoryBudgetBytes when a compaction reports CompactionMemoryBudgetError:

await db.compactTable("orders", { memoryBudgetBytes: 256 * 1024 * 1024 });

Automatic compaction backs off a table whose attempt fails, rather than retrying it on every query.

Jobs are records in the store, so listCompactionJobs() finds one a previous session left behind and resumeCompactionJob(jobId) picks it up. cancelCompactionJob(jobId) stops one cleanly — compaction is visible-data-neutral by construction, so cancelling it can never lose data.

Garbage collection

Reclaims blocks no live version references any more:

await db.collectGarbage();

A block is only collectable when no manifest, no open reader lease, and no non-terminal job still roots it. That is what makes it safe to run while a report is being read: an open snapshot scope pins its version, and collection skips everything that version needs.

The same stepped shape applies:

await db.collectGarbageStep({ jobId, maxItems: 128 });

Scheduling

Neither job needs to run on a timer. Reasonable triggers:

• After a bulk load, compact the tables it touched.
• On an idle callback, take one step of whatever is outstanding.
• At startup, resume jobs a previous session left and call cleanupQuerySpill().

Leaving them undone costs storage and scan speed. It never costs correctness — a database that is never compacted or collected still answers every query correctly, just over more blocks and more bytes than it needs.