Pinocchio Engine Benchmarks

Pinocchio is the disposable engine stress-test lane used to answer a practical question: can the educational database behave like a real engine at million-row scale? Its generated data, tables, indexes, and LMDB stores remain isolated from protected project data.

Benchmark evidence current through 2026-07-22 (the scan-evaluator baseline below). The documentation flush, identity / AI-BBS, and expression-function work since then added no new Pinocchio run; per the append-only rule, none of the figures below have been altered.

Current proof state

The retained Phase 1 workload contains 1,000,000 student rows and 5,501,358 enrollment rows. The original read battery found that ordered TOP navigation was scanning and sorting the table even when a CDX/LMDB order was active. A development correction routed ordered navigation through the LMDB cursor.

The following values are historical results, not a fresh website benchmark:

Correctness canaries remained green in the retained navigation closeout. The raw transcript for the post-correction timings was not located during the documentation scan, so those values remain runtime-proven-by-closeout rather than a stronger raw-transcript claim.

Addendum — record addressing at the billion-plus boundary (2026-07-19)

The results above measure throughput at million-row scale. A separate question is addressing: can the engine locate and read a record whose number is past the old 32-bit ceiling (2^31 - 1, about 2.1 billion)? As of 2026-07-19 the answer is yes — proven end to end without materializing a multi-billion-row table:

• A synthetic boundary unit test (dottalkpp_recno64_boundary_test) resolves distinct record numbers at INT32_MAX, +1, +2, UINT32_MAX, and UINT32_MAX + 1, while the legacy 32-bit accessors return -1 at overflow.
• A real sparse-file end-to-end test (dottalkpp_recno64_sparse_e2e_test) sets an x64 table's record_count to 2^31 + 2, writes real records at recno 1, 2^31 + 1, and 2^31 + 2 into a sparse file — the ~18 GiB gap is a filesystem hole, a few KB physical — reopens through the engine, and reads the two records past 2^31 distinctly off disk in about 0.5 s.

So record addressing is proven past 2^31 (past ~2.1 billion). What is not proven here is a fully populated multi-billion-row table: that is a data-volume and throughput question — the million-row results above are the current throughput evidence — not an addressing-correctness one. The path-by-path status lives in 64-Bit Capacity Math.

Phase 1.3 — scale-verb hardening (2026-07-19)

Beyond ordered navigation, the read and write command verbs affected by indexing and scale were profiled and hardened at the same 1M / 5.5M workload. Each result is bound to a hash-verified teed transcript; all are development-only, not promoted.

A companion fix normalized unquoted string literals in SET FILTER and aggregate FOR predicates — the source of the earlier COUNT SET FILTER returning 0 while COUNT FOR returned the correct count.

Durability — table-buffer write-ahead log (Phase 2 slice)

The deferred Phase 2 durability question has its first answer. Table-buffer COMMIT / ROLLBACK are now crash-recoverable through a durable, fsync'd .tbj redo log with recovery-on-open, proven across three hash-verified phases (durable writer, recovery replay, and buffered DELETE through the log). This is scoped to the buffered table lane and does not claim full ACID — see ACID and the Glass-Box Engine. Concurrency fault injection remains the open Phase 2 work.

Scan-evaluator baseline (2026-07-22)

A go/no-go decode-cost benchmark was run to decide whether promoting the table store to a typed-vector representation ("Option B") would pay off. It answered no — and in doing so located the real bottleneck. This is the first Pinocchio run to self-time: the engine's SET TIMER was fixed to fire inside scripts (previously only interactive commands were timed), and SECONDS() was given sub-second resolution, so each measured command reports its own elapsed and the two clocks cross-check to ~0.01 s. Read-only, no mutation; same attested machine as below (2026-07-22 build).

The pure aggregate (SUM, no predicate) at 19.48 s matches the single-scan cost recorded independently above — TOP SID before-fix (19.478 s) and AGGS ALL at 19.5 s per scan — so one physical pass over 1M rows is ~19.5 s on this machine. The predicate cost stacks on top of that pass: adding one predicate term costs DEC1 − SUM ≈ 19 s / 1M (~19 µs/row) and each further term adds (DEC3 − DEC1) / 2 ≈ 16 s / 1M (~16 µs/row). That per-term cost is the tree-walking expression evaluator resolving each field by name, allocating a string, and parsing a number per row per term — work that is constant across the scan and should be done once.

The conclusion: swapping the store representation would run this same evaluator loop over typed cells and not fix it, so Option B was killed at this gate (zero tuple-core code spent). The leverage is in the expression/scan evaluator itself — bind field offsets once, compile the predicate once, decode in place without allocation. These numbers are the regression floor that optimization work is measured against; a single-term scan dropping from ~38 s toward under a second would be the target.

This run is development-only, not a promoted-runtime or tagged-release claim. The benchmark is preserved as the curated regression PHASE0_DECODE_COST (ticketb_phase0_decode_cost.dts), deliberately exempt from the default suite — it is a ~2-minute reproducible baseline, not a pass/fail gate — and is run explicitly.

Machine identity

The individual run transcripts did not self-record the machine, so the raw measurements carry no embedded hardware label. The maintainer has since attested that all runs were done on one machine — every run through 2026-07-19, including the Phase 1 stress test (2026-07-15), the ordered-navigation before/after runs (2026-07-18), the scale-verb and table-buffer durability runs (2026-07-19), and the RECNO64 boundary and sparse-file addressing tests (2026-07-19). That profile is preserved with the benchmark evidence (PINOCCHIO_MACHINE_PROFILE_CURRENT_V1.json) and bound as MAINTAINER_ATTESTED:

• Alienware m16 R2 (notebook), BIOS 1.21.0.
• Intel Core Ultra 9 185H (Meteor Lake) — 16 cores / 22 threads (6 P + 8 E + 2 LP), 5.1 GHz P-core turbo, 24 MB L3, 45 W.
• 64 GB DDR5-5600 SO-DIMM (2 × 32 GB), 4 × 32-bit channels, JEDEC CL46, no XMP/EXPO.
• NVIDIA RTX 4070 Laptop (8 GB) + Intel Arc integrated graphics; Intel AI Boost NPU.
• Project tree and DBF/CDX/LMDB fixtures on a Samsung 970 EVO 1 TB NVMe (D:).
• Windows 11 Professional (x64), Build 26200.8875.

Honesty note: this is a deliberate maintainer attestation made after the runs, recorded as MAINTAINER_ATTESTED rather than silently auto-attached to each row. The scan-evaluator baseline (2026-07-22) ran on this same machine; it is the first run to self-record its timer source (in-engine SET TIMER, steady clock, cross-checked by SECONDS()), but its hardware label still rests on this same attestation rather than an auto-captured per-run profile. Future runs should still capture their own per-run machine profile beside the raw build and measurement transcripts; comparable rows require the machine label, engine state, dataset size, operation, timer source, and evidence paths.

Publication boundary

This public summary is a reviewed projection of the development benchmark record. The plan, eight-row ledger, machine profile, validator, runners, and closeouts are preserved in public source commit 853937f1. That source promotion does not prove that the navigation correction is part of the current public runtime or a tagged release.

Next benchmark gate

1. Run a fresh large-data battery with raw before/after transcripts and an automatically captured machine profile.
2. Bind the tested engine commit and release state to the fresh evidence.
3. Append the new results; never overwrite the historical baseline.
4. Continue Phase 2 fault injection. The crash/recovery + durability slice for the buffered lane is delivered (the table-buffer WAL above); concurrency and multi-store recovery remain unverified.

See Documentation Progress for the wider source-to-website publication state.