Engine Architecture

Principles

Runtime truth first - command behavior and source boundaries matter more than aspirational product language.
Object-oriented hybrid runtime - DbArea, XBaseEngine, workspace, memo, index, relation, and metadata objects carry state while preserving DBF-family interoperability.
Visible database layers - tables, records, fields, indexes, relations, work areas, metadata, HELP, and validation are teaching surfaces.
Separation of concerns - table storage, memo payloads, indexes, command handling, metadata, HELP, validation, and UI lanes are separate responsibilities.
Proof language - distinguish runtime-proven behavior from active, partial, experimental, and canary work.
Co-development - the implementation and documentation system are developed together. SelfDoc and the Master Documentation Organizer are part of the architecture, not a separate publishing chore.
Open cross-platform architecture - the runtime should remain portable across Windows, WSL/Linux, Ubuntu, and macOS while keeping UI and index integrations behind open boundaries.

Canonical diagram

Diagram attachment: DIAG-ARCHLAYERS-009

Canonical source: docs/manuals/assets/diagrams/dottalkpp_architecture_layers_v1.svg

Select the diagram to inspect the full-resolution SVG. This is a system-context map rather than an entity-relationship model: its arrows describe authority, execution, evidence, consumption, and reviewed promotion. Status labels keep active runtime behavior separate from alpha, partial, experimental, and reserved work.

The most important boundary is that the Laboratory Campus consumes and organizes DotTalk++ and x64base evidence. It may launch, sequence, explain, and teach the runtime, but it does not redefine engine or command behavior. The curriculum is an evolving crosswalk and collection of proof-backed paths; it is not yet a fixed seven-module syllabus.

Database core and teaching system

The database engine is the core architecture. Tables, records, fields, work areas, indexes, memos, relations, buffering, locking, validation, and mutation behavior are the project root.

DotTalk++, SelfDoc, MDO, manualgen, diagrams, and the Laboratory Campus are the analysis and teaching architecture around that root. Their purpose is to keep the engine observable: commands can be run, behavior can be proved, contracts can be collected, documentation can be generated, and lessons can remain tied to runtime truth.

This is why the documentation system is part of the architecture. It is not only a website workflow. It is the mechanism that keeps the database work, the teaching work, and the analyst work aligned.

Major subsystems

The local project is organized around these implementation areas:

src/xbase - table runtime, DbArea behavior, cursor state, field handling, locks, record views, and the DBF flavor trinity.
src/xindex - INX/CNX/CDX/LMDB index infrastructure and command bridges.
src/memo - memo store, memo manager, memo object, memo reference, and memo canaries.
src/cli - DotTalk++ command surface, HELP, browsers, workspace commands, DDL, DrawIO, validation, and scripts.
src/datadict - data dictionary readers, object resolution, catalog paths, and catalog bridge work.
src/xexpr and src/cli/expr - expression and function evaluation.
bindings/pydottalk - Python binding work and smoke probes.
docs/gui and GUI source lanes - GUI-neutral services, wxWidgets frontend work, Python/Tk preview tools, and TUI alignment.

Workspaces and areas

Workspaces are wrappers over active areas. An area is a DbArea-style object that owns the active table state, cursor position, selected order, relation hooks, field metadata, memo bindings, and validation context.

This matters for documentation and APIs:

workspace save/load is object state management.
scripted workspace operations should describe the area objects they affect.
GUI panes should bind to workspace and area state instead of duplicating table state.
relation and index views should be derived from active area objects.

DBF flavor trinity

The local headers document three active storage dialects: classic/MS-DOS DBF, Visual FoxPro DBF, and x64 DBF_64. Classic and FoxPro-like files are bridged through the VFP descriptor layer; VFP files carry newer field/data type compatibility; x64 files add large-header metadata, vector table/field names, and fallback descriptor mangling.

See DBF Flavors and Index Defaults.

Memo and field extension model

The memo subsystem is object-oriented: memo stores, memo managers, memo objects, and memo references need their own documentation lane.

In addition to student code hooks, x64base exposes custom field types through a register-once codec registry: a type registers its type code, codec, on-disk width, and flavor eligibility once, then flows through CREATE, validation, and storage with no core patch. The built-in binary types (I/B/Y/T) use the same registry. Custom field types let education labs, domain experiments, and downstream products add semantics such as validators, display formatting, import/export rules, and editor behavior without patching the core DBF runtime.

Open API boundaries

The engine should maintain explicit open boundaries for:

Open Index API - index creation, rebuild, verification, lookup, order selection, and backend-neutral lifecycle calls.
Open GUI API - GUI-neutral services for table browsing, workspace inspection, command execution, validation reports, and editor/view synchronization.
DotScript - repeatable command files over the same shell/runtime concepts.

Student and Custom Index Extension Lane

The source reserves index space for students, researchers, and downstream users to experiment without colliding with the core INX, CNX, and CDX/LMDB families. The current surface is intentionally mixed-status:

Surface	Current evidence	Status
`SIX`	Registered command with `CREATE`, `BUILD`, and `INFO` over the local single-tag index stub.	Active educational stub
`SCX`	Student/local compound index-file lab family and the current second custom slot.	Active educational stub
`REINDEX CUSTOM`	Explicitly runs `SIX + SCX`; `REINDEX ALL` excludes them by design.	Active opt-in dispatcher
`SNX`	Header/container/catalog groundwork exists, but `edu_snx.cpp` exports no command and shell registration is disabled.	Reserved and partial
User-supplied backend attachment	Intended adapter boundary for custom indexing systems through the Open Index API.	Planned contract

This proves an extension laboratory and reserved namespace, not yet a universal plug-in ABI. The goal is for a student or user to implement an index family, attach it through a documented lifecycle interface, exercise it against a DbArea, compare behavior and timing, and preserve results as proof artifacts without changing the core index guarantees.

Current build surface

The top-level CMake project is DotTalkpp version 0.6, C++20.

Important options:

DOTTALK_WITH_TV
DOTTALK_WITH_GUI
DOTTALK_WITH_WX
DOTTALK_WITH_INDEX
DOTTALK_WITH_EDUCATION
DOTTALK_WITH_RELATIONS
BUILD_PYDOTTALK
DOTTALK_PROFILE

The current Windows release payload keeps dottalkpp.exe, lmdb.dll, and sqlite3.dll separate. LMDB and SQLite are streamlined dynamic dependencies; the larger dottalkpp.exe is the compiled command/runtime host containing the shell, engine services, scripting, HELP/SelfDoc, metadata, browser, relation, workspace, and TUI-enabled command surfaces.

See Runtime Footprint and Dependencies.

Current focus

The current useful work is not broad feature expansion. The local tree points toward validation, congruence, GUI/runtime bridging, metadata reports, CDX/LMDB lifecycle cleanup, memo proof, and keeping browser/workspace smoke paths honest.

For the index teaching model, see CDX and LMDB Indexing. CDX is the logical container and command-facing manager; LMDB is the physical backend used for built index environments.

See Current Project Truth.