Data Pipeline & Design Console — Plan¶
Document Type: Technical Plan Version: 0.1 (Plan only, nothing built) Status: Planned Last Updated: 2026-08-25 Vault destination:
02_Technical DocumentsRelated:[04_Factory & Resource System Design](<../01_Design Documents/04_Factory & Resource System Design.md>)·[08_Research Tree Design](<../01_Design Documents/08_Research Tree Design.md>)·[01_Technical Architecture Document](<./01_Technical Architecture Document.md>)
1. The Problem¶
Buildings, items and recipes link everything together. Once ~47 buildings and their recipes exist, the chain becomes impossible to hold in your head or verify by hand.
Recipe data is currently scattered across 04_Factory & Resource System Design and 08_Research Tree Design, with no single place to check consistency.
2. Core Principle¶
Never draw the production chain by hand.
A hand-drawn diagram desyncs from the data the first time a recipe changes, leaving two sources of truth and no way to tell which is right.
The graph must be generated from the data, every time.
This rules out Obsidian Canvas, Miro, draw.io, and plain markdown as the source. They are fine as outputs, never as the master.
3. Architecture — One Source, Four Outputs¶
┌──────────────┐
│ SQLite │ ← single source of truth
│ (design DB) │ authored offline
└──────┬───────┘
│ export scripts
┌──────────┬───────┴────────┬─────────────┐
▼ ▼ ▼ ▼
.tres JSON Markdown In-game
resources (visualiser) (vault wiki) recipe browser
(game) (from .tres)
Why SQL over a spreadsheet: foreign keys catch "this recipe references an item that does not exist" automatically. That class of error is invisible in a spreadsheet and painful to find later. Constraints do validation work that would otherwise be written by hand.
4. The Database Does NOT Ship¶
The game reads .tres resources, never the database.
| Reason | Detail |
|---|---|
| Desync risk ⚠️ | The architecture is authoritative host networking. Every client must hold byte-identical recipe data. Baked resources guarantee that. A database that can differ between machines, or be edited at runtime, is a deliberate desync vector. |
| Native dependency | SQLite in Godot needs the godot-sqlite GDExtension, with export templates per platform |
| Version control | A binary file produces useless git diffs. Design data is currently versioned and readable. |
| Editor integration | .tres gives the resource picker and inspector for free. A database gives neither. |
| No runtime need | A few hundred recipes loaded once at startup is a dictionary, not a database problem. |
5. Schema Sketch¶
Three core tables plus join tables:
- items — id, display name, tier, stack size, icon, description
- buildings — id, display name, footprint w/h, category, base power draw, base Mk range, machine Mk range
- recipes — id, building id, required machine Mk, craft time, power draw
- recipe_inputs — recipe id, item id, quantity
- recipe_outputs — recipe id, item id, quantity
- research_nodes — id, branch, credit cost, prerequisites
- research_unlocks — node id, recipe id or building id
Detail to be worked out at implementation time.
6. Validation — The Real Value¶
Errors that are invisible in a spreadsheet and obvious in a graph:
- Items with no producer (unobtainable)
- Items nothing consumes (dead end)
- Circular dependencies in the chain
- Buildings with no recipe
- Recipes referencing items that do not exist
- Recipes requiring a Mk level the building cannot reach
- Recipes unlocked by no research node (unreachable)
- Win-critical recipes behind optional research branches (see
25_Power System Design§8a.3 — this must never happen)
7. The Wiki — Nearly Free¶
Game wikis go stale because they are maintained separately from the game. This one cannot, because both are generated from the same rows.
Obsidian trick: if the generator writes one markdown file per item and building into the vault, using [[wikilinks]] for every input and output, then Obsidian's own graph view visualises the production chain with zero additional work.
Limitations: Obsidian's graph is undirected and clusters oddly, so it will not replace a proper directed graph eventually. But it is immediate and costs nothing.
The same generated pages become the in-game recipe browser content, and the basis of any public wiki later.
8. Console — Staged, and NOT UI-First¶
Do not build the editing UI first. CRUD screens are the most work and the least value here. DB Browser for SQLite already edits tables perfectly well, for free.
What cannot be bought off the shelf is the validation and the exports.
| Stage | Contents | Priority |
|---|---|---|
| 1 | Schema, seed data, validator, both exporters. No UI at all. | Highest value |
| 2 | Read-only console: browse, visualise the chain, run validation, see balance ratios | High |
| 3 | Editing UI — only if DB Browser proves annoying | Low |
| 4 | Wiki generation into the vault | Medium |
Stack: Node, already in use for scripts/generate-icons.js. better-sqlite3 handles this without a framework.
9. Separate Tool vs In-Game — Both, Unequally¶
| Tool | Where | Why |
|---|---|---|
| Chain visualiser and validator | Standalone web tool | A design-time activity. Wants to be open beside Obsidian, not inside a running game. |
| Recipe inspection while debugging | Extend the existing F3 debug overlay | Show a selected building's recipe, its inputs, and where they come from. An addition to existing tooling, not new software. |
10. Balance Work This Enables¶
The arithmetic that is currently impossible by hand:
- How many Smelters feed one Assembly Machine at a given rate
- Total power draw of a chain running at N items per minute
- Where ratios come out non-integer and irritate players
- Whether a Mk3 building's superlinear power draw (
25_Power System Design§8a.1) actually balances
11. Status¶
Plan only. Nothing built. Revisit when moving from design into implementation.
The first useful step is Stage 1, which requires no design decisions to be finalised — the schema can hold placeholder values and the validator works regardless.