Ingesting External Content

This engine is built to be a base: cloned, then loaded with maps, lore, and content on top — or run as a package on top of an existing campaign. Content that arrives from outside is raw material, not finished content. Until it is rebuilt into the engine's format — typed, chunked, named, tagged, anchored, linked — it is invisible to retrieval.

The rule was learned from the first large conversion. A setting's vehicles arrived as a single list file ("vehicles extant in the setting"); the /Vehicles/ domain was never generated from it, and the list could not be recalled as vehicles. Lists are signals, not storage. This chapter is the playbook that prevents that failure.

The companion instruction is Content Ingestion in /assistant-instructions/; this chapter is its authoritative, clone-safe copy.

The format to fit

Every concept becomes a typed file in the correct folder, following the standing conventions:

• Typed — the nearest type in the Ontology Map; never a foreign shape.
• Chunked — one file, one fact cluster, with summary + keyFacts as the recall surface.
• Named — recall-tagged file name (<Subject>-<tag>), disambiguated from any same-named sibling.
• Tagged — the tags vocabulary, so retrieval scores it.
• Anchored — canonLevel and era on every file.
• Linked — references over prose; no duplicated facts.

The ingestion pipeline

1. Inventory. List everything that arrived — folders, files, maps, pasted text. Classify each item's shape: typed files, aggregate lists, prose dumps, maps, foreign stat blocks, media.
2. Map to types. Choose the nearest type for every concept. Foreign fields are mapped or dropped (migration-map pattern, Migration and Lifecycle Conventions); never invent a type to avoid converting.
3. Atomize. Split aggregates: one typed file per entity. A list is a to-do list, not a chunk.
4. Format. Apply naming, chunking, folder, and truth conventions to every new file.
5. Anchor and link. Set canonLevel + era; write references to existing files; create missing targets instead of leaving draft refs.
6. Verify and index. Reference-integrity check; update the Knowledge-Index same session (single-writer, serial); recount with ls.

Source shapes: actions

Rules

1. Lists are signals, not chunks. "Vehicles extant in the setting" generates one vehicle file per vehicle. Same for item lists, creature rosters, NPC registers, tale catalogues — every enumerated entity is a chunk of its own type.
2. Maps generate places. Each named place on an imported map becomes a Location (settlement, landmark) or Region file with terrain tags, and the map's pins link to those files, not just an image.
3. Foreign fields are migrated, never carried. The source system's shape is data, not rules. Map what maps; drop what doesn't; rename what collides. A character from another system does not keep its foreign stats.
4. No draft refs. If ingested content references a place or a person, create that file — or drop the reference. Never leave a dangling draft (see Migration and Lifecycle Conventions).
5. Anchor everything. Default era to the World Profile's; default canonLevel to established, downgrade to rumor/myth when the source is hearsay or legend. Do not silently mark ingested claims canon.
6. No silent overwrites. Same-named concepts from the merged source get recall tags or renames, never silent replacement. Follow the legacy ladder when the base project's content collides with the incoming world's.
7. Pins outlive files. Map pins, tokens, and labels are separate content from location files: moving or retiring a location file does not remove its pins. Sweep, relink, repurpose, or remove each pin as part of the merge, and never leave a pin pointing at a retired slug.

Merging an existing campaign — the engine as a package

The engine is a module meant to be layered on top of a world or campaign that already exists — a live campaign hosted in another system, a wiki, a document set. The base project is not raw input for a blank rebuild; it is a living world the engine now hosts and answers against. The merge is only safe when the engine's own layer is differentiable from the systems already running the world — and that differentiation is what makes the rework callable.

• Provenance tag — source-ontology-project. Every file the engine creates or reworks as part of the merge — the World Profile, each chunked Lore file, each typed entity generated from lists, maps, prose, or foreign stat blocks — carries the provenance tag source-ontology-project. It marks the engine's footprint, so engine-reworked content can be told apart from the base system's untouched native files. Count it: source-ontology-project is the guide for "how much of this merge is done."
• The merge is callable, on demand. The tag exists to be called: a user (or a later session) asks "rework this world under the engine" or "merge this campaign in," and the steps run. There is no automatic trigger — a tag is a query surface, not a switch. When a merge is initiated:

1. Inventory. List the base project's folders, files, maps, and exports. Classify each item: typed files, aggregate lists, prose dumps, maps, foreign stat blocks, media.
2. Footprint. Ensure the World Profile carries the provenance tag; every file produced in the steps below gets source-ontology-project on top of its normal recall tags.
3. Rebuild under the ontology. Run the pipeline above — lists into typed chunks, prose into /Lore/ clusters, maps into linked places — anchoring everything to the profile's era and canon.
4. Keep the base world the authority. The World Profile's era, genre, and truth defaults rule; a base-format file is reworked into engine format rather than silently overwritten.
5. Verify. Sweep map pins; reference-integrity check; update the Knowledge-Index (single-writer, serial); recount with ls; confirm every merge-created file carries source-ontology-project.

After a merge

• Update the Knowledge-Index the same session: new file lines, counts, and the type lifecycle registry (single-writer — one serial edit pass, never parallel).
• Run a reference-integrity check; recount with ls before closing the phase.
• Sweep the maps for orphans: locate every pinned concept that points at a file that was renamed, retired, or deleted during the merge, and confirm no map still carries pins pointing at the source project's slugs.
• Record known-empty flag in the domain registry.

Where this fits

• Naming, chunking, and folders — Naming, Chunking, and Folders
• Linking and retrieval discipline — Relational Linkage and Retrieval Discipline
• Type mapping and relational guide — The Ontology Map
• Slug verification, no draft refs, single-writer, registry — Migration and Lifecycle Conventions
• Provenance tagging and merging — this chapter
• Existing-content handling — The Startup Contract and The Startup Checklist (legacy ladder)