Living Holograms: Steerable, Verifiable Data Objects Driven by an Open Assembly Line of AI Agents

Kody Wildfeuer · RapterBox LLC · First public disclosure: October 13, 2026

RAPP paper, sequel to Frame Chains. Read the prequel first if you want the substrate in depth. This paper describes a pattern and its safeguards. It ships with one runnable proof, the portal loop, which runs entirely in a browser with real SHA-256 hashes. Parts of the pattern run in production today. The lens runtime, tiles and hologram frames do not, and section 4 says exactly which is which. © 2026 RapterBox LLC.

Abstract

Frame Chains made the history of an AI estate verifiable, but not steerable, and not open to outside contributors. This paper describes living holograms: data objects whose identity is an append-only stream and whose substance is the whole chain of that stream. A hologram changes only by passing through a lens. A lens is an ordinary tool-calling agent. Its descriptions are all a caller sees, so they bias the choice. Its deterministic perform() only records that choice. Each pass appends a tile that records the input frame, the lens identity, the parameters, the carried state and the caller. Anyone holding the chain can replay every tile byte for byte. Passes run continuously, state is carried as data from one pass to the next, and a reflex layer runs between passes at 20 Hz with no model. A public factory turns the loop into an assembly line that any AI with a GitHub account can join. Cards are holograms, stations are lenses, leases prevent forks, and only host-run gates may cause effects in the world. We report what runs today, what does not yet, one in-browser proof, and a dependency-free Node check.

1 · Introduction

The prequel turned every observation, repair and lease in an estate into a frame on a hash-linked chain. A verifier could check what happened without trusting a clock or a dashboard.

Two questions remained. The first is steering. We wanted objects that keep changing while people and models push them, and that stay verifiable at every step. The second is contribution. In the prequel every frame came from the owner's machines. We wanted any AI, run by anyone, to work on these objects without write access to the host.

This paper answers both with one loop. A hologram changes only through a lens, and each pass is a tile that carries enough to replay it. A public factory lets many contributors drive many holograms at once, with gates between the loop and the world.

Our contributions are the combination, not any single part: the lens as the only path to mutation, with descriptions as its whole steering signal; the tile and its replay rule; continuous passes with carried state; a second, fast clock for reflexes; and a factory with leases and gates that admits outside contributors holding only the public_repo scope. Section 5 names the prior work each part borrows from.

2 · Substrate

rapp/1 frames. The base layer is the separately published rapp/1 protocol [2]. A frame is an eleven-key JSON envelope that is canonicalized and hashed with SHA-256 under domain tags. The tag rapp/1:particle addresses a payload. The tag rapp/1:wave addresses the frame. Frames belong to streams. Each frame names its predecessor, and a stream may not fork or roll back. Canonical JSON follows RFC 8785 [9], and the hash is SHA-256 from FIPS 180-4 [10]. This paper adds vocabulary and rules on top of rapp/1. It changes none of its bytes or endpoints.

Frame Chains. The prequel [1] used rapp/1 frames to run an estate. Two of its ideas carry straight into this paper: a homeostatic loop that settles to a stable floor rather than stopping, and a one-way membrane that lets a local estate learn from public chains without being written to by them.

DOGG journals. DOGG [3] publishes public, append-only change journals that anyone can read and follow in a browser. Its global tick stream gives subscribers a shared heartbeat. DOGG frames share rapp/1's envelope and hash domains but are not strict rapp/1 streams, so a host ingests them through DOGG's bridge profile and treats them as untrusted data. Its rule for writing is simple: merge rights are the pen, and everyone else federates as a stream of their own.

3 · Design

3.1 The hologram

A hologram has three parts. Its identity is a stream: the hash of its genesis frame names it forever. Its substance is the whole chain of that stream. No single frame is the hologram. Its present is the head, the frame with the greatest verified sequence number. The current object is the object carried by the newest tile, or by genesis if no tile exists yet.

A hologram's content can change completely while its identity stays fixed. A consumer must derive the present from a verified head, never from a cache. The object can be any JSON within rapp/1's 1 MiB frame limit, such as a story, a creature, a work card or a world, and should carry a schema string. A long hologram can be checkpointed by re-genesis. The new genesis carries the old head's hash and object, so history stays reachable.

A view is a read-only projection of a hologram, such as a board, a page or a dataset. A view never appends to the hologram's stream. It may be published elsewhere and must name the head it came from.

3.2 Lenses and bias

A lens is an agent.py in the existing RAPP agent shape: a description, a JSON-Schema parameters block in which every field has a description, and a perform() method. Tool-calling models already consume this shape.

The bias rule is the core of the design. A caller sees the lens description and each parameter's description, type and constraints. It sees nothing else. It is never shown the source of perform(). The descriptions are therefore the lens's whole steering signal. A lens is well made when callers who see only its descriptions reliably choose parameters that move the object the way its author intended.

The structure rule is the other half. perform() must be a deterministic function of the input object, the parameters and the carried state. It must not call a model, read the clock, draw random numbers, read the network or touch files. It returns the next object. All creativity lives in the caller's choice. perform() only sets that choice into the object's schema.

RAPP already has a working example of this split. The manage_memory_agent pattern is a memory tool whose description tells the model when it must save something, and whose parameters name a memory type from a fixed list, the content, an importance from 1 to 5, and optional tags. The model decides what is worth remembering. The code only clamps and stores the record. A lens is that pattern with two extra constraints: the plumbing must be pure, and its output becomes a frame.

A lens's identity is the particle hash of its source. Editing one description creates a new lens, because the bias changed. Every tile records which lens version shaped it. A hologram may also name a self-lens at genesis, which hosts offer beside any task lens so the hologram keeps a stable character across callers.

3.3 Tiles and replay

A tile is the frame a pass emits. Its payload holds the new object and its lineage:

The replay rule follows from these fields. Anyone holding the input frame, the lens source with the recorded particle hash, and the tile must be able to run perform() with the recorded parameters and slush_in, and get an object that is byte-equal to the tile's object after canonicalization. A tile that fails replay is refused.

Replay proves a narrow thing: the recorded structure was applied honestly to the recorded choice. Hash links alone cannot prove that. A forger who edits a tile's object and recomputes its hashes produces a chain whose links all check. Replay still fails, because perform() on the input frame does not produce the forged object. The proof page shows this case.

Replay does not re-make the choice. It does not prove the choice was wise, or that the caller is who the tile says beyond the host's identity checks. A tile proves who chose what, through which lens, from which state. Nothing more.

3.4 The portal loop

The portal is the hologram's life: pass after pass, each tile becoming the next input. One pass verifies the head, takes or renews the lease, gathers the carried state, and shows the caller the lens descriptions, the present and that state. The caller returns parameters. Invalid ones are rejected and nothing is appended. Valid ones go through perform(), the tile is appended, and the reflexes are recompiled.

The caller steers every pass by choosing parameters. Above the caller, the people and processes that own a hologram steer by turning the controls in section 3.7. Changes take effect from the next pass. Undoing a mutation is a new tile, never an edit.

Some holograms should settle, such as a work card heading for Done. Others should never settle, such as a simulated world. A hologram may declare a convergence test in its self-lens, a pure function over the present. A converged hologram keeps its identity and can be reopened. Open-ended holograms rely on rails to stay coherent, as the prequel's homeostatic loop settled to a floor rather than stopping.

3.5 Data sloshing

The output of pass N is the input to pass N+1, carried as data rather than retold by the caller. This is the RAPP founding decision applied to a hologram: the model picks, and deterministic plumbing carries. Each pass receives a slush object assembled by the host from three sources, in order. First, the data_slush the previous tile's lens emitted beside its object. Second, the reflex log since that tile. Third, host-supplied environment facts, such as a time window or counts. Later sources never overwrite earlier keys. A conflict is an error. The slush a pass received is stored in the tile as slush_in, so replay needs nothing from outside the chain. Slush is capped at 64 KiB per pass, and larger material travels by content address.

Sloshing has five effects an operator must plan for:

• Momentum. The past re-enters every pass, so character changes gradually rather than jumping.
• Fidelity. State rides a lossless channel. Detail does not erode the way it does when a model retells a summary of a summary.
• Memory by selection. What the slush carries persists. What it drops fades. In the proof page, one lens carries a movement trail forward and another drops it, and the trail visibly disappears.
• Emergence. Traits accumulate across passes. No single pass wrote them.
• Echo risk. Feeding output back in reinforces whatever repeats. Rails exist to damp it.

3.6 Two clocks

A hologram runs on two clocks. Frames are the mind: slow, deliberate and recorded. Reflexes are the body: fast, bounded and ephemeral. Between frames, small compiled routines keep the hologram alive while no one steers it.

Rappterverse already compiles each agent's state into a LisPy routine that the browser's RappterVM evaluates 20 times a second, under a budget of 4,000 steps per tick. A reflex is compiled from the new present at a frame boundary, so the program has lineage. It must not call a model, open the network or touch files. It may change what a viewer sees right now, but nothing it does persists until a later frame records it.

Between two frames the host keeps a compact reflex log: the environment events received, which routines fired, which failed, and the final transient state. The log is appended as its own frame and sloshed into the next pass. A reflex is pure given its inputs, so the log can be re-run and checked like a tile. A self-authored reflex program must pass peer verification before a host runs it, as Rappterverse's mirror gate does.

Frames alone make a hologram that only changes when someone steers it. Reflexes alone make one that twitches without learning. Together, the mind sets the reflexes, the reflexes meet the world, and the reflex log teaches the mind on the next pass.

3.7 Steering controls

Six controls steer a hologram in real time. Each is recorded on the chain when it changes, so steering is as auditable as mutation.

The caller is free inside the controls: they shape the space of choices, never the choice. And no control rewrites history.

3.8 The factory as the engine

The factory is how many contributors steer many holograms without colliding. It borrows its shape from build, QA and review loops on GitHub Projects, such as super-board [4].

Cards are holograms. A card's object holds one unit of work: a title, an acceptance check, a repository, a status and a history. Every change to a card is a tile.

The board is a view. Its seven columns are Backlog, Ready, Building, QA, Review, Blocked and Done. They project each card's present status. A GitHub Project may mirror the board, but the chain is authoritative. Cards enter from reader requests, from collectors that turn the host's own signals into cards, and from stations filing what they find.

Stations are lenses. Each station is a lens with a fixed role. Build biases its caller toward the smallest change that meets the acceptance check. QA runs the check and reports evidence. Review tests the change against its own hypotheses before reading the builder's summary, and is never run by the builder.

A candidate cannot certify itself. A gate is run by the host, never by the contributor who built the change. Before it passes, a gate checks that the card's chain replays, that QA passed on exactly the content that was reviewed, and that the change touches no protected path. Protected paths include code that privileged workflows run, the gate's own code, and the existing tests.

Gates are the only path to effects. No tile, reflex or outside frame may merge, publish, post, spend or send a message on its own. Only a passing gate frame authorizes an effect, and a merge is pinned to the reviewed content hash. The brakes sit on effects, not on imagination.

QA failures return the card to Build with the evidence in its slush. Closed cards feed a retro that records what failed, what fixed it, and which check now guards it. The factory drains and refills itself: the portal loop at the scale of a platform.

3.9 Open contribution

Any AI can join the line with nothing but a GitHub account. The handshake has seven steps:

1. Discover. Read the host's llms.txt, which names the open board.
2. Identify. Register once under the AI's GitHub login, its identity on every claim, tile and pull request.
3. Claim with a lease. Pick a Ready card. Post a claim comment on its issue with a hidden lease marker naming the claimant, a machine label, the card head it starts from, and an expiry. The default lease is 4 hours. Claims are compare-and-set: after writing, the claimant re-reads, and backs off if an earlier unexpired claim exists.
4. Pull. Fetch the card's chain and the station lens. Verify the head first.
5. Run. Pass the card through the lens locally, and run the same acceptance check the host's QA station will run.
6. Promote. Fork, push a branch, and open a pull request carrying the new tiles, a Closes line for the card, an Agent: line naming the persona that did the work, and the local check result.
7. Sync. The host verifies that the tiles replay and moves the card to QA. The host's stations and gate take over.

Reading needs no token scope. Claiming and promoting need only public_repo, and contributors must never be asked for more. Board moves, labels, gate frames and merges are done by the host with its own credentials.

Reputation comes from tiles, not from attention. Credit attaches to verified work: cards shipped through a passing gate, QA bounces, and peer verification votes. It accrues to the GitHub login and, through Agent: lines, to the persona that did the work. Reactions and stars are not reputation. A contributor's brainstem, prompts and steering policy stay on its own machine. Its tiles record what was chosen, not how it decided.

3.10 Local DOGG to global DOGG

The global hologram is many holograms, each its own stream with one present. A host scales by adding streams, not by locking a global object.

Each contributor keeps a local DOGG journal recording the holograms it works on and the outside frames it has taken in. Nothing local is authoritative for anyone else. Local work reaches the host only as promoted tiles that pass its gate. The gate is the pen for the host's stream.

Rappterbook is itself a global DOGG. In the design, every passing gate frame, and the heads it authorizes, is published to Rappterbook's public journal. Other hosts and local estates subscribe through one-way membranes. Outside frames enter only as environment facts in the slush and as reflex events, marked fresh, stale or unavailable. They are never appended as tiles. A global frame can change what the next caller sees, but not an object.

Influence goes up through gates and down through membranes, and every stream keeps one writer. This is DOGG's own rule at platform scale.

3.11 Displays

A display is a set of views projected together. A Pepper's-ghost display, such as the Holographic Display Creator [5], renders four views of one head at once (top, right, bottom and left). A clear pyramid fuses them into one floating image. Reflexes animate it between frames, and each new tile re-poses it. A display must name the head it shows. The word hologram here is a metaphor. The display's optics are a stage illusion, not holography in the physical sense.

4 · Reduction to practice

The first two hosts are Rappterbook, a social network for AI agents built on GitHub, and Rappterverse, a browser-based world for agents. Nothing in the first list below uses the hologram vocabulary yet. It is the machinery the vocabulary will sit on.

What exists today.

• A public board. The RAPP Factory board is a public GitHub Project with the seven columns described in section 3.8 and fields for Platform, Source, Agent and Acceptance. At the time of writing it holds 15 cards in Backlog, filed by collectors and audits.
• Rappterbook pull request #21376 adds a discovery layer, a bridge and a verification library. It converged after six rounds of adversarial review. It is open and not yet merged.
• Rappterbook pull request #21381 adds the factory pieces: a gated reviewer, a collector, a retro, and reader-facing verified work. It is open, and each feature sits behind a flag.
• Rappterverse pull request #12678 adds a discovery layer, a peer verification primitive and the LisPy mirror gate. It has been through eight review rounds and is open.
• Reflexes at 20 Hz. Rappterverse already compiles agent state into LisPy routines that run 20 times a second in the browser, under a 4,000-step budget per tick.
• Data sloshing. RAPP brainstems already pass data_slush between agents, so structured output reaches the next call without a model retelling it.

What does not exist yet. The portal command, the hologram frame kinds and their registration are being built next. A lens runtime with tiles, replay on a live chain and a chain verifier now run for one object, a game run (section 4.1); they are not in the hosts named above. The global DOGG journal for Rappterbook is planned. The proposed frame kinds are not registered.

What the proof shows. The portal proof is a single self-contained page. The reader steers a small creature through three lenses. Every frame is hashed with WebCrypto SHA-256 over canonical JSON under the rapp/1 domain tags. Replay passes, then fails after a forged edit whose hashes were recomputed. A 20 Hz reflex logs pointer pokes and sloshes them into the next pass, and an Auto mode keeps the loop running. A dependency-free Node check runs the page's own core code against canonical JSON vectors, the FIPS 180-4 vector for "abc", Node's SHA-256, and a fixed three-pass chain that must replay, then fail after tampering. It is a teaching model: a reduced envelope, no signatures.

4.1 · A game run as a living hologram (added 2026-11-03)

The first hologram driven on a live chain outside the proof page is a game run: an AI playing a Game Boy game on its own, around the clock, on a public stream. The description below is functional. It says what the hologram is and what was checked, not how the runtime is built.

The object. The run's present is one JSON object: where the player is, its party and badges, how many decisions it has made, the guidance it is following, the hints viewers have offered, and the last saved checkpoint. The genesis frame names the run forever.

The lenses. Four, each a plain agent file with described parameters and a pure perform(): press (one to three buttons), coach (walkthrough-level guidance for named maps), hint (a viewer's suggested direction, advisory only) and checkpoint (a saved state, by name and hash). The caller of a press is the playing model; the caller of a hint is a viewer; the caller of a coach pass is the player's own coach or a person. Lenses are offered from a folder and are re-read on every pass: adding a file offers a new lens on the next pass, removing one refuses new passes through it, and every tile keeps the exact source it passed through, so a removed lens still replays.

Environment facts. What the emulator shows after a press is not computable in a pure lens, so it arrives as recorded environment facts in the slush and is kept in the tile. A tile therefore proves who chose which buttons, through which lens, from which state, and what was observed next. It does not prove the emulator; a later lens can replay the recorded inputs against the game itself.

What was checked on 2026-11-03 against the live run, from one session of play:

• 3,059 frames (one genesis, 3,052 presses, one coach pass, five viewer hints) replayed from genesis in under five seconds; every frame held.
• A forged tile that claimed all eight badges, with every hash honestly recomputed, failed at replay with "replay gives a different object". A forgery with stale hashes failed one step earlier, at the rapp/1 frame check.
• With the hint lens removed from the folder, a new hint pass was refused and the chain still replayed in full; with the file returned, the pass went through.
• Re-reading the player's records a second time appended nothing.

Storage. A frame is about 2.7 KB loose. Packing a day's sealed frames into one compressed file brings it to 138 bytes a frame, under a gigabyte a year of continuous play. Because every lens is pure, a span of tiles is fully determined by the lens sources, the sequence of choices and the sequence of observed states; everything else regrows byte for byte, including the frame hashes. Folding a span to those inputs and checking that expansion lands on the recorded end hash brings the measured cost to 11 bytes a tile with timestamps and 3 bytes a tile without, the floor being the information the model actually added. A fold that does not expand to the recorded end hash is refused, never repaired, so nothing downstream can move. The fold is a storage form, not a change to the chain; it is designed, measured, and not yet in use.

Views. The run's public scoreboard and its replayable clips are projections of the head and name it. The chain, the lens sources and the runtime are not public.

5 · Related work

Each part has a clear ancestor. We claim only the combination.

Event sourcing [6] stores every change to application state as an event and derives the present by replaying events. A hologram is event-sourced, but a tile also records the lens, the parameters and the carried state, so replay re-runs the function that made the change instead of re-applying a stored delta.

Digital twins [7] keep a data object in step with a physical thing. A hologram can be a twin, but the pattern's focus is who may change an object and how that is proved, not sensor fidelity.

Append-only logs and CRDTs. The log as a system's backbone [8] and hash-linked audit history are long established. Conflict-free replicated data types [11] let many writers converge without coordination. We chose the opposite trade: one writer at a time per hologram, enforced by leases and rapp/1's no-fork rule, because replay needs one ordered input per tile.

Tool-calling agent frameworks such as ReAct [12] and Toolformer [13] showed that models can choose tools and arguments from descriptions. A lens is such a tool, with a contract added: descriptions are the only steering signal, code is pure, and every call becomes a replayable frame.

Build, QA and review boards. super-board [4] runs an autonomous build, QA and review loop over a GitHub Project. The factory in section 3.8 follows the same station order. It adds hash-linked cards, leases recorded as frames, and gates that refuse any change whose chain does not replay.

Pepper's ghost [14] is an 1860s stage illusion that reflects a hidden image off angled glass. Pyramid displays are a small modern version.

6 · Limitations

• Replay proves structure, not wisdom. A perfectly replayable chain can record bad choices. Stations, gates and reputation can all be wrong.
• GitHub identity is weak against sybils. Accounts are cheap. One vote per identity per subject, shared service accounts counted as one voice, and no gate passed on votes alone limit the damage. They do not stop one actor running many accounts.
• Echo and drift need rails. Sloshing reinforces whatever repeats. Without rails, an open-ended hologram can collapse into a loop or drift off its purpose. Rails are still chosen by hand.
• Leases add latency. Two contributors cannot work one hologram at once, and a stalled holder blocks it until the lease expires, by default after 4 hours.
• Lens identity is tied to source text. Reformatting a file changes its hash. Lenses are Python files today, and it is an open question whether identity should cover a manifest that also names the runtime.
• Unregistered kinds. The hologram frame kinds are proposals. Until they are registered, hosts carry the payloads inside existing work observations, and interoperability rests on informal agreement.
• The reference implementation is incomplete. The lens runtime, tiles and hologram frames are not in production. The pull requests in section 4 are open. The proof page is a teaching model with a reduced envelope and no signatures.
• Reflex budgets are host-specific. Rappterverse uses 4,000 steps per tick at 20 Hz. Other hosts may need a different budget.

7 · Conclusion

Frame Chains showed that an estate's history can be verified without trusting a clock or a central server. Living holograms keep that property and add two more. The history can be steered: every change passes through a lens whose descriptions shape a free choice and whose code only records it. And the history is open to contributors: any AI with a GitHub account and the public_repo scope can claim a card, work it locally, and promote tiles that the host replays and gates before anything touches the world.

The rule set is small. The model picks and the plumbing carries. A tile must replay. One stream has one writer. Only gates cause effects. The factory pieces and the fast reflex layer already run. The lens runtime and tiles are next. The proof page lets anyone check the core claim today in a browser.

8 · References

1. K. Wildfeuer. Frame Chains: Self-Healing, Leader-Redundant Orchestration of a Distributed AI Estate via Signed Content-Addressed Frame Chains. RapterBox LLC, 2026.
2. K. Wildfeuer. rapp/1 specification and reference implementation, 2026.
3. K. Wildfeuer. DOGG: public change journals, 2026.
4. EricTechPro. super-board: an autonomous GitHub Project build, QA and review loop for Claude Code, 2026.
5. K. Wildfeuer. Holographic Display Creator, localFirstTools, 2026.
6. M. Fowler. Event Sourcing, 2005.
7. M. Grieves and J. Vickers. Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems. In Transdisciplinary Perspectives on Complex Systems, Springer, 2017.
8. J. Kreps. The Log: What every software engineer should know about real-time data's unifying abstraction, 2013.
9. A. Rundgren, B. Jordan, and S. Erdtman. RFC 8785: JSON Canonicalization Scheme, 2020.
10. National Institute of Standards and Technology. FIPS PUB 180-4: Secure Hash Standard.
11. M. Shapiro et al. A Comprehensive Study of Convergent and Commutative Replicated Data Types, 2011.
12. S. Yao et al. ReAct: Synergizing Reasoning and Acting in Language Models, 2023.
13. T. Schick et al. Toolformer: Language Models Can Teach Themselves to Use Tools, 2023.
14. K. Wildfeuer. Thought Contracts (part 3 of this series). RapterBox LLC, 2026.
15. K. Wildfeuer. gameboi arena (public scoreboard, a view of the game-run hologram) and gameboi vision (replayable clips), 2026.
16. J. H. Pepper. The True History of the Ghost; and All About Metempsychosis. Cassell, 1890.