# Build-knowledge — Editors + Captures

The rule that decides everything here: the UI is never the source of truth. Every useful thing in this layer is a shared operation first, and a React surface second — so the same crop, trim or sensor...


**The rule that decides everything here: the UI is never the source of truth.** Every useful thing in
this layer is a shared operation first, and a React surface second — so the same crop, trim or sensor
read is reachable from a Studio panel, an Action, a Workflow node, the Function Index, a toolbank and
MCP without being written twice.

If you are about to build an editor, a capture, or anything that turns a device into a file, read the
first two sections before you write code.

---

## The one seam

```
                    ONE OPERATION (an adapter function)
                                  │
    ┌──────────┬──────────┬───────┴────────┬──────────────┬──────────┐
  React      Action    Workflow node   Function Index   MCP tool  Chatbot fn
  editor   (curated)  {executor:...}   (harvestAdapters) (free)   (toolRegistry)
```

**An operation is an adapter function.** That single choice is why nothing else needed registering:
an adapter function is *already* an MCP tool (`adapterToServer`), a Function Index row
(`harvestAdapters` reads its `metadata.json` inputs as `from:'metadata'`, which is trusted and
therefore enforced), a workflow node (`{executor:{kind:'adapter', ref:'image.crop'}}`) and a chatbot
functioncall.

**It is NOT one Action per operation.** `app.js` already wrote that doctrine down: registering all
~1,700 adapter functions as actions "would bury the ~40 curated ones in a picker nobody could scan."
So the catalog stays adapter-side and exactly **one** curated action fronts it —
`run-artifact-op`, whose `operationId` is declared as `options: OPERATION_IDS`. Because that
descriptor is trusted, `validateInputs` enforces the list and a workflow node renders a `<select>`.

| Want to… | Do this |
|---|---|
| add an editing operation | add a function to a `@leumas/adapter-*` pack, then a row in `@leumas/editors`' `OPERATIONS` |
| add an editor | `defineEditor({id, accepts, operations, load})` in `@leumas/editors/editors` |
| add a capture source | a row in `CAPTURE_MANIFESTS`, then `defineCapture({...manifest, open, close, grab, load})` |
| add a sensor type | one row in `@leumas/devices/grobotics`' `CHANNEL_TYPES` — nothing else |

---

## Artifacts

An artifact **is a datacenter asset row plus one namespaced `artifact` key**. No new store, no
migration — `store.updateAsset` merges shallowly and `readMetadata` spreads `...a` last, so unknown
fields already round-trip.

```js
artifact: { v, kind, scope, expiresAt, sessionId, source, parents, op, media, filters, checksum }
```

- **`scope: 'scratch'`** writes to `.scratch/<sessionId>` with `bundleOf: 'scratch:<id>'` — which is
  the store's *existing* "do not list me on my own" flag, so scratch is invisible in the file
  explorer with zero store changes. TTL 24h (5 min for an abandoned session).
- **`parents` is a DAG, and derive NEVER mutates.** Undo is re-opening `parents[0]`; there is nothing
  in memory to lose on a reload, and no way for an undo stack to disagree with what is stored.
- `artifactId === assetId`. `artifactId` resolves through `inferEntity`, so every such prop gets a
  picker for free.

**The ingestion policy is `marketplaceMedia.js`'s**: size capped *before* decode, type sniffed from
the **decoded bytes** (never the filename), images re-encoded through sharp (which is what makes an
SVG structurally impossible to store), content-addressed names, quota checked before the write.

---

## Captures

| id | produces | permission |
|---|---|---|
| `camera.photo` | `image/webp`, `image/png` | `video` |
| `camera.video` | `video/webm` | `video+audio` |
| `mic.audio` | `audio/webm` | `audio` |
| `screen.video` | `video/webm` | `display` |

**No capture opens a device itself.** Everything goes through `@leumas/devices/media`, which owns the
permission latch, the ref-counted stream and the hung-prompt timeout.

### Four things that will bite you

1. **The latch keys are `audio`/`video`, not `microphone`/`camera`.** The latch's own `@param` docs
   are wrong about this and say so.
2. **Screen share must NOT use the latch.** Chrome shows the picker every call, and *cancelling*
   throws `NotAllowedError` — the same name a real denial uses. Latching it means the first cancelled
   picker kills screen capture for the session, behind a Retry pointing at a setting that does not
   exist. `screen.js` calls `getDisplayMedia` directly and keeps no latch.
3. **Photo and video share ONE camera** by calling `startBrowserCamera` with the same `source` key.
   Toggling modes never re-prompts, never blinks the capture light, and cannot hit `NotReadableError`.
4. **Always pass the WHOLE handle to `stopBrowserCamera`/`stopBrowserMic`.** A bare stream bypasses
   the refcount and takes every other surface on that device dark.

### A server-side "take a photo" action

`getUserMedia` has no server half, so `capture-image` pushes an **addressed SSE directive** and waits.
Three constraints, all mechanics rather than politeness:

- **Addressed, never broadcast.** `push({})` reaches *every* operator.
- **The browser never captures without a click.** `getUserMedia` outside a user gesture is refused by
  every browser, and that refusal is latched *sticky* — so a silent server-driven capture is not
  merely disallowed, it is impossible to implement correctly.
- **Bounded.** A workflow `wait` node caps at 30s, so a longer capture timeout buys nothing.

---

## Editors

`image` · `audio` · `video` · `text` · `code` · `json`, resolved by **specificity → priority →
registration order**. `text/plain` is legitimately claimed by both the prose and code editors;
priority decides and the operator can flip it once.

Every editor `load` is a **thunk**, always. `defineEditor` throws on a component, and `check:editors`
enforces it — passing one works, looks fine in review, and quietly puts that editor's whole
dependency tree on the registry's chunk.

**The browser never re-implements a pixel operation.** Crop, resize, rotate and the filters all run on
the server through `@leumas/adapter-image`; preview is CSS. `code.format` is declared
**unavailable** — there is no formatter in this repo, and a fake one returning the input unchanged is
indistinguishable from a file that was already tidy.

---

## Grobotics — DEVICE → CAPABILITY → CAPTURE

**Grobotics is not the capture. It is the provider; each sensor channel is a capability.**

```
Grobotics board → 80 channels → capability `grobotics-01/m1.c6` (`sensor.co2`) → read/record/stream
```

A capability is addressed by **channel position** with an operator label stored in Leumas. The
*semantic type* is the contract (`sensor.temperature`), the letter code is the wire — which is what
will let an Arduino or a remote probe answer the same `read-sensor` action later.

### [critical] Four traps that produce a plausible WRONG number, not an error

| Type | Trap |
|---|---|
| `sensor.temperature` | `mc` is **milli**-degrees (`raw×500000/1023 − 50000`). Read as a percent, 22 °C reports as 22000. |
| `sensor.co2` | `i` is an **index 0–100, not ppm**. A rule comparing it to 400 can never fire. |
| `sensor.soil`, `sensor.rain` | **Inverted** (`{1023,300}`, `{800,200}`) — a wet probe pulls the ADC *down*. Backwards, a downpour reads as a drought. |
| `sensor.button` | `s = raw <= high`, a boolean the firmware already computed — not a threshold map. |

And one in the wire format: **`t` is the timestamp on a frame and the TYPE on a reading**, whose own
timestamp is `ts`.

### Bounds are not optional

`STREAM` **cannot be turned off** — `streamEnabled = (v >= 0)` with `v` clamped non-negative, so the
default is hundreds of frames a second forever. Therefore:

- the recorder **decimates at ingest** (five numbers per capability, constant size at any rate);
- buckets close on the **wall clock** — `millis()` wraps at 49.7 days and resets on reboot;
- the line rate is capped and excess is **dropped and counted**, never queued;
- deadband + stability + minimum-post-interval run **in the session**, so the SSE feed, the recorder
  and the trigger lane inherit one answer. A TMP36 dithers ±1 °C, and an edge-triggered rule on a
  value sitting at its limit fires hundreds of times a second without this.

### Recording is fire-and-forget

A workflow `wait` node caps at 30s, so **start → wait → stop silently produces a 30-second dataset**
from a ten-minute request. The recorder finalises *itself* and fires `sensor-recording-finished`; a
second workflow reacts. Rows go to a dataset **asset**; metadata goes to a queryable **row** —
12,000 samples through `/db` is a denial of service on our own database.

### Thresholds are DERIVED triggers

One per semantic type, generated from the channel-type table over `device-telemetry` — which is
already edge-triggered with per-sink memory. **The session posts the CAPABILITY id as `deviceId`**,
because that trigger keys its edge on that field: the board's id would make all 80 channels share one
edge, and a channel that stopped firing would look like a broken sensor.

### The firmware

Leumas owns it now (`shared/engines/adapters/hardware/grobotics-firmware`). The shipped V3 image had
a **buffer overflow** — `appendReading` returned snprintf's would-be length, so `off` ran past a
192-byte array and the next call got an underflowed 16-bit `cap - off` (~65535) and wrote past it,
putting adjacent SRAM on the wire. Patched, buffer raised to 384, and **not compiled or flashed** —
`arduino-cli` is not installed; the arithmetic is proven by simulation.

---

## Guards

```sh
pnpm check:editors      # barrels isomorphic · subpaths resolve · no component as `load` · not on a boot path
pnpm check:devices      # the browser/Node split, the tab strip, driver honesty
pnpm check:props        # every routed verb declares its props
pnpm check:boot         # nothing new is eager
```

`check:editors --self-test` re-runs every matcher against deliberately broken input. Its own
self-test caught a regex that would have let a `./react` re-export through — a guard that passes
because its matcher is wrong is worse than no guard.


---
Source: shared/services/knowledge/build-knowledge/editors-and-captures.md
Canonical: https://docs.leumas.tech/p/how-to/editors-and-captures