# Architecture Goal: develop and tune almost everything in a desktop browser without redeploying to the phone. ``` ┌────────────── UI (web stack, same in browser and in the app's webview) ──────────────┐ │ Sites · Setup · Capture · Review · Export │ └───────────────▲───────────────────────────────────────────────▲──────────────────────┘ │ frames + orientation samples │ results, previews, files ┌───────────────┴──────────────┐ ┌───────────────┴──────────────────────┐ │ Platform adapter (thin) │ │ WebAssembly core (pure, no I/O) │ │ web: getUserMedia, │ ───────────▶ │ orientation + north, targeting, │ │ DeviceOrientation │ │ sky mask, edge/profile, stitching, │ │ native (later, optional): │ │ exporters (.hrz, landscape.ini, │ │ AVFoundation/CoreMotion│ │ horizon.txt, PNG, zip) │ │ CameraX/rotation vector│ │ │ │ replay: recorded session │ │ │ └──────────────────────────────┘ └──────────────────────────────────────┘ ``` ## Why this split - Wasm cannot reach hardware by itself. The host always supplies camera frames and sensor data. Keep that layer as thin as possible. - **Replay adapter**: a recorded session (see below) is fed into the same core API. Record one or two sweeps on the phone, then iterate on detection, stitching and export entirely in a desktop browser and in unit tests. - Start with the **web adapter only** (PWA or webview). Add native camera adapters only if needed. ### Known web limitations - **No reliable exposure or white-balance lock** across frames, so seams in the Stellarium panorama are possible. Mitigate with per-frame gain compensation in the stitcher. The NINA horizon is barely affected. - **No camera intrinsics or FOV** from the browser. Use a per-model FOV table, a one-time calibration, or let the stitcher refine FOV from frame overlaps. Store the source in `device.camera.fovSource`. - **Heading on iOS web**: `webkitCompassHeading` (magnetic, with `webkitCompassAccuracy`) after `DeviceOrientationEvent.requestPermission()`. **Chromium/Android**: `deviceorientationabsolute`, or `AbsoluteOrientationSensor` where available. Accuracy of a few degrees is acceptable because of the safety margin. - Wasm threads need cross-origin isolation (COOP/COEP headers). Keep a single-threaded path that works without them. ## Core API (language-neutral sketch; e.g. Rust + wasm-bindgen, exposed to JS) ```ts type Quat = [w: number, x: number, y: number, z: number]; // device→world, world = ENU (x East, y North, z Up) interface SessionMeta { site: {name; latDeg; lonDeg; elevM}; camera: {widthPx; heightPx; hfovDeg?; vfovDeg?}; plannedFrames: number; targetTiltDeg: number; dateIso: string } createSession(meta: SessionMeta): Session session.declinationDeg(): number // WMM at site + date session.pushSensor(t: number, q: Quat, magHeadingDeg?: number, accDeg?: number): void session.target(now: {t; q}): { dAzDeg; dTiltDeg; steady: boolean; ready: boolean; frameIndex: number } session.addFrame(t: number, rgba: Uint8Array, w: number, h: number): FrameResult // FrameResult { index; coverage: {azFrom; azTo; altBottom; altTop}; // edgePreview: Float32Array /* alt per screen column for the live line */; // touchesTop: {azDeg} | null } session.removeLastFrame(): void session.coverage(): { coveredDeg: number; cells: boolean[] } session.applyMaskEdits(edits: MaskEdit[]): void // brush strokes in (az, alt) space, sky|ground, radiusDeg session.horizon(opts?: { stepDeg?: 1; smooth?: 'off'|'light'|'strong' }): Float32Array // alt per az session.setNorthOffset(deg: number): void exportHrz(profile, { marginDeg: 2, stepDeg: 1 }): string exportStellariumSpherical(session, { widthPx: 8192, heightPx: 2048, topDeg, bottomDeg }): Map // landscape.ini, panorama.png, horizon.txt exportStellariumPolygonal(profile, meta): Map zip(files): Uint8Array ``` Rules: deterministic, no clocks, no randomness, no I/O. All times are passed in. Large buffers cross the boundary as typed arrays. ## Algorithms (first versions; keep them simple) **North.** heading_true = heading_magnetic + declination + userOffset. If the platform gives a full rotation quaternion, use it for pitch and roll. Optionally (setting) lock heading to the gyro after the first second (game rotation vector) to avoid drift from nearby steel, with a loop-closure check when the sweep returns to the first frame. Spread the residual linearly over the frames. **Auto-shutter.** Target azimuths every 360/N (N = 24, i.e. 15° steps; large overlap because stitching relies on sensor orientation, not feature matching). Capture when |dAz| < 2°, |tilt − target| < 5°, roll < 5° and angular speed < 5°/s for 300 ms. Target tilt +10° in portrait. With a vertical FOV of about 64°, that covers about −22…+42°. **Sky mask (per frame).** Classic first: a per-pixel sky score from brightness, blueness (b − r) and low local gradient. Seed from the top rows, flood-fill or connected component from the top, then morphological cleanup. Treat sky seen through branches conservatively (the edge is the **highest** non-sky pixel per column). An ML segmenter (e.g. ONNX Runtime Web) only if this proves insufficient. **Edge → profile.** For each output azimuth column, take the highest altitude of non-sky across all frames covering it (after projection). Resample to 1° steps, then apply optional smoothing (median over ±1–2°). The margin is added only at NINA export. **Top-touch detection.** If non-sky reaches a frame's top row over more than ~1° of azimuth, report `touchesTop` with that azimuth and propose the extra higher frame. **Stitching (Stellarium only).** Output an equirectangular band, 8192 px over 360° horizontally, `heightPx` over [bottomDeg, topDeg]. For each output pixel: compute the world direction, choose the frame(s) whose centre is closest, project with the frame quaternion and pinhole intrinsics, sample bilinearly, and feather-blend in overlaps. Compensate per-frame gain from overlap means. Alpha = 0 where the final mask says sky. Column 0 = azimuth 90° (East); see FORMATS.md. ## Recorded session bundle (`examples/session/`) A folder (or zip) the phone can export and the desktop can replay: - `session.json`: meta, device and camera (with `fovSource`), site, north settings, capture plan, frame list (`id`, `file`, `t`, `q`, heading accuracy), mask edits, optional result. - `frames/fNNN.jpg`: the captured frames. - `sensors.jsonl`: the continuous orientation stream (`t`, `q`, `magHeadingDeg`, `headingAccuracyDeg`), around 60 Hz. Use the same bundle as unit-test fixtures (expected profile within ±0.5°, exported files byte-stable).