Files
2026-10-24 13:20:55 +02:00

88 lines
7.5 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Architecture
Goal: develop and tune almost everything in a desktop browser without redeploying to the phone.
```
┌────────────── UI (web stack, same in browser and in the app's webview) ──────────────┐
│ Sites · Setup · Capture · Review · Export │
└───────────────▲───────────────────────────────────────────────▲──────────────────────┘
│ frames + orientation samples │ results, previews, files
┌───────────────┴──────────────┐ ┌───────────────┴──────────────────────┐
│ Platform adapter (thin) │ │ WebAssembly core (pure, no I/O) │
│ web: getUserMedia, │ ───────────▶ │ orientation + north, targeting, │
│ DeviceOrientation │ │ sky mask, edge/profile, stitching, │
│ native (later, optional): │ │ exporters (.hrz, landscape.ini, │
│ AVFoundation/CoreMotion│ │ horizon.txt, PNG, zip) │
│ CameraX/rotation vector│ │ │
│ replay: recorded session │ │ │
└──────────────────────────────┘ └──────────────────────────────────────┘
```
## Why this split
- Wasm cannot reach hardware by itself. The host always supplies camera frames and sensor data. Keep that layer as thin as possible.
- **Replay adapter**: a recorded session (see below) is fed into the same core API. Record one or two sweeps on the phone, then iterate on detection, stitching and export entirely in a desktop browser and in unit tests.
- Start with the **web adapter only** (PWA or webview). Add native camera adapters only if needed.
### Known web limitations
- **No reliable exposure or white-balance lock** across frames, so seams in the Stellarium panorama are possible. Mitigate with per-frame gain compensation in the stitcher. The NINA horizon is barely affected.
- **No camera intrinsics or FOV** from the browser. Use a per-model FOV table, a one-time calibration, or let the stitcher refine FOV from frame overlaps. Store the source in `device.camera.fovSource`.
- **Heading on iOS web**: `webkitCompassHeading` (magnetic, with `webkitCompassAccuracy`) after `DeviceOrientationEvent.requestPermission()`. **Chromium/Android**: `deviceorientationabsolute`, or `AbsoluteOrientationSensor` where available. Accuracy of a few degrees is acceptable because of the safety margin.
- Wasm threads need cross-origin isolation (COOP/COEP headers). Keep a single-threaded path that works without them.
## Core API (language-neutral sketch; e.g. Rust + wasm-bindgen, exposed to JS)
```ts
type Quat = [w: number, x: number, y: number, z: number]; // device→world, world = ENU (x East, y North, z Up)
interface SessionMeta { site: {name; latDeg; lonDeg; elevM}; camera: {widthPx; heightPx; hfovDeg?; vfovDeg?};
plannedFrames: number; targetTiltDeg: number; dateIso: string }
createSession(meta: SessionMeta): Session
session.declinationDeg(): number // WMM at site + date
session.pushSensor(t: number, q: Quat, magHeadingDeg?: number, accDeg?: number): void
session.target(now: {t; q}): { dAzDeg; dTiltDeg; steady: boolean; ready: boolean; frameIndex: number }
session.addFrame(t: number, rgba: Uint8Array, w: number, h: number): FrameResult
// FrameResult { index; coverage: {azFrom; azTo; altBottom; altTop};
// edgePreview: Float32Array /* alt per screen column for the live line */;
// touchesTop: {azDeg} | null }
session.removeLastFrame(): void
session.coverage(): { coveredDeg: number; cells: boolean[] }
session.applyMaskEdits(edits: MaskEdit[]): void // brush strokes in (az, alt) space, sky|ground, radiusDeg
session.horizon(opts?: { stepDeg?: 1; smooth?: 'off'|'light'|'strong' }): Float32Array // alt per az
session.setNorthOffset(deg: number): void
exportHrz(profile, { marginDeg: 2, stepDeg: 1 }): string
exportStellariumSpherical(session, { widthPx: 8192, heightPx: 2048, topDeg, bottomDeg }): Map<string, Uint8Array> // landscape.ini, panorama.png, horizon.txt
exportStellariumPolygonal(profile, meta): Map<string, Uint8Array>
zip(files): Uint8Array
```
Rules: deterministic, no clocks, no randomness, no I/O. All times are passed in. Large buffers cross the boundary as typed arrays.
## Algorithms (first versions; keep them simple)
**North.** heading_true = heading_magnetic + declination + userOffset. If the platform gives a full rotation quaternion, use it for pitch and roll. Optionally (setting) lock heading to the gyro after the first second (game rotation vector) to avoid drift from nearby steel, with a loop-closure check when the sweep returns to the first frame. Spread the residual linearly over the frames.
**Auto-shutter.** Target azimuths every 360/N (N = 24, i.e. 15° steps; large overlap because stitching relies on sensor orientation, not feature matching). Capture when |dAz| < 2°, |tilt − target| < 5°, roll < 5° and angular speed < 5°/s for 300 ms. Target tilt +10° in portrait. With a vertical FOV of about 64°, that covers about −22…+42°.
**Sky mask (per frame).** Classic first: a per-pixel sky score from brightness, blueness (b − r) and low local gradient. Seed from the top rows, flood-fill or connected component from the top, then morphological cleanup. Treat sky seen through branches conservatively (the edge is the **highest** non-sky pixel per column). An ML segmenter (e.g. ONNX Runtime Web) only if this proves insufficient.
**Edge → profile.** For each output azimuth column, take the highest altitude of non-sky across all frames covering it (after projection). Resample to 1° steps, then apply optional smoothing (median over ±1–2°). The margin is added only at NINA export.
**Top-touch detection.** If non-sky reaches a frame's top row over more than ~1° of azimuth, report `touchesTop` with that azimuth and propose the extra higher frame.
**Stitching (Stellarium only).** Output an equirectangular band, 8192 px over 360° horizontally, `heightPx` over [bottomDeg, topDeg]. For each output pixel: compute the world direction, choose the frame(s) whose centre is closest, project with the frame quaternion and pinhole intrinsics, sample bilinearly, and feather-blend in overlaps. Compensate per-frame gain from overlap means. Alpha = 0 where the final mask says sky. Column 0 = azimuth 90° (East); see FORMATS.md.
## Recorded session bundle (`examples/session/`)
A folder (or zip) the phone can export and the desktop can replay:
- `session.json`: meta, device and camera (with `fovSource`), site, north settings, capture plan, frame list (`id`, `file`, `t`, `q`, heading accuracy), mask edits, optional result.
- `frames/fNNN.jpg`: the captured frames.
- `sensors.jsonl`: the continuous orientation stream (`t`, `q`, `magHeadingDeg`, `headingAccuracyDeg`), around 60 Hz.
Use the same bundle as unit-test fixtures (expected profile within ±0.5°, exported files byte-stable).