7.5 KiB
Architecture
Goal: develop and tune almost everything in a desktop browser without redeploying to the phone.
┌────────────── UI (web stack, same in browser and in the app's webview) ──────────────┐
│ Sites · Setup · Capture · Review · Export │
└───────────────▲───────────────────────────────────────────────▲──────────────────────┘
│ frames + orientation samples │ results, previews, files
┌───────────────┴──────────────┐ ┌───────────────┴──────────────────────┐
│ Platform adapter (thin) │ │ WebAssembly core (pure, no I/O) │
│ web: getUserMedia, │ ───────────▶ │ orientation + north, targeting, │
│ DeviceOrientation │ │ sky mask, edge/profile, stitching, │
│ native (later, optional): │ │ exporters (.hrz, landscape.ini, │
│ AVFoundation/CoreMotion│ │ horizon.txt, PNG, zip) │
│ CameraX/rotation vector│ │ │
│ replay: recorded session │ │ │
└──────────────────────────────┘ └──────────────────────────────────────┘
Why this split
- Wasm cannot reach hardware by itself. The host always supplies camera frames and sensor data. Keep that layer as thin as possible.
- Replay adapter: a recorded session (see below) is fed into the same core API. Record one or two sweeps on the phone, then iterate on detection, stitching and export entirely in a desktop browser and in unit tests.
- Start with the web adapter only (PWA or webview). Add native camera adapters only if needed.
Known web limitations
- No reliable exposure or white-balance lock across frames, so seams in the Stellarium panorama are possible. Mitigate with per-frame gain compensation in the stitcher. The NINA horizon is barely affected.
- No camera intrinsics or FOV from the browser. Use a per-model FOV table, a one-time calibration, or let the stitcher refine FOV from frame overlaps. Store the source in
device.camera.fovSource. - Heading on iOS web:
webkitCompassHeading(magnetic, withwebkitCompassAccuracy) afterDeviceOrientationEvent.requestPermission(). Chromium/Android:deviceorientationabsolute, orAbsoluteOrientationSensorwhere available. Accuracy of a few degrees is acceptable because of the safety margin. - Wasm threads need cross-origin isolation (COOP/COEP headers). Keep a single-threaded path that works without them.
Core API (language-neutral sketch; e.g. Rust + wasm-bindgen, exposed to JS)
type Quat = [w: number, x: number, y: number, z: number]; // device→world, world = ENU (x East, y North, z Up)
interface SessionMeta { site: {name; latDeg; lonDeg; elevM}; camera: {widthPx; heightPx; hfovDeg?; vfovDeg?};
plannedFrames: number; targetTiltDeg: number; dateIso: string }
createSession(meta: SessionMeta): Session
session.declinationDeg(): number // WMM at site + date
session.pushSensor(t: number, q: Quat, magHeadingDeg?: number, accDeg?: number): void
session.target(now: {t; q}): { dAzDeg; dTiltDeg; steady: boolean; ready: boolean; frameIndex: number }
session.addFrame(t: number, rgba: Uint8Array, w: number, h: number): FrameResult
// FrameResult { index; coverage: {azFrom; azTo; altBottom; altTop};
// edgePreview: Float32Array /* alt per screen column for the live line */;
// touchesTop: {azDeg} | null }
session.removeLastFrame(): void
session.coverage(): { coveredDeg: number; cells: boolean[] }
session.applyMaskEdits(edits: MaskEdit[]): void // brush strokes in (az, alt) space, sky|ground, radiusDeg
session.horizon(opts?: { stepDeg?: 1; smooth?: 'off'|'light'|'strong' }): Float32Array // alt per az
session.setNorthOffset(deg: number): void
exportHrz(profile, { marginDeg: 2, stepDeg: 1 }): string
exportStellariumSpherical(session, { widthPx: 8192, heightPx: 2048, topDeg, bottomDeg }): Map<string, Uint8Array> // landscape.ini, panorama.png, horizon.txt
exportStellariumPolygonal(profile, meta): Map<string, Uint8Array>
zip(files): Uint8Array
Rules: deterministic, no clocks, no randomness, no I/O. All times are passed in. Large buffers cross the boundary as typed arrays.
Algorithms (first versions; keep them simple)
North. heading_true = heading_magnetic + declination + userOffset. If the platform gives a full rotation quaternion, use it for pitch and roll. Optionally (setting) lock heading to the gyro after the first second (game rotation vector) to avoid drift from nearby steel, with a loop-closure check when the sweep returns to the first frame. Spread the residual linearly over the frames.
Auto-shutter. Target azimuths every 360/N (N = 24, i.e. 15° steps; large overlap because stitching relies on sensor orientation, not feature matching). Capture when |dAz| < 2°, |tilt − target| < 5°, roll < 5° and angular speed < 5°/s for 300 ms. Target tilt +10° in portrait. With a vertical FOV of about 64°, that covers about −22…+42°.
Sky mask (per frame). Classic first: a per-pixel sky score from brightness, blueness (b − r) and low local gradient. Seed from the top rows, flood-fill or connected component from the top, then morphological cleanup. Treat sky seen through branches conservatively (the edge is the highest non-sky pixel per column). An ML segmenter (e.g. ONNX Runtime Web) only if this proves insufficient.
Edge → profile. For each output azimuth column, take the highest altitude of non-sky across all frames covering it (after projection). Resample to 1° steps, then apply optional smoothing (median over ±1–2°). The margin is added only at NINA export.
Top-touch detection. If non-sky reaches a frame's top row over more than ~1° of azimuth, report touchesTop with that azimuth and propose the extra higher frame.
Stitching (Stellarium only). Output an equirectangular band, 8192 px over 360° horizontally, heightPx over [bottomDeg, topDeg]. For each output pixel: compute the world direction, choose the frame(s) whose centre is closest, project with the frame quaternion and pinhole intrinsics, sample bilinearly, and feather-blend in overlaps. Compensate per-frame gain from overlap means. Alpha = 0 where the final mask says sky. Column 0 = azimuth 90° (East); see FORMATS.md.
Recorded session bundle (examples/session/)
A folder (or zip) the phone can export and the desktop can replay:
session.json: meta, device and camera (withfovSource), site, north settings, capture plan, frame list (id,file,t,q, heading accuracy), mask edits, optional result.frames/fNNN.jpg: the captured frames.sensors.jsonl: the continuous orientation stream (t,q,magHeadingDeg,headingAccuracyDeg), around 60 Hz.
Use the same bundle as unit-test fixtures (expected profile within ±0.5°, exported files byte-stable).