Files
2026-10-24 13:20:55 +02:00

7.5 KiB
Raw Permalink Blame History

Architecture

Goal: develop and tune almost everything in a desktop browser without redeploying to the phone.

┌────────────── UI (web stack, same in browser and in the app's webview) ──────────────┐
│ Sites · Setup · Capture · Review · Export                                            │
└───────────────▲───────────────────────────────────────────────▲──────────────────────┘
                │ frames + orientation samples                  │ results, previews, files
┌───────────────┴──────────────┐                ┌───────────────┴──────────────────────┐
│ Platform adapter (thin)      │                │ WebAssembly core (pure, no I/O)      │
│  web: getUserMedia,          │  ───────────▶  │  orientation + north, targeting,     │
│       DeviceOrientation      │                │  sky mask, edge/profile, stitching,  │
│  native (later, optional):   │                │  exporters (.hrz, landscape.ini,     │
│       AVFoundation/CoreMotion│                │  horizon.txt, PNG, zip)              │
│       CameraX/rotation vector│                │                                      │
│  replay: recorded session    │                │                                      │
└──────────────────────────────┘                └──────────────────────────────────────┘

Why this split

  • Wasm cannot reach hardware by itself. The host always supplies camera frames and sensor data. Keep that layer as thin as possible.
  • Replay adapter: a recorded session (see below) is fed into the same core API. Record one or two sweeps on the phone, then iterate on detection, stitching and export entirely in a desktop browser and in unit tests.
  • Start with the web adapter only (PWA or webview). Add native camera adapters only if needed.

Known web limitations

  • No reliable exposure or white-balance lock across frames, so seams in the Stellarium panorama are possible. Mitigate with per-frame gain compensation in the stitcher. The NINA horizon is barely affected.
  • No camera intrinsics or FOV from the browser. Use a per-model FOV table, a one-time calibration, or let the stitcher refine FOV from frame overlaps. Store the source in device.camera.fovSource.
  • Heading on iOS web: webkitCompassHeading (magnetic, with webkitCompassAccuracy) after DeviceOrientationEvent.requestPermission(). Chromium/Android: deviceorientationabsolute, or AbsoluteOrientationSensor where available. Accuracy of a few degrees is acceptable because of the safety margin.
  • Wasm threads need cross-origin isolation (COOP/COEP headers). Keep a single-threaded path that works without them.

Core API (language-neutral sketch; e.g. Rust + wasm-bindgen, exposed to JS)

type Quat = [w: number, x: number, y: number, z: number]; // device→world, world = ENU (x East, y North, z Up)

interface SessionMeta { site: {name; latDeg; lonDeg; elevM}; camera: {widthPx; heightPx; hfovDeg?; vfovDeg?};
                        plannedFrames: number; targetTiltDeg: number; dateIso: string }

createSession(meta: SessionMeta): Session
session.declinationDeg(): number                          // WMM at site + date
session.pushSensor(t: number, q: Quat, magHeadingDeg?: number, accDeg?: number): void
session.target(now: {t; q}): { dAzDeg; dTiltDeg; steady: boolean; ready: boolean; frameIndex: number }
session.addFrame(t: number, rgba: Uint8Array, w: number, h: number): FrameResult
     // FrameResult { index; coverage: {azFrom; azTo; altBottom; altTop};
     //               edgePreview: Float32Array /* alt per screen column for the live line */;
     //               touchesTop: {azDeg} | null }
session.removeLastFrame(): void
session.coverage(): { coveredDeg: number; cells: boolean[] }

session.applyMaskEdits(edits: MaskEdit[]): void           // brush strokes in (az, alt) space, sky|ground, radiusDeg
session.horizon(opts?: { stepDeg?: 1; smooth?: 'off'|'light'|'strong' }): Float32Array // alt per az
session.setNorthOffset(deg: number): void

exportHrz(profile, { marginDeg: 2, stepDeg: 1 }): string
exportStellariumSpherical(session, { widthPx: 8192, heightPx: 2048, topDeg, bottomDeg }): Map<string, Uint8Array> // landscape.ini, panorama.png, horizon.txt
exportStellariumPolygonal(profile, meta): Map<string, Uint8Array>
zip(files): Uint8Array

Rules: deterministic, no clocks, no randomness, no I/O. All times are passed in. Large buffers cross the boundary as typed arrays.

Algorithms (first versions; keep them simple)

North. heading_true = heading_magnetic + declination + userOffset. If the platform gives a full rotation quaternion, use it for pitch and roll. Optionally (setting) lock heading to the gyro after the first second (game rotation vector) to avoid drift from nearby steel, with a loop-closure check when the sweep returns to the first frame. Spread the residual linearly over the frames.

Auto-shutter. Target azimuths every 360/N (N = 24, i.e. 15° steps; large overlap because stitching relies on sensor orientation, not feature matching). Capture when |dAz| < 2°, |tilt − target| < 5°, roll < 5° and angular speed < 5°/s for 300 ms. Target tilt +10° in portrait. With a vertical FOV of about 64°, that covers about −22…+42°.

Sky mask (per frame). Classic first: a per-pixel sky score from brightness, blueness (b − r) and low local gradient. Seed from the top rows, flood-fill or connected component from the top, then morphological cleanup. Treat sky seen through branches conservatively (the edge is the highest non-sky pixel per column). An ML segmenter (e.g. ONNX Runtime Web) only if this proves insufficient.

Edge → profile. For each output azimuth column, take the highest altitude of non-sky across all frames covering it (after projection). Resample to 1° steps, then apply optional smoothing (median over ±1–2°). The margin is added only at NINA export.

Top-touch detection. If non-sky reaches a frame's top row over more than ~1° of azimuth, report touchesTop with that azimuth and propose the extra higher frame.

Stitching (Stellarium only). Output an equirectangular band, 8192 px over 360° horizontally, heightPx over [bottomDeg, topDeg]. For each output pixel: compute the world direction, choose the frame(s) whose centre is closest, project with the frame quaternion and pinhole intrinsics, sample bilinearly, and feather-blend in overlaps. Compensate per-frame gain from overlap means. Alpha = 0 where the final mask says sky. Column 0 = azimuth 90° (East); see FORMATS.md.

Recorded session bundle (examples/session/)

A folder (or zip) the phone can export and the desktop can replay:

  • session.json: meta, device and camera (with fovSource), site, north settings, capture plan, frame list (id, file, t, q, heading accuracy), mask edits, optional result.
  • frames/fNNN.jpg: the captured frames.
  • sensors.jsonl: the continuous orientation stream (t, q, magHeadingDeg, headingAccuracyDeg), around 60 Hz.

Use the same bundle as unit-test fixtures (expected profile within ±0.5°, exported files byte-stable).