EdLines quad extraction & segmentation — lessons
Status: ACTIVE (EdLines ships in high_accuracy); two follow-up improvements FALSIFIED, listed below.
Last updated: 2026-07-19
Owning code: crates/locus-core/src/edlines.rs (5-phase pipeline: extract_boundary_segments, IRLS line fit, refine_edge_subpixel, sub-pixel IRLS re-fit, refine_corners_gauss_newton); imbalance gate ~edlines.rs:1088/:1224; called from quad.rs:280; post-EdLines refine_corner/fit_edge_line at quad.rs:1286/:1340; ERF finder in edge_refinement.rs (opt-in edlines_phase3_erf). Routed via crates/locus-core/profiles/high_accuracy.json.
TL;DR
EdLines is the line-based quad extractor high_accuracy uses for high-PPB tags: it segments the component boundary into four angular arcs, fits an IRLS line per arc in binary space, refines each to sub-pixel via a micro-ray parabolic gradient fit, re-fits, then runs a joint 8-DoF (4-corner) chord-locked Gauss-Newton. What ships is exactly that pipeline — static axis-imbalance gate, chord-coupling regulariser, ±1.5px parabolic clamp, 5% corner-bleed exclusion zone. Two principled-looking follow-ups were built and reverted the same day when regression_render_tag rejected them: (F-a) a continuous arc-balance Phase-1 mode selector to replace the static >40%/<16% gate, and (F-b) design-memo "Improvement #2", an 8-DoF line-parameterised Phase 5 that drops the chord-coupling. They matter because both encode a durable lesson: the decoder's Hamming-margin rejection is invisible to any geometry-only selector, and chord-coupling is a genuine variance-reducing regulariser, not a hack. The motivating case (scene_0008) is a detector-internal ~3.8px corner bias with no small-surface single-frame fix; it is bounded at runtime by PR #281's outlier-aware corner-drop + Σ_pose inflation gates.
What shipped (durable design decisions)
- Phase 1 — monotone-scan boundary + angular-arc partition. Column/row extremal scan collects ~2·(W+H) outer-boundary pixels and filters interior data-bit pixels. Four arcs (T→R, R→B, B→L, L→T) pivot at extremals from one of two systems: AxisAligned (TRBL) tried first, Diagonal (NW/NE/SE/SW) fallback if any TRBL arc < 5°. Tags near 0° prefer TRBL, tags near 45° prefer Diagonal.
- Static imbalance gate (
edlines.rs:1088): rejects AxisAligned partitions where one arc > 40% and another < 16% of points — the collapse mode where two adjacent corners share a TRBL extremal. Kept static after F-a falsified the continuous replacement. - Phase 2 — binary IRLS line fit (Huber δ=1.5px, 3 iters, TLS). Lives in binary-image space with ±0.5px construction ambiguity.
- Phase 3 — micro-ray parabolic sub-pixel: walk in
sample_step=1.5px, 5 samples at k∈{−2..+2} along the normal, gate|g₀|≥8, parabola vertex clamped to [−1.5,+1.5]px (accept boundary rather than extrapolate — the safety mechanism). - Phase 4 — sub-pixel IRLS re-fit (tighter Huber δ=0.5, 2 iters); falls back to scaled Phase-2 line if <
min_edge_pts=5 sub-pixel points. - Phase 5 — joint 8-DoF chord-locked Gauss-Newton over 4 corners, cost = perpendicular distance to the chord
n̂(θ)between the two bounding corners; unrolled 8×8 Cholesky, Tikhonov λ=1e-6, 3 iters, ±5px divergence guard. The chord-coupling is a structural regulariser: each corner is constrained by two edges' observations, so per-edge noise averages across edges (see F-b). Includes a 5% corner-bleed exclusion zone (α∈[0.05,0.95]) rejecting near-corner points contaminated by the perpendicular edge's PSF tail. - Architectural coupling (known limitation, by design): every phase inherits its line direction from the prior phase; a wrong Phase-2 slope cannot be corrected downstream. This is why post-EdLines
refine_corner(which locks direction to the input-corner chord viafit_edge_line) also can't repair it. - S1 corner-exclusion is NOT shipped.
corner_exclusion_pxdefaults 0.0 (byte-identical off-path); the field/env-var/filter were reverted. What survives is the confirmed finding + theLOCUS_DUMP_EDLINES_PHASE1telemetry hook (dead code, no env-var, zero overhead).
What was tried and falsified
| Attempt | What it changed | Why it failed (mechanism) | Evidence | Re-attempt only if |
|---|---|---|---|---|
| F-a: continuous arc-balance Phase-1 selector (2026-05-25) | Compute both AXIS & DIAG partitions, pick lower max/min arc ratio (0.90 strict-better margin, 1.40 trigger); replace static gate |
Phase 2–5 succeeds on bad-geometry DIAG corners (passes every geometric sanity check); the downstream decoder rejects them on Hamming margin — invisible to the selector. Decoder-rejection failure mode dominates the criterion space | 640×480 render-tag mean RMSE +15%, scene_0049 worst (rmse 2.26px) at every tuning; tightening min_arc/bbox_area gates instead dropped recall 1.0→0.86 (5 missed tags) |
(1) selection-by-attempt-decode (plumb decoder into EdLines, ~2× cost); (2) Phase-1 telemetry proving arc-balance correlates with decode-success; (3) a criterion that anticipates decode failure without running it |
| F-b: Phase-5 8-DoF line parameterisation (design memo Improvement #2) | Replace 8-DoF corner state with 4 independent per-edge lines (θ_k,d_k); corners derived as intersections; deletes chord-cost + 8×8 Cholesky + 5% exclusion zone (~170 LoC) |
Chord-coupling is a variance-reducing regulariser. Decoupling makes each corner a derived intersection of two independently-fit lines; per-edge noise (σ≈0.1–0.3px on Blender) no longer averages — it amplifies through the intersection Jacobian. Variance increase dominates bias reduction | Render-tag mean RMSE +30–43%, p99 rot +30–100% at every resolution (e.g. 720p mean 0.208→0.282, p99_rot 0.652→1.130); 12/14 render-tag tests failed. scene_0008 only partially fixed (2.06→1.79px) | (1) chord-cost as a soft regulariser (λ·corner-consistency penalty — reintroduces a calibration knob); (2) 8-DoF corner state with chord-relaxed (d−d_ref)² cost; (3) per-line IRLS+covariance (same amplification risk); (4) real-camera evidence F4 is prevalent |
| Improvement #1: iterate Phase 3/4 on refined-line trajectory (2026-05-03) | Second Phase-3 walk along the Phase-4 v1 sub-pixel line, then Phase-4 v2 re-fit | Phase 5's chord-locked GN is ill-conditioned: v2's ~0.05px IRLS shift amplifies non-monotonically. On scene_0005 it tipped the LM into Cholesky failure (NaN cov, 1.2m error) | scene_0008 c1 3.833→4.086px (worse); corpus KL 13.93→24.34 | Stabilise Phase-5 conditioning first (Tikhonov λ=1e-6 far too small) |
| F5: shrink Phase-5 exclusion zone 5%→1% | Widen α to [0.01,0.99] | Two-sided: corner-residual improves but pose-d² regresses (removing points shrinks σ² faster than ‖r‖ → confidently-wrong Σ) | c1 ‖r‖ −4.6% (only −0.18px), corpus mean ‖r‖ −2.1%, but mean d² +7.6% | — (metric-pleasing on ‖r‖ proxy only) |
| S1: corner-region pixel exclusion (2026-05-04) | Drop boundary points within Δ of each arc's two extremals | Confirmed hypothesis (corner-pixel contamination biases IRLS slope) but insufficient: the remaining ~1.1px is everything-along-edge integer rounding (Phase-2 collapses to exact y=456.5) that no point-removal fixes — needs a gradient-from-the-start extractor |
Δ-sweep {1,2,3,5}px: best −1.19px on scene_0008 at Δ=5 (target −2px, NOT MET; drops 1 scene to no-detection). Corpus mean ‖r‖ −14% but mean d² +12–23% | Real-camera evidence of scene_0008-class failures at non-trivial rates; ship as opt-in corpus-bias reducer only if d² regression acceptable |
| S3: ED-Lines gradient-anchor walk (design only, never built) | Replace binary boundary tracing with Akinlar–Topal anchor/chain/NFA on the gray image | Deferred — inherits the ~0.6px Blender-PSF sub-pixel floor; multi-day on synthetic data; converging negative memos say the needed data is real-camera | Not executed; sketch preserved | Real-camera evidence; revival should measure rotation tail directly, not ‖r‖ proxies |
Motivating case
scene_0008_cam_0000 (locus_v1_tag36h11_1920x1080) is a tag rotated ≈90° but axis-aligned in image space — the degenerate geometry where TRBL extremals land on corners, not edge midpoints. Its canonical corner-1 is detected 3.8px off along the top edge, while the actual image edges fit GT to 0.06px RMS (intersection 0.24px from GT) — a detector-internal failure with no support in the pixel data. The bias originates in the Phase 1→2 cascade (integer-pixel rounding floor, Phase-2 IRLS collapsing to exact y=456.5), is fully upstream of Phase 5, and has no small-surface single-frame fix. The practical resolution is the runtime gate: the corner_geometry_outlier classifier (min_irls=0.219, max_corner_d²=37.7 vs χ²(1) α=1e-4=15.137) drops the corner / inflates Σ_pose / rejects the tag (PR #281), for a p99-rotation drop of 0.171° at zero architectural risk.
Re-attempt only if
- Real-camera evidence shows scene_0008-class (near-axis-aligned, mode-mismatched) failures at non-trivial rates — all synthetic evidence shows these are tail-only. This is the gating condition for S3, F-a, and F-b alike.
- F-b (chord decoupling): only as a soft regulariser (λ-penalty or chord-relaxed
(d−d_ref)²cost) that preserves the cross-edge variance-reduction — never as a bare per-line refit. - F-a (arc-balance selector): only with the decoder plumbed into the selection loop (selection-by-attempt-decode) or a criterion that predicts decode failure from geometry.
- Any Phase-3/4 iteration: only after Phase-5 conditioning is hardened (λ=1e-6 is too small).
- Prefer measuring
rotation_error_chosen_deg(the metric of record) directly, not ‖r‖/d² proxies — S1's ‖r‖ win with a d² regression was the trap.
Provenance
Distilled 2026-07-19 from (removed; see git history): edlines_sota_design_2026-05-03, edlines_segmentation_design_2026-05-03, edlines_sota_experiments_2026-05-03, edlines_phase5_decoupling_2026-05-03, edlines_s1_corner_exclusion_2026-05-04, edlines_sota_followup_postmortem_2026-05-25, scene_0008_root_cause_2026-05-03. Related: MEMORY anti-patterns "EdLines competitive Phase-1 selector (2026-05-25)" and "EdLines Phase-5 line parameterisation (2026-05-25)".