Files
sigil/improvements.md
2026-05-19 18:51:40 +02:00

8.5 KiB

Audio-Visualizer Architecture Improvements

This document outlines a strategy for refactoring the audio-visualizer codebase to improve modularity, simplify the creation of new visualizers, and extract highly valuable components currently locked within specific implementations.

0. Prioritization & Risk Assessment

Not all three sections carry the same value-to-risk ratio. Recommended order:

Priority Section Verdict
Do first §3 Extracting reusable components High value, low risk. Pure logic, no GPU, mechanical move + re-export.
Maybe §3 MorphState<T> Worthwhile, but limited — see caveat below.
Defer §1 Visualizer trait Lateral, not a net win — see caveat. Worth it only if several new visualizers are planned.
Decide first §2 generic ShaderPipeline<U> Contradicts a stated architecture policy. Do not start without an explicit decision.

Risk callouts (must survive any refactor)

  • Determinism / --render bit-reproducibility. Any moved component that integrates state (Spring, the Attr RK4 attractors) must keep the advance-only-in-update discipline. This is the same rule that holds today; it just has to survive relocation. Verify by diffing a --render of the same input before/after the move — output must be bit-identical.
  • Central analysis invariant. Analyzer remains the only place STFT/AGC/onset math lives (live ≡ offline). Structure/Arch is a viz-side consumer of Bands, so moving it is safe — but if it becomes shared, keep its state mutation confined to update for the same determinism reason.
  • breakcore::NP (64) Rust↔WGSL coupling + hard-capped march steps. A generic shader base (§2) must not abstract these guardrails away — a runaway march = GPU device-lost / machine hang. naga only validates the WGSL at pipeline-create on a real GPU, so a bad layout cannot be caught in the sandbox.

Caveat on §1 (Visualizer trait)

The is_gpu() branch is a real dichotomy, not accidental complexity: Draw-based visualizers (Sigil/Scope) share the chromatic-aberration channel passes dispatched in the bin, whereas Breakcore presents/captures its own raymarch target directly. A trait with optional draw / render_gpu methods does not delete this branch — the caller still switches on is_gpu_driven(). The trait relocates the branch and standardizes the contract; it does not remove the structural split. Net positive only if multiple new visualizers are actually on the roadmap.

Caveat on §2 (generic ShaderPipeline<U>)

CLAUDE.md deliberately designates breakcore.rs as "the lone, walled-off exception" to the "no hand-written wgpu pipelines" rule (post.rs and the rest stay on nannou's validated renderer). A reusable ShaderPipeline<U> base is, by design, an invitation to add more raw wgpu pipelines — i.e. it reverses that policy. This may well be desirable, but it is a deliberate architectural decision that must be made explicitly before the extraction, not a side effect of it. If adopted, the base must still preserve the NP/UBO-layout coupling and the march-step caps as non-negotiable guardrails.

Caveat on MorphState<T>

scope and breakcore share the shape of the morph (hold a seeded state, lerp currenttarget over a timer), but their triggers differ fundamentally: scope morphs on a cooldown-gated rising-flux transient or a 12 s idle; breakcore cross-fades on a long-vs-short loudness-EMA section state machine. A generic MorphState<T> can own only the current/target/t bookkeeping and the lerp/timing; the trigger logic stays per-visualizer. Useful, but scoped — it does not unify the interesting (musical) part.

1. Modularizing the Visualizer System

Currently, the Visual type in src/bin/sigil.rs is an enum hardcoding the available visualizers (Sigil, Scope, Breakcore). It also uses an awkward branching mechanism (is_gpu()) to handle the divide between nannou::Draw-based visualizers and raw wgpu pipelines. This violates the Open-Closed Principle and makes adding new visualizers cumbersome.

Recommendation: The Visualizer Trait Refactor the enum into a dynamic trait object system. This cleans up sigil.rs and standardizes the contract for all visualizers.

pub struct RenderContext<'a> {
    pub pal: &'a Palette,
    pub scale: f32,
    pub warp: f32,
    pub glow: bool,
    pub tint: [f32; 3],
    // ... other shared gains/tunables
}

pub trait Visualizer {
    fn name(&self) -> &'static str;
    fn seed(&self) -> u64;
    fn reseed(&mut self, seed: u64);
    fn update(&mut self, bands: &Bands, dt: f32);
    fn element_count(&self) -> usize;

    /// Indicates if this visualizer manages its own wgpu pipeline.
    fn is_gpu_driven(&self) -> bool { false }

    /// Used by CPU-driven visualizers (nannou::Draw)
    fn draw(&self, draw: &Draw, ctx: &RenderContext) {}

    /// Used by GPU-driven visualizers to return their raymarch/compute target
    fn render_gpu(&mut self, device: &wgpu::Device, queue: &wgpu::Queue, ctx: &RenderContext) -> Option<&wgpu::Texture> { None }
}

2. Designing a New Visualizer Base

To support rapid development of completely new visualizer modes (especially WGSL-based ones like Breakcore), we should abstract the boilerplate into unified bases.

The ShaderPipeline Base

The Gpu struct inside breakcore.rs is a fantastic, self-contained raw wgpu pipeline. It should be extracted into a generic viz::shader::ShaderPipeline<U> where U is a bytemuck::Pod trait bound for the Uniform Buffer Object.

This base would automatically handle:

  • WGSL compilation and wgpu layout boilerplate.
  • Dual-texture ping-ponging for frame feedback.
  • UBO layout mapping and writing.
pub struct ShaderPipeline<U> {
    // pipeline, bindings, and ping-pong textures
}

impl<U: bytemuck::Pod> ShaderPipeline<U> {
    pub fn new(device: &wgpu::Device, wgsl_source: &str) -> Self { ... }
    pub fn render(&mut self, queue: &wgpu::Queue, ubo: &U) -> &wgpu::Texture { ... }
}

A new GPU-based visualizer would then only need to define its UBO struct, its .wgsl shader, and implement Visualizer::render_gpu.

3. Extracting Reusable Components

Several brilliant components are currently tightly coupled to breakcore.rs and scope.rs. They should be hoisted to common modules so any new visualizer can use them.

From breakcore.rs

  1. Structure & Arch (Musical Archetype Classifier): The slow feature-vector classifier that detects Arch (Ambient, Build, Drop, Breakdown, Groove) and manages tension/release is incredibly powerful. It converts raw FFT data into musical context.

    • Action: Move this to src/viz/structure.rs (or even src/audio.rs). Any new visualizer should be able to query structure.tension to drive camera FOV, or trigger a massive event on Arch::Drop.
  2. Critically Damped Spring: The Spring struct applies physics-based smoothing to visual targets. It prevents jitter while maintaining immediate snap on transient hits.

    • Action: Move to a new src/viz/math.rs. This is the perfect primitive for mapping raw audio bands to visual scales/rotations across all visualizers.
  3. RK4 Attractors & Knots: The Attr (Lorenz/Rössler) and Knot geometry generators create stable, chaotic 3D paths.

    • Action: Move to a new src/viz/geometry.rs.

From scope.rs

  1. 3D Vector Figures: The Figure generator (Gielis supershapes, 3D Lissajous, Harmonographs) is excellent for generating deterministic, organic geometry.

    • Action: Move to src/viz/geometry.rs alongside the attractors.
  2. The Morphing State Machine: Both scope and breakcore implement a pattern of holding a seed, tracking an idle timer or waiting for a transient, and smoothly morphing to a new state.

    • Action: Abstract this into a MorphState<T> primitive that wraps a current and target state, handling the lerp/transition logic automatically.

Summary of Proposed Directory Structure

src/viz/
├── mod.rs
├── math.rs       <-- (NEW) Spring, smoothing, interpolations
├── geometry.rs   <-- (NEW) Figures, Knots, Attractors
├── structure.rs  <-- (NEW) Arch, Structure classifier
├── shader.rs     <-- (NEW) Generic wgpu ShaderPipeline
├── core.rs       <-- (NEW) Visualizer trait and RenderContext
├── breakcore.rs  (Refactored to use new bases)
├── scope.rs      (Refactored to use new bases)
└── sigil.rs