8.5 KiB
Audio-Visualizer Architecture Improvements
This document outlines a strategy for refactoring the audio-visualizer codebase to improve modularity, simplify the creation of new visualizers, and extract highly valuable components currently locked within specific implementations.
0. Prioritization & Risk Assessment
Not all three sections carry the same value-to-risk ratio. Recommended order:
| Priority | Section | Verdict |
|---|---|---|
| Do first | §3 Extracting reusable components | High value, low risk. Pure logic, no GPU, mechanical move + re-export. |
| Maybe | §3 MorphState<T> |
Worthwhile, but limited — see caveat below. |
| Defer | §1 Visualizer trait |
Lateral, not a net win — see caveat. Worth it only if several new visualizers are planned. |
| Decide first | §2 generic ShaderPipeline<U> |
Contradicts a stated architecture policy. Do not start without an explicit decision. |
Risk callouts (must survive any refactor)
- Determinism /
--renderbit-reproducibility. Any moved component that integrates state (Spring, theAttrRK4 attractors) must keep the advance-only-in-updatediscipline. This is the same rule that holds today; it just has to survive relocation. Verify by diffing a--renderof the same input before/after the move — output must be bit-identical. - Central analysis invariant.
Analyzerremains the only place STFT/AGC/onset math lives (live ≡ offline).Structure/Archis a viz-side consumer ofBands, so moving it is safe — but if it becomes shared, keep its state mutation confined toupdatefor the same determinism reason. breakcore::NP(64) Rust↔WGSL coupling + hard-capped march steps. A generic shader base (§2) must not abstract these guardrails away — a runaway march = GPU device-lost / machine hang. naga only validates the WGSL at pipeline-create on a real GPU, so a bad layout cannot be caught in the sandbox.
Caveat on §1 (Visualizer trait)
The is_gpu() branch is a real dichotomy, not accidental complexity: Draw-based visualizers (Sigil/Scope) share the chromatic-aberration channel passes dispatched in the bin, whereas Breakcore presents/captures its own raymarch target directly. A trait with optional draw / render_gpu methods does not delete this branch — the caller still switches on is_gpu_driven(). The trait relocates the branch and standardizes the contract; it does not remove the structural split. Net positive only if multiple new visualizers are actually on the roadmap.
Caveat on §2 (generic ShaderPipeline<U>)
CLAUDE.md deliberately designates breakcore.rs as "the lone, walled-off exception" to the "no hand-written wgpu pipelines" rule (post.rs and the rest stay on nannou's validated renderer). A reusable ShaderPipeline<U> base is, by design, an invitation to add more raw wgpu pipelines — i.e. it reverses that policy. This may well be desirable, but it is a deliberate architectural decision that must be made explicitly before the extraction, not a side effect of it. If adopted, the base must still preserve the NP/UBO-layout coupling and the march-step caps as non-negotiable guardrails.
Caveat on MorphState<T>
scope and breakcore share the shape of the morph (hold a seeded state, lerp current→target over a timer), but their triggers differ fundamentally: scope morphs on a cooldown-gated rising-flux transient or a 12 s idle; breakcore cross-fades on a long-vs-short loudness-EMA section state machine. A generic MorphState<T> can own only the current/target/t bookkeeping and the lerp/timing; the trigger logic stays per-visualizer. Useful, but scoped — it does not unify the interesting (musical) part.
1. Modularizing the Visualizer System
Currently, the Visual type in src/bin/sigil.rs is an enum hardcoding the available visualizers (Sigil, Scope, Breakcore). It also uses an awkward branching mechanism (is_gpu()) to handle the divide between nannou::Draw-based visualizers and raw wgpu pipelines. This violates the Open-Closed Principle and makes adding new visualizers cumbersome.
Recommendation: The Visualizer Trait
Refactor the enum into a dynamic trait object system. This cleans up sigil.rs and standardizes the contract for all visualizers.
pub struct RenderContext<'a> {
pub pal: &'a Palette,
pub scale: f32,
pub warp: f32,
pub glow: bool,
pub tint: [f32; 3],
// ... other shared gains/tunables
}
pub trait Visualizer {
fn name(&self) -> &'static str;
fn seed(&self) -> u64;
fn reseed(&mut self, seed: u64);
fn update(&mut self, bands: &Bands, dt: f32);
fn element_count(&self) -> usize;
/// Indicates if this visualizer manages its own wgpu pipeline.
fn is_gpu_driven(&self) -> bool { false }
/// Used by CPU-driven visualizers (nannou::Draw)
fn draw(&self, draw: &Draw, ctx: &RenderContext) {}
/// Used by GPU-driven visualizers to return their raymarch/compute target
fn render_gpu(&mut self, device: &wgpu::Device, queue: &wgpu::Queue, ctx: &RenderContext) -> Option<&wgpu::Texture> { None }
}
2. Designing a New Visualizer Base
To support rapid development of completely new visualizer modes (especially WGSL-based ones like Breakcore), we should abstract the boilerplate into unified bases.
The ShaderPipeline Base
The Gpu struct inside breakcore.rs is a fantastic, self-contained raw wgpu pipeline. It should be extracted into a generic viz::shader::ShaderPipeline<U> where U is a bytemuck::Pod trait bound for the Uniform Buffer Object.
This base would automatically handle:
- WGSL compilation and wgpu layout boilerplate.
- Dual-texture ping-ponging for frame feedback.
- UBO layout mapping and writing.
pub struct ShaderPipeline<U> {
// pipeline, bindings, and ping-pong textures
}
impl<U: bytemuck::Pod> ShaderPipeline<U> {
pub fn new(device: &wgpu::Device, wgsl_source: &str) -> Self { ... }
pub fn render(&mut self, queue: &wgpu::Queue, ubo: &U) -> &wgpu::Texture { ... }
}
A new GPU-based visualizer would then only need to define its UBO struct, its .wgsl shader, and implement Visualizer::render_gpu.
3. Extracting Reusable Components
Several brilliant components are currently tightly coupled to breakcore.rs and scope.rs. They should be hoisted to common modules so any new visualizer can use them.
From breakcore.rs
-
Structure&Arch(Musical Archetype Classifier): The slow feature-vector classifier that detectsArch(Ambient, Build, Drop, Breakdown, Groove) and managestension/releaseis incredibly powerful. It converts raw FFT data into musical context.- Action: Move this to
src/viz/structure.rs(or evensrc/audio.rs). Any new visualizer should be able to querystructure.tensionto drive camera FOV, or trigger a massive event onArch::Drop.
- Action: Move this to
-
Critically Damped
Spring: TheSpringstruct applies physics-based smoothing to visual targets. It prevents jitter while maintaining immediate snap on transient hits.- Action: Move to a new
src/viz/math.rs. This is the perfect primitive for mapping raw audio bands to visual scales/rotations across all visualizers.
- Action: Move to a new
-
RK4 Attractors & Knots: The
Attr(Lorenz/Rössler) andKnotgeometry generators create stable, chaotic 3D paths.- Action: Move to a new
src/viz/geometry.rs.
- Action: Move to a new
From scope.rs
-
3D Vector Figures: The
Figuregenerator (Gielis supershapes, 3D Lissajous, Harmonographs) is excellent for generating deterministic, organic geometry.- Action: Move to
src/viz/geometry.rsalongside the attractors.
- Action: Move to
-
The Morphing State Machine: Both
scopeandbreakcoreimplement a pattern of holding a seed, tracking anidletimer or waiting for a transient, and smoothlymorphing to a new state.- Action: Abstract this into a
MorphState<T>primitive that wraps acurrentandtargetstate, handling the lerp/transition logic automatically.
- Action: Abstract this into a
Summary of Proposed Directory Structure
src/viz/
├── mod.rs
├── math.rs <-- (NEW) Spring, smoothing, interpolations
├── geometry.rs <-- (NEW) Figures, Knots, Attractors
├── structure.rs <-- (NEW) Arch, Structure classifier
├── shader.rs <-- (NEW) Generic wgpu ShaderPipeline
├── core.rs <-- (NEW) Visualizer trait and RenderContext
├── breakcore.rs (Refactored to use new bases)
├── scope.rs (Refactored to use new bases)
└── sigil.rs