diff --git a/Cargo.lock b/Cargo.lock index c8e6e9c..b014de6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -530,6 +530,8 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" name = "weld-core" version = "0.1.0" dependencies = [ + "crossbeam-queue", + "rustfft", "serde", "serde_json", ] diff --git a/README.md b/README.md index 5469577..07e5ba9 100644 --- a/README.md +++ b/README.md @@ -24,12 +24,11 @@ note. Each plugin is a separate product with its own version, guide and release. | **[Crucible](docs/crucible/README.md)** | Records the last bars all the time, then slices, cleans and exports key-labelled samples with one button | End of the chain to resample | none | [Guide](docs/crucible/README.md) · [Changes](docs/crucible/CHANGELOG.md) | | **[Lathe](docs/lathe/README.md)** | EQ with one band on each harmonic of the played note: a cut on harmonic 3 stays on harmonic 3 | Bass, after the synth | ≈ 67 ms | [Guide](docs/lathe/README.md) · [Changes](docs/lathe/CHANGELOG.md) | | **[Temper](docs/temper/README.md)** | Distortion that drives every note the same: level and tone filters follow the note | Bass | ≈ 67 ms + oversampling | [Guide](docs/temper/README.md) · [Changes](docs/temper/CHANGELOG.md) | -| **[Weld](docs/weld/README.md)** *(in development)* | Phase-aligns one bass layer to another layer through the sidechain, so the layers sum in the mids instead of cancelling | One layer's track | none | [Guide](docs/weld/README.md) · [Changes](docs/weld/CHANGELOG.md) | +| **[Weld](docs/weld/README.md)** | Phase-aligns one bass layer to another layer through the sidechain, so the layers sum in the mids instead of cancelling | One layer's track, other layer as sidechain | none | [Guide](docs/weld/README.md) · [Changes](docs/weld/CHANGELOG.md) | Plugins with latency report it to the host, so FL Studio compensates the delay. Anvil, Lathe and Temper detect pitch from the audio. They can also take the -pitch from MIDI. Weld is a sidechain test build at the moment; the phase -alignment comes next. +pitch from MIDI. The names come from metalwork: a **rivet** joins pieces; **patina** changes a surface and keeps its character; an **anvil** is the solid base that everything @@ -54,6 +53,10 @@ finished. Hosts list the plugins under the vendor name *Bass Foundry*. Lathe editor
Lathe: harmonic EQ Temper editor
Temper: tracked distortion + + Weld editor
Weld: layer phase alignment + + The previews are rendered headless from the real editor code by @@ -123,8 +126,8 @@ Every push and pull request runs these checks in CI - **Engine tests.** Every engine has tests for its signal behaviour and latency against measured targets. -- **Realtime safety.** Patina, Anvil, Crucible, Lathe and Temper have a test - that fails if the audio thread allocates memory. +- **Realtime safety.** Every audio effect has a test that fails if the audio + thread allocates memory. - **Host tests.** Each wrapper is loaded through the exported CLAP ABI, without a DAW. The tests check the ports, the reported latency, state save and load, and processing at 44.1, 48 and 96 kHz. diff --git a/docs/development.md b/docs/development.md index 50cc304..4466d70 100644 --- a/docs/development.md +++ b/docs/development.md @@ -32,7 +32,7 @@ NIH-plug Git dependency stays out of the root workspace. Host identities and saved state are separate for every product. The engines run allocation-free on the audio thread. Analysis that is too heavy -for the audio thread (Rivet's generation jobs, and Weld's planned phase fit) runs on a worker +for the audio thread (Rivet's generation jobs, Weld's cross-spectrum and fit) runs on a worker thread. Data goes to the editor through lock-free queues. ## Checks diff --git a/docs/ideas.md b/docs/ideas.md index 3b60fb8..a794618 100644 --- a/docs/ideas.md +++ b/docs/ideas.md @@ -67,16 +67,15 @@ Input level is normalised before the shaper, so drive does not depend on how loud the patch is, and the output is gain-matched. Oversampled to keep aliasing (which is never harmonic) out. See [Temper](temper/README.md). -## Ideas - -### Weld — layer phase alignment (problem 10) — in development +### Weld — layer phase alignment (problem 10) Takes a second bass layer through the sidechain. It fits polarity and a chain of up to four allpass sections to the measured cross-spectrum (no added delay), so the layers sum instead of cancel. A display shows how much level the cancellation cost and how much Weld recovered. Anvil solves the sub band; Weld -solves the mids. A sidechain test build exists; the fit comes after the FL -Studio routing test. See [Weld](weld/README.md). +solves the mids. See [Weld](weld/README.md). + +## Ideas ### Ballast — switch-up loudness leveller (problem 2) diff --git a/docs/releases.md b/docs/releases.md index a0a7511..8cd64cb 100644 --- a/docs/releases.md +++ b/docs/releases.md @@ -13,7 +13,7 @@ workflows are in `.gitea/workflows/`. A plugin's version is the `version` in | Crucible | `crucible-v0.1.0` | `crucible-windows-x86_64-0.1.0.zip` | | Lathe | `lathe-v0.1.0` | `lathe-windows-x86_64-0.1.0.zip` | | Temper | `temper-v0.1.0` | `temper-windows-x86_64-0.1.0.zip` | -| Weld | not released yet | | +| Weld | `weld-v0.1.0` | `weld-windows-x86_64-0.1.0.zip` | ## Make a release diff --git a/docs/weld/CHANGELOG.md b/docs/weld/CHANGELOG.md index 06e011d..9cd6a30 100644 --- a/docs/weld/CHANGELOG.md +++ b/docs/weld/CHANGELOG.md @@ -1,7 +1,15 @@ # Weld changelog -## Unreleased +## 0.1.0 -- Sidechain test build: stereo or mono main input with a matching sidechain - input, a sidechain level readout, Listen (Layer, Sum, Sum dry), Output and - Bypass. No phase alignment yet. +- Introduce Weld, a phase aligner for layered bass. +- Hears the other layer through the sidechain input and fits polarity plus up to + four allpass sections, so the layers sum instead of cancelling. No latency; + the layer on its own keeps its sound. +- Modes: Follow (refits only when clearly better), Learn (measure one take, + apply on exit), Hold. The alignment is saved with the project. +- Display: level per band against a perfect alignment (lost and won back), + phase before and after, recovered dB, cancellation cost, time offset, and + warnings for a late layer, unrelated layers and a missing sidechain. +- Controls: Mode, Sections, Response, Low, High, Gate, Listen (Layer, Sum, + Sum dry), Output, Bypass. diff --git a/docs/weld/README.md b/docs/weld/README.md index c8f0e2e..1c85af5 100644 --- a/docs/weld/README.md +++ b/docs/weld/README.md @@ -1,32 +1,110 @@ # Weld -Weld aligns the phase of one bass layer to another layer so the two sum -instead of cancelling in the mids. +Weld fuses two bass layers so they add up instead of cancelling. Current version +**0.1.0**. -> **Status: sidechain test build.** This build only checks that FL Studio -> delivers a sidechain signal to Weld. It passes the main layer through -> unchanged and shows the sidechain level. Phase alignment comes in the next -> build. +Bass patches are layered: a sub, one or more mid growls, a top. Where the layers +overlap in the mids, they partly cancel. The causes are a polarity difference, a +small time offset between the layers, and the phase shift of the filters and EQs +in one chain. The result is a thin, hollow sum that changes with every +switch-up. Anvil fixes this for the sub band; Weld fixes it for the mids. + +Weld sits on one layer and hears the other one through the sidechain input. It +measures how the two layers line up, then turns the phase of its own layer with +a polarity flip and up to four allpass filters. An allpass changes only phase, +so the layer on its own sounds the same; only the sum changes. Weld adds no +latency. + +![Weld](assets/editor.png) + +## Install on Windows + +Copy `Weld.clap` into `C:\Program Files\Common Files\CLAP\`. In FL Studio 2024.1 +or newer, open Options → Manage plugins, enable Verify plugins, and Find installed +plugins. Keep the font license with redistributed packages. ## Set up in FL Studio -1. Load Weld on the mixer track of one layer (the main layer). -2. On the mixer track of the other layer, right-click the route arrow to the - main layer's track and choose **Sidechain to this track**. -3. In Weld's plugin wrapper, open the settings (gear icon) and, under the - processing or input routing options, set Weld's sidechain input to that - track. -4. Play both layers. The large readout must show `sidechain: −NN dBFS`, and the - SIDECHAIN meter must move. +1. Put each layer on its own mixer track. Load Weld on the track of one layer, + the main layer. +2. Select the other layer's track. Right-click the route arrow under the main + layer's track and choose **Sidechain to this track**. The other layer still + goes to the master on its own; the sidechain only lets Weld hear it. +3. In Weld's plugin wrapper, open the settings (gear icon) and set Weld's + sidechain input to that track. +4. Play both layers. The SIDECHAIN meter must move. If the display says "No + sidechain signal", check step 3. + +## Work with it + +1. Leave **Mode** on **FOLLOW** and play the part where both layers sound. After + a second or two, Weld shows its alignment: polarity, and the allpass sections + with their frequency and Q. +2. Read the display: + - **Recovered** is how much level the sum gains, as an average per band + between Low and High. **Cancellation cost** is how much a perfect alignment + would gain. When the two numbers are close, Weld got nearly everything. + - The upper plot shows each band against a perfectly aligned sum (0 dB). Red + is what the unprocessed sum loses to cancellation; pink is the part Weld + wins back. The white line is the level with Weld. + - The lower plot shows the phase of this layer against the other one, before + (grey) and after (pink), and Weld's own phase curve (blue). After alignment + the pink line sits near 0°. +3. Compare by ear: mute the other layer's own route to the master, set + **Listen** to **SUM**, then to **SUM DRY**. Set Listen back to **LAYER** and + unmute the route for the mix. +4. When the alignment is right, switch to **HOLD**. It is saved with the project + and comes back when the project opens. + +For one drop or switch-up, use **LEARN**: press it, play the part, then press +**HOLD** (or LEARN again). Weld measures the whole take with equal weight and +applies the result when Learn ends. ## Controls -- **Listen**: - - **Layer** outputs the main layer; use it in the mix. - - **Sum** outputs the main layer plus the sidechain layer. Mute the other - layer's own send to the master while you listen to Sum. - - **Sum dry** outputs the unprocessed sum, for comparison. -- **Output**: output trim. -- **Bypass**. +- **Mode**: + - **Follow** keeps measuring and refits only when a new alignment is clearly + better (at least 0.3 dB on average), so the filter does not wander. + - **Learn** starts a fresh measurement and applies its result when Learn ends. + - **Hold** keeps the current alignment. The display keeps measuring. +- **Sections** (1–4) is the largest number of allpass sections the fit may use. + Weld uses fewer when fewer are enough. +- **Response** (0.3–5 s) is the averaging time in Follow. Longer is steadier. +- **Low** and **High** set the range the fit listens to. Low defaults to 60 Hz, + leaving the sub to Anvil. The allpass still acts below Low, and the display + shows that honestly. +- **Gate** (−70 to −30 dB): moments where either layer is quieter are not + measured, so tails and silence do not pull the fit. +- **Listen**: **Layer** outputs this layer, aligned, for the mix. **Sum** outputs + this layer aligned plus the sidechain layer. **Sum dry** outputs both + unprocessed. +- **Output** trims the level; **Bypass** passes this layer unchanged. -Weld adds no latency. +## Limits + +- **A late layer cannot be moved earlier.** An allpass only delays. If the layer + Weld sits on is the later one, the display shows "This layer is late" and the + offset in ms. Put Weld on the other layer instead. +- **Free-running layers cannot be aligned.** Without oscillator retrigger, or + with detuned unison, the phase between the layers drifts all the time. Weld + marks those bands as *loose* (hatched) and does not fit them. Enable retrigger + in the synth if you want Weld to align the layers. +- **Delays leave a comb.** A time offset between the layers cancels at regular + frequencies. A few allpass sections cannot follow a delay all the way up, so + Weld aligns the lower range and moves the higher notches rather than removing + them. For a large offset, move the layer in time first. +- **One alignment for the whole patch.** Weld does not change the alignment per + note. If switch-ups need different alignments, use Follow, or automate Mode. +- **The sidechain must arrive.** With no sidechain signal Weld changes nothing. + +## How it works + +Every 256 samples the audio thread hands mono copies of both layers to a worker +thread through a lock-free queue. The worker estimates their cross-spectrum +(Welch, Hann window, about 11.7 Hz per bin, 50 % overlap) and their coherence. +It fits polarity plus up to four second-order allpass sections so that +Σ w(f)·Re(S(f)·H(f)) is largest, with w = 1/f so every octave counts, ignoring +bins where the layers are not coherent, and with a penalty for any band that +would lose more than 1 dB. A new fit fades in over 40 ms after its filters have +settled for 40 ms, so there are no clicks. One fit takes about 15 ms on one +core, twice per second in Follow. diff --git a/docs/weld/assets/editor.png b/docs/weld/assets/editor.png new file mode 100644 index 0000000..f571c61 Binary files /dev/null and b/docs/weld/assets/editor.png differ diff --git a/plugins/weld/core/Cargo.toml b/plugins/weld/core/Cargo.toml index bd047af..9025a54 100644 --- a/plugins/weld/core/Cargo.toml +++ b/plugins/weld/core/Cargo.toml @@ -1,16 +1,14 @@ [package] - name = "weld-core" version = "0.1.0" edition.workspace = true license.workspace = true - [dependencies] - +crossbeam-queue = "0.3" +rustfft = "6.4" serde = { workspace = true } - [dev-dependencies] - +rustfft = "6.4" serde_json = { workspace = true } diff --git a/plugins/weld/core/src/analyzer.rs b/plugins/weld/core/src/analyzer.rs new file mode 100644 index 0000000..6922157 --- /dev/null +++ b/plugins/weld/core/src/analyzer.rs @@ -0,0 +1,429 @@ +//! The measuring half of Weld. Runs on the worker thread; deterministic, so tests drive +//! it directly. +//! +//! A Welch estimate of the cross-spectrum S = F_A·conj(F_B) and both auto-spectra runs +//! over Hann-windowed frames with 50 % overlap. Both layers share one complex FFT +//! (A in the real part, B in the imaginary part). +use crate::chain::Fit; +use crate::fit::{search, Point, Problem}; +use crate::report::{band_of, curve_frequency, Report, Status, BANDS}; +use crate::settings::{Mode, Settings}; +use rustfft::{num_complex::Complex32, Fft, FftPlanner}; +use std::sync::Arc; + +/// Bins below this coherence do not steer the fit. +const MIN_COHERENCE: f32 = 0.3; +/// Below this mean coherence the layers count as unrelated. +const LOOSE_COHERENCE: f32 = 0.3; +/// Frames needed before the first fit. +const MIN_FRAMES: u32 = 8; +/// A new fit replaces the current one only when it gains this much (Follow mode). +const FOLLOW_GAIN_DB: f32 = 0.3; +const LEARN_GAIN_DB: f32 = 0.1; +/// Lag search range and the smallest lag reported as "late". +const LAG_RANGE_S: f32 = 0.005; +const LATE_S: f32 = 0.0001; +const LATE_STRENGTH: f32 = 0.3; +/// Report and fit intervals in seconds of analysed audio. +const REPORT_S: f32 = 0.25; +const FIT_S: f32 = 0.5; +/// −90 dBFS: anything quieter on the sidechain counts as no signal. +const SILENCE: f32 = 3.2e-5; + +/// What a poll produced. +#[derive(Debug, Clone, Copy)] +pub struct Update { + /// Set when a new alignment was accepted. + pub fit: Option, + pub report: Report, +} + +pub struct Analyzer { + rate: f32, + n: usize, + hop: usize, + window: Vec, + ring_a: Vec, + ring_b: Vec, + write: usize, + filled: usize, + since_frame: usize, + fft: Arc>, + ifft: Arc>, + buf: Vec, + scratch: Vec, + sab: Vec, + saa: Vec, + sbb: Vec, + frames: u32, + settings: Settings, + current: Fit, + since_report: usize, + since_fit: usize, + finish_learn: bool, + side_level: f32, + gated: bool, +} + +impl Analyzer { + pub fn new(rate: f32, fit: Fit) -> Self { + // About 11.7 Hz per bin at any rate. + let n = ((4096. * rate / 48000.) as usize) + .next_power_of_two() + .clamp(1024, 65536); + let mut planner = FftPlanner::new(); + let fft = planner.plan_fft_forward(n); + let ifft = planner.plan_fft_inverse(n); + let scratch_len = fft + .get_inplace_scratch_len() + .max(ifft.get_inplace_scratch_len()); + let window = (0..n) + .map(|i| 0.5 - 0.5 * (std::f32::consts::TAU * i as f32 / n as f32).cos()) + .collect(); + let bins = n / 2 + 1; + Self { + rate, + n, + hop: n / 2, + window, + ring_a: vec![0.; n], + ring_b: vec![0.; n], + write: 0, + filled: 0, + since_frame: 0, + fft, + ifft, + buf: vec![Complex32::default(); n], + scratch: vec![Complex32::default(); scratch_len], + sab: vec![Complex32::default(); bins], + saa: vec![0.; bins], + sbb: vec![0.; bins], + frames: 0, + settings: Settings::default(), + current: fit.sanitized(rate), + since_report: 0, + since_fit: 0, + finish_learn: false, + side_level: 0., + gated: false, + } + } + pub fn fft_size(&self) -> usize { + self.n + } + pub fn current(&self) -> &Fit { + &self.current + } + pub fn frames(&self) -> u32 { + self.frames + } + pub fn set_settings(&mut self, settings: Settings) { + let old = self.settings.mode; + if settings.mode == Mode::Learn && old != Mode::Learn { + self.clear(); + } + if old == Mode::Learn && settings.mode != Mode::Learn { + self.finish_learn = true; + } + self.settings = settings; + } + /// Forgets the measurement. + pub fn clear(&mut self) { + self.sab.fill(Complex32::default()); + self.saa.fill(0.); + self.sbb.fill(0.); + self.frames = 0; + } + /// Feeds mono samples of the main layer `a` and the sidechain layer `b`. + pub fn push(&mut self, a: &[f32], b: &[f32]) { + for (&x, &y) in a.iter().zip(b) { + self.ring_a[self.write] = x; + self.ring_b[self.write] = y; + self.write = (self.write + 1) % self.n; + self.filled = (self.filled + 1).min(self.n); + self.since_frame += 1; + if self.filled == self.n && self.since_frame >= self.hop { + self.since_frame = 0; + self.frame(); + } + } + self.since_report += a.len().min(b.len()); + self.since_fit += a.len().min(b.len()); + } + fn frame(&mut self) { + let n = self.n; + let (mut ea, mut eb) = (0f32, 0f32); + for i in 0..n { + let j = (self.write + i) % n; + let (a, b) = (self.ring_a[j], self.ring_b[j]); + ea += a * a; + eb += b * b; + self.buf[i] = Complex32::new(a * self.window[i], b * self.window[i]); + } + let (ra, rb) = ((ea / n as f32).sqrt(), (eb / n as f32).sqrt()); + self.side_level = rb; + let gate = 10f32.powf(self.settings.gate / 20.); + self.gated = ra < gate || rb < gate; + if self.gated { + return; + } + self.fft + .process_with_scratch(&mut self.buf, &mut self.scratch); + self.frames = self.frames.saturating_add(1); + let count = 1. / self.frames as f32; + let alpha = match self.settings.mode { + Mode::Learn => count, + _ => { + let follow = 1. - (-(self.hop as f32) / (self.settings.response * self.rate)).exp(); + count.max(follow) + } + }; + for k in 0..=n / 2 { + let x = self.buf[k]; + let xc = self.buf[(n - k) % n].conj(); + let fa = (x + xc) * 0.5; + let fb = (x - xc) * Complex32::new(0., -0.5); + let old = self.sab[k]; + self.sab[k] = old + (fa * fb.conj() - old) * alpha; + self.saa[k] += (fa.norm_sqr() - self.saa[k]) * alpha; + self.sbb[k] += (fb.norm_sqr() - self.sbb[k]) * alpha; + } + } + fn frequency(&self, k: usize) -> f32 { + k as f32 * self.rate / self.n as f32 + } + fn coherence(&self, k: usize) -> f32 { + let d = self.saa[k] * self.sbb[k]; + if d > 0. { + (self.sab[k].norm_sqr() / d).min(1.) + } else { + 0. + } + } + fn range(&self) -> (f32, f32) { + let high = self.settings.high.min(0.45 * self.rate); + (self.settings.low.min(high * 0.5), high) + } + /// Fit points between Low and High: one per bin at the bottom, merged to 1/32 octave + /// higher up. + fn points(&self) -> Vec { + let (low, high) = self.range(); + let mut points: Vec = Vec::new(); + let mut last = usize::MAX; + for k in 1..=self.n / 2 { + let f = self.frequency(k); + if f < low || f > high { + continue; + } + let slot = (32. * (f / low).log2()) as usize; + if slot != last { + points.push(Point::new(f, self.rate)); + last = slot; + } + let p = points.last_mut().expect("pushed above"); + if self.coherence(k) >= MIN_COHERENCE { + p.fit += self.sab[k]; + } + p.all += self.sab[k]; + p.power += self.saa[k] + self.sbb[k]; + } + points + } + /// Produces an update once enough audio has been analysed since the last one. + pub fn poll(&mut self) -> Option { + let report_due = self.since_report as f32 >= REPORT_S * self.rate; + if !report_due && !self.finish_learn { + return None; + } + self.since_report = 0; + let fit_due = self.since_fit as f32 >= FIT_S * self.rate; + let mode = self.settings.mode; + let threshold = if self.finish_learn { + Some(LEARN_GAIN_DB) + } else if fit_due && mode == Mode::Follow { + Some(FOLLOW_GAIN_DB) + } else if fit_due && mode == Mode::Learn { + Some(LEARN_GAIN_DB) + } else { + None + }; + if fit_due { + self.since_fit = 0; + } + self.finish_learn = false; + Some(self.analyse(threshold)) + } + /// Measures, fits when `threshold` is given (minimum predicted gain in dB over the + /// current fit), and reports. + pub fn analyse(&mut self, threshold: Option) -> Update { + let points = self.points(); + let problem = Problem::new(points, self.rate); + let (weight, coherent) = problem.points.iter().fold((0., 0.), |(w, c), p| { + let x = p.all.norm(); + (w + p.w * x, c + p.w * x * coherence_of(p)) + }); + let mean_coherence = if weight > 0. { coherent / weight } else { 0. }; + let (lag_ms, lag_strength) = self.lag(); + let late = lag_ms > LATE_S * 1000. && lag_strength > LATE_STRENGTH; + let status = if self.side_level < SILENCE && self.frames == 0 { + Status::NoSidechain + } else if self.frames < MIN_FRAMES { + if self.gated { + Status::Quiet + } else { + Status::Waiting + } + } else if mean_coherence < LOOSE_COHERENCE || problem.is_empty() { + Status::Loose + } else if late { + Status::Late + } else { + Status::Aligned + }; + let mut changed = None; + if let Some(threshold) = threshold { + if matches!(status, Status::Aligned | Status::Late) { + let candidate = search(&problem, self.settings.sections, &self.current); + let gain = problem.mean_gain( + &problem.bands(&self.current), + &problem.bands(&candidate.fit), + ); + if gain.is_finite() && gain >= threshold && candidate.fit != self.current { + self.current = candidate.fit; + changed = Some(candidate.fit); + } + } + } + let mut report = self.report(status, &problem); + report.lag_ms = lag_ms; + report.lag_strength = lag_strength; + report.changed = changed.is_some(); + Update { + fit: changed, + report, + } + } + fn report(&self, status: Status, problem: &Problem) -> Report { + let fit = self.current; + let mut report = Report { + status, + fit, + frames: self.frames, + low: self.range().0, + high: self.range().1, + ..Report::default() + }; + // Display bands over all bins. + // Sums per band: before, after, ideal, coherence·power, power. + let mut sums = [[0f32; 5]; BANDS]; + for k in 1..=self.n / 2 { + let f = self.frequency(k); + if !(20. ..=0.45 * self.rate).contains(&f) { + continue; + } + let (hr, hi) = fit.response(f, self.rate); + let s = self.sab[k]; + let p = self.saa[k] + self.sbb[k]; + let b = &mut sums[band_of(f)]; + b[0] += p + 2. * s.re; + b[1] += p + 2. * (s * Complex32::new(hr, hi)).re; + b[2] += p + 2. * s.norm(); + b[3] += self.coherence(k) * p; + b[4] += p; + } + for (band, [before, after, ideal, coherent, power]) in report.bands.iter_mut().zip(sums) { + if ideal > 1e-12 && power > 0. { + band.before = db_ratio(before, ideal); + band.after = db_ratio(after, ideal); + band.coherence = (coherent / power).clamp(0., 1.); + band.valid = true; + } + } + // Phase curve: bins within ±1/24 octave of each point. + for (i, point) in report.curve.iter_mut().enumerate() { + let f = curve_frequency(i); + if f > 0.45 * self.rate { + continue; + } + let lo = ((f * 2f32.powf(-1. / 24.)) * self.n as f32 / self.rate).floor() as usize; + let hi = ((f * 2f32.powf(1. / 24.)) * self.n as f32 / self.rate).ceil() as usize; + let centre = (f * self.n as f32 / self.rate).round() as usize; + let (lo, hi) = ( + lo.max(1).min(centre.max(1)), + hi.min(self.n / 2).max(centre.max(1)), + ); + let (mut s, mut pa, mut pb) = (Complex32::default(), 0f32, 0f32); + for k in lo..=hi { + s += self.sab[k]; + pa += self.saa[k]; + pb += self.sbb[k]; + } + if pa * pb <= 0. { + continue; + } + let (hr, hi) = fit.response(f, self.rate); + let h = Complex32::new(hr, hi); + point.before = s.arg().to_degrees(); + point.after = (s * h).arg().to_degrees(); + point.chain = h.arg().to_degrees(); + point.coherence = (s.norm_sqr() / (pa * pb)).min(1.); + point.valid = true; + } + // Headline numbers over the fit range, every band weighted the same. + if !problem.is_empty() { + let before = problem.before_bands(); + report.cost_db = problem.mean_gain(&before, &problem.ideal_bands()); + report.recovered_db = problem.mean_gain(&before, &problem.bands(&fit)); + } + report + } + /// Time offset from the phase transform of the cross-spectrum (GCC-PHAT) between + /// Low and High. Returns (ms, strength); positive means this layer is late. + fn lag(&mut self) -> (f32, f32) { + let n = self.n; + let (low, high) = self.range(); + self.buf.fill(Complex32::default()); + let mut used = 0usize; + for k in 1..n / 2 { + let f = self.frequency(k); + if f < low || f > high || self.coherence(k) < MIN_COHERENCE { + continue; + } + let s = self.sab[k]; + let m = s.norm(); + if m <= 0. { + continue; + } + let u = s / m; + self.buf[k] = u; + self.buf[n - k] = u.conj(); + used += 1; + } + if used < 4 { + return (0., 0.); + } + self.ifft + .process_with_scratch(&mut self.buf, &mut self.scratch); + let range = (LAG_RANGE_S * self.rate) as isize; + let (mut best, mut at) = (0f32, 0isize); + for tau in -range..=range { + let v = self.buf[tau.rem_euclid(n as isize) as usize].re.abs(); + if v > best { + best = v; + at = tau; + } + } + (at as f32 / self.rate * 1000., best / (2 * used) as f32) + } +} + +fn coherence_of(p: &Point) -> f32 { + if p.all.norm() > 0. { + p.fit.norm() / p.all.norm() + } else { + 0. + } +} +fn db_ratio(a: f32, b: f32) -> f32 { + (10. * (a.max(1e-20) / b.max(1e-20)).log10()).clamp(-40., 40.) +} diff --git a/plugins/weld/core/src/chain.rs b/plugins/weld/core/src/chain.rs new file mode 100644 index 0000000..472b22c --- /dev/null +++ b/plugins/weld/core/src/chain.rs @@ -0,0 +1,163 @@ +//! Polarity plus a cascade of second-order allpass sections. Magnitude is flat by +//! construction; only the phase of the main layer changes. +use serde::{Deserialize, Serialize}; + +pub const MAX_SECTIONS: usize = 4; +pub const MIN_FC: f32 = 30.; +pub const MIN_Q: f32 = 0.2; +pub const MAX_Q: f32 = 4.; + +#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize)] +pub struct Section { + /// Centre frequency in Hz, where the section's phase passes −180°. + pub fc: f32, + pub q: f32, +} +impl Default for Section { + fn default() -> Self { + Self { fc: 1000., q: 0.7 } + } +} + +/// The result of a fit: what Weld applies to the main layer. +#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize, Default)] +pub struct Fit { + /// Invert the main layer. + pub flip: bool, + /// Number of active sections, 0..=MAX_SECTIONS. + pub count: usize, + pub sections: [Section; MAX_SECTIONS], +} +impl Fit { + /// The identity: no polarity change, no sections. + pub const NONE: Fit = Fit { + flip: false, + count: 0, + sections: [Section { fc: 1000., q: 0.7 }; MAX_SECTIONS], + }; + pub fn active(&self) -> &[Section] { + &self.sections[..self.count.min(MAX_SECTIONS)] + } + /// Clamps every value into the range the filter supports; repairs restored state. + pub fn sanitized(mut self, rate: f32) -> Self { + self.count = self.count.min(MAX_SECTIONS); + let top = max_fc(rate); + for s in &mut self.sections { + s.fc = if s.fc.is_finite() { + s.fc.clamp(MIN_FC, top) + } else { + 1000f32.min(top) + }; + s.q = if s.q.is_finite() { + s.q.clamp(MIN_Q, MAX_Q) + } else { + 0.7 + }; + } + self + } + /// Complex response at `f` Hz as (re, im). + pub fn response(&self, f: f32, rate: f32) -> (f32, f32) { + let w = std::f32::consts::TAU * f / rate; + let (mut re, mut im) = (if self.flip { -1. } else { 1. }, 0.); + for s in self.active() { + let c = Coefs::allpass(s.fc, s.q, rate); + let (hr, hi) = c.response(w.cos(), w.sin(), (2. * w).cos(), (2. * w).sin()); + (re, im) = (re * hr - im * hi, re * hi + im * hr); + } + (re, im) + } +} + +pub fn max_fc(rate: f32) -> f32 { + 0.45 * rate +} + +/// Normalised allpass coefficients: H(z) = (a2 + a1 z⁻¹ + z⁻²) / (1 + a1 z⁻¹ + a2 z⁻²). +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct Coefs { + pub a1: f32, + pub a2: f32, +} +impl Coefs { + /// RBJ cookbook allpass. + pub fn allpass(fc: f32, q: f32, rate: f32) -> Self { + let fc = fc.clamp(MIN_FC, max_fc(rate)); + let q = q.clamp(MIN_Q, MAX_Q); + let w = std::f32::consts::TAU * fc / rate; + let alpha = w.sin() / (2. * q); + let a0 = 1. + alpha; + Self { + a1: -2. * w.cos() / a0, + a2: (1. - alpha) / a0, + } + } + /// Response at z⁻¹ = e^{−jω}, given cos/sin of ω and 2ω. + /// For an allpass the numerator is z⁻²·conj(den), so H = z⁻²·conj(den)/den. + #[inline] + pub fn response(&self, c1: f32, s1: f32, c2: f32, s2: f32) -> (f32, f32) { + let x = 1. + self.a1 * c1 + self.a2 * c2; + let y = -(self.a1 * s1 + self.a2 * s2); + let m = x * x + y * y; + // conj(den)/den = conj(den)² / |den|² + let (qr, qi) = ((x * x - y * y) / m, -2. * x * y / m); + // times z⁻² = c2 − j s2 + (qr * c2 + qi * s2, qi * c2 - qr * s2) + } +} + +/// Transposed direct form II state, in f64 so sections near 30 Hz stay quiet. +#[derive(Debug, Clone, Copy, Default)] +struct State { + s1: f64, + s2: f64, +} + +/// A running chain for up to two channels. Allocation-free. +#[derive(Debug, Clone)] +pub struct Chain { + fit: Fit, + sign: f32, + coefs: [(f64, f64); MAX_SECTIONS], + state: [[State; MAX_SECTIONS]; 2], +} +impl Chain { + pub fn new(fit: Fit, rate: f32) -> Self { + let mut chain = Self { + fit: Fit::NONE, + sign: 1., + coefs: [(0., 0.); MAX_SECTIONS], + state: [[State::default(); MAX_SECTIONS]; 2], + }; + chain.set(fit, rate); + chain + } + pub fn fit(&self) -> &Fit { + &self.fit + } + /// Replaces the coefficients and clears the state. + pub fn set(&mut self, fit: Fit, rate: f32) { + self.fit = fit.sanitized(rate); + self.sign = if self.fit.flip { -1. } else { 1. }; + for (c, s) in self.coefs.iter_mut().zip(self.fit.active()) { + let k = Coefs::allpass(s.fc, s.q, rate); + *c = (k.a1 as f64, k.a2 as f64); + } + self.reset(); + } + pub fn reset(&mut self) { + self.state = [[State::default(); MAX_SECTIONS]; 2]; + } + #[inline] + pub fn process(&mut self, channel: usize, x: f32) -> f32 { + let mut y = (self.sign * x) as f64; + for (k, &(a1, a2)) in self.coefs[..self.fit.count].iter().enumerate() { + let st = &mut self.state[channel][k]; + let out = a2 * y + st.s1; + st.s1 = a1 * y - a1 * out + st.s2; + st.s2 = y - a2 * out; + y = out; + } + y as f32 + } +} diff --git a/plugins/weld/core/src/fit.rs b/plugins/weld/core/src/fit.rs new file mode 100644 index 0000000..2ba8d30 --- /dev/null +++ b/plugins/weld/core/src/fit.rs @@ -0,0 +1,372 @@ +//! Fits polarity plus allpass sections to a measured cross-spectrum. +//! +//! With S = F_A·conj(F_B), the power of the sum after rotating A by H is +//! |A|² + |B|² + 2·Re(S·H). The fit maximises Σ w(f)·Re(S·H) with w = 1/f, so every +//! octave counts the same. Flipping polarity negates the sum, so the best polarity is +//! the sign of the result and only the sections need searching. +use crate::chain::{max_fc, Coefs, Fit, Section, MAX_Q, MAX_SECTIONS, MIN_FC, MIN_Q}; +use crate::report::{band_of, BANDS, DISPLAY_HIGH, DISPLAY_LOW}; + +/// Bands for the loss penalty. Narrower bands make the penalty brittle next to the +/// notches of a comb, which a delay between the layers always has. +const FINE: usize = 10; +fn fine_of(f: f32) -> usize { + let t = (f / DISPLAY_LOW).ln() / (DISPLAY_HIGH / DISPLAY_LOW).ln(); + ((t * FINE as f32).max(0.) as usize).min(FINE - 1) +} +use rustfft::num_complex::Complex32; + +/// One fit point: bins merged over at most 1/32 octave. +#[derive(Debug, Clone, Copy)] +pub(crate) struct Point { + pub c1: f32, + pub s1: f32, + pub c2: f32, + pub s2: f32, + /// 1/f weight. + pub w: f32, + /// Cross-spectrum of the coherent bins only; drives the objective. + pub fit: Complex32, + /// Cross-spectrum and summed auto-spectra of all bins; drive the predicted levels. + pub all: Complex32, + pub power: f32, + pub band: usize, + pub fine: usize, +} +impl Point { + pub fn new(f: f32, rate: f32) -> Self { + let w = std::f32::consts::TAU * f / rate; + Self { + c1: w.cos(), + s1: w.sin(), + c2: (2. * w).cos(), + s2: (2. * w).sin(), + w: 1. / f, + fit: Complex32::default(), + all: Complex32::default(), + power: 0., + band: band_of(f), + fine: fine_of(f), + } + } +} + +pub(crate) struct Problem { + pub points: Vec, + pub rate: f32, + /// Σ w·|S| over the coherent bins: the objective of a perfect per-bin alignment. + norm: f32, + /// Per-band level of the unprocessed sum. + before: [f32; BANDS], + /// The same in third-octave bands, for the loss penalty. + before_fine: [f32; FINE], +} + +/// Score of a perfectly aligned sum is 1. Losing more than 1 dB in any band against the +/// unprocessed sum costs extra, so a fit cannot win the mids by giving away the top. +const LOSS_FREE_DB: f32 = 1.0; +const LOSS_WEIGHT: f32 = 0.05; +/// The smallest model that reaches this share of the best model's improvement wins. +const MODEL_SHARE: f32 = 0.95; +/// Improvements below this score never justify another section. +const MODEL_FLOOR: f32 = 0.001; + +#[derive(Debug, Clone, Copy)] +pub struct Candidate { + pub fit: Fit, + pub score: f32, +} + +impl Problem { + pub fn new(points: Vec, rate: f32) -> Self { + let norm = points.iter().map(|p| p.w * p.fit.norm()).sum::(); + let mut before = [0f32; BANDS]; + let mut before_fine = [0f32; FINE]; + for p in &points { + before[p.band] += p.power + 2. * p.all.re; + before_fine[p.fine] += p.power + 2. * p.all.re; + } + Self { + points, + rate, + norm, + before, + before_fine, + } + } + pub fn is_empty(&self) -> bool { + self.norm <= 0. + } + fn coefs(&self, x: &[f32], out: &mut [Coefs; MAX_SECTIONS]) -> (usize, f32) { + let k = x.len() / 2; + let mut outside = 0.; + let (lo_f, hi_f) = (MIN_FC.ln(), max_fc(self.rate).ln()); + let (lo_q, hi_q) = (MIN_Q.ln(), MAX_Q.ln()); + for i in 0..k { + let (f, q) = (x[2 * i], x[2 * i + 1]); + outside += + (lo_f - f).max(0.) + (f - hi_f).max(0.) + (lo_q - q).max(0.) + (q - hi_q).max(0.); + out[i] = Coefs::allpass( + f.clamp(lo_f, hi_f).exp(), + q.clamp(lo_q, hi_q).exp(), + self.rate, + ); + } + (k, outside) + } + #[inline] + fn response(p: &Point, coefs: &[Coefs]) -> Complex32 { + let mut h = Complex32::new(1., 0.); + for c in coefs { + let (re, im) = c.response(p.c1, p.s1, p.c2, p.s2); + h *= Complex32::new(re, im); + } + h + } + /// Score and polarity for sections given as [ln fc, ln q, …]. + fn score(&self, x: &[f32]) -> (f32, bool) { + let mut coefs = [Coefs { a1: 0., a2: 0. }; MAX_SECTIONS]; + let (k, outside) = self.coefs(x, &mut coefs); + let coefs = &coefs[..k]; + let mut j = 0f32; + let mut cross = [0f32; FINE]; + let mut power = [0f32; FINE]; + for p in &self.points { + let h = Self::response(p, coefs); + j += p.w * (p.fit * h).re; + cross[p.fine] += (p.all * h).re; + power[p.fine] += p.power; + } + let flip = j < 0.; + let sign = if flip { -1. } else { 1. }; + let mut penalty = outside * outside; + for b in 0..FINE { + let before = self.before_fine[b]; + if before <= 0. { + continue; + } + let after = (power[b] + 2. * sign * cross[b]).max(before * 1e-6); + let loss = 10. * (before / after).log10(); + if loss > LOSS_FREE_DB { + penalty += LOSS_WEIGHT * (loss - LOSS_FREE_DB).powi(2); + } + } + (j.abs() / self.norm - penalty, flip) + } + fn candidate(&self, x: &[f32]) -> Candidate { + let (score, flip) = self.score(x); + let mut fit = Fit { + flip, + count: x.len() / 2, + ..Fit::NONE + }; + for (i, s) in fit.sections[..fit.count].iter_mut().enumerate() { + *s = Section { + fc: x[2 * i].exp().clamp(MIN_FC, max_fc(self.rate)), + q: x[2 * i + 1].exp().clamp(MIN_Q, MAX_Q), + }; + } + Candidate { fit, score } + } + /// Level of the sum per display band with `fit` applied (sign included). + pub fn bands(&self, fit: &Fit) -> [f32; BANDS] { + let mut coefs = [Coefs { a1: 0., a2: 0. }; MAX_SECTIONS]; + for (c, s) in coefs.iter_mut().zip(fit.active()) { + *c = Coefs::allpass(s.fc, s.q, self.rate); + } + let coefs = &coefs[..fit.active().len()]; + let sign = if fit.flip { -1. } else { 1. }; + let mut out = [0f32; BANDS]; + for p in &self.points { + out[p.band] += p.power + 2. * sign * (p.all * Self::response(p, coefs)).re; + } + out + } + /// Level per display band of a perfectly aligned sum. + pub fn ideal_bands(&self) -> [f32; BANDS] { + let mut out = [0f32; BANDS]; + for p in &self.points { + out[p.band] += p.power + 2. * p.all.norm(); + } + out + } + /// Mean change in dB over the bands that carry signal, from `from` to `to`. Every + /// band counts the same, so a hollow octave in the mids weighs as much as the sub. + pub fn mean_gain(&self, from: &[f32; BANDS], to: &[f32; BANDS]) -> f32 { + let total: f32 = self.before.iter().sum(); + let (mut sum, mut count) = (0f32, 0); + for b in 0..BANDS { + if self.before[b] > total * 1e-4 && from[b] > 0. { + sum += 10. * (to[b].max(from[b] * 1e-6) / from[b]).log10(); + count += 1; + } + } + if count > 0 { + sum / count as f32 + } else { + 0. + } + } + pub fn before_bands(&self) -> [f32; BANDS] { + self.before + } + /// The frequency where the current sections leave the most cross-energy unaligned, + /// smoothed over one third of an octave. + fn worst_frequency(&self, x: &[f32]) -> f32 { + let mut coefs = [Coefs { a1: 0., a2: 0. }; MAX_SECTIONS]; + let (k, _) = self.coefs(x, &mut coefs); + let (_, flip) = self.score(x); + let sign = if flip { -1. } else { 1. }; + let mut buckets = [0f32; 48]; + let mut first = None; + for p in &self.points { + let h = Self::response(p, &coefs[..k]); + let residual = p.w * (p.fit.norm() - sign * (p.fit * h).re); + let f = 1. / p.w; + let low = *first.get_or_insert(f); + let i = ((3. * (f / low).log2()) as usize).min(47); + buckets[i] += residual; + } + let low = first.unwrap_or(100.); + let best = (0..48) + .max_by(|&a, &b| buckets[a].total_cmp(&buckets[b])) + .unwrap_or(0); + low * 2f32.powf((best as f32 + 0.5) / 3.) + } +} + +/// Searches 0..=`max_sections` sections, warm-started from `warm`. Deterministic. +pub(crate) fn search(problem: &Problem, max_sections: usize, warm: &Fit) -> Candidate { + let max_sections = max_sections.min(MAX_SECTIONS); + let mut best: Vec = Vec::with_capacity(max_sections + 1); + best.push(problem.candidate(&[])); + let mut previous: Vec = Vec::new(); + for k in 1..=max_sections { + let mut seeds: Vec> = Vec::new(); + let worst = problem.worst_frequency(&previous); + for f in [worst, worst * 0.4, worst * 2.5] { + let mut x = previous.clone(); + x.extend([f.ln(), 0.7f32.ln()]); + seeds.push(x); + } + if k == 1 { + // Low-Q sections high up act like a short delay below their centre. + for (f, q) in [ + (120f32, 0.7f32), + (400., 0.7), + (1500., 0.7), + (3000., 0.5), + (6000., 0.5), + ] { + seeds.push(vec![f.ln(), q.ln()]); + } + } + if warm.count == k { + seeds.push( + warm.active() + .iter() + .flat_map(|s| [s.fc.ln(), s.q.ln()]) + .collect(), + ); + } + let mut top: Option<(Vec, f32)> = None; + for seed in seeds { + let (x, value) = nelder_mead(|x| -problem.score(x).0, &seed, 0.35, 120 * k * 2); + if top.as_ref().is_none_or(|(_, v)| value < *v) { + top = Some((x, value)); + } + } + let (x, _) = top.expect("at least one seed"); + let candidate = problem.candidate(&x); + let top = best.iter().map(|c| c.score).fold(f32::MIN, f32::max); + let gained = candidate.score - top; + let slack = MODEL_FLOOR.max((1. - MODEL_SHARE) * (top - best[0].score)); + best.push(candidate); + previous = x; + // A section that adds next to nothing is not worth it; more of them rarely are. + if k >= 2 && gained < slack { + break; + } + } + let top = best.iter().map(|c| c.score).fold(f32::MIN, f32::max); + let slack = MODEL_FLOOR.max((1. - MODEL_SHARE) * (top - best[0].score)); + *best + .iter() + .find(|c| c.score >= top - slack) + .expect("the best model qualifies") +} + +/// Downhill simplex. Returns the best point and its value. +fn nelder_mead( + mut f: impl FnMut(&[f32]) -> f32, + x0: &[f32], + step: f32, + max_evals: usize, +) -> (Vec, f32) { + let n = x0.len(); + let mut simplex: Vec<(Vec, f32)> = Vec::with_capacity(n + 1); + simplex.push((x0.to_vec(), f(x0))); + for i in 0..n { + let mut x = x0.to_vec(); + x[i] += step; + let v = f(&x); + simplex.push((x, v)); + } + let mut evals = n + 1; + let point = |a: &[f32], b: &[f32], t: f32| -> Vec { + a.iter().zip(b).map(|(a, b)| a + t * (b - a)).collect() + }; + while evals < max_evals { + simplex.sort_by(|a, b| a.1.total_cmp(&b.1)); + let spread = simplex[n].1 - simplex[0].1; + let size = simplex[1..] + .iter() + .flat_map(|(x, _)| x.iter().zip(&simplex[0].0).map(|(a, b)| (a - b).abs())) + .fold(0f32, f32::max); + if spread < 1e-7 && size < 1e-3 { + break; + } + let mut centroid = vec![0f32; n]; + for (x, _) in &simplex[..n] { + for (c, v) in centroid.iter_mut().zip(x) { + *c += v / n as f32; + } + } + let worst = simplex[n].clone(); + let reflected = point(¢roid, &worst.0, -1.); + let fr = f(&reflected); + evals += 1; + if fr < simplex[0].1 { + let expanded = point(¢roid, &worst.0, -2.); + let fe = f(&expanded); + evals += 1; + simplex[n] = if fe < fr { + (expanded, fe) + } else { + (reflected, fr) + }; + } else if fr < simplex[n - 1].1 { + simplex[n] = (reflected, fr); + } else { + let contracted = if fr < worst.1 { + point(¢roid, &reflected, 0.5) + } else { + point(¢roid, &worst.0, 0.5) + }; + let fc = f(&contracted); + evals += 1; + if fc < worst.1.min(fr) { + simplex[n] = (contracted, fc); + } else { + let best = simplex[0].0.clone(); + for (x, v) in &mut simplex[1..] { + *x = point(&best, x, 0.5); + *v = f(x); + } + evals += n; + } + } + } + simplex.sort_by(|a, b| a.1.total_cmp(&b.1)); + simplex.swap_remove(0) +} diff --git a/plugins/weld/core/src/lib.rs b/plugins/weld/core/src/lib.rs index c240a6e..1ab6093 100644 --- a/plugins/weld/core/src/lib.rs +++ b/plugins/weld/core/src/lib.rs @@ -1,9 +1,79 @@ //! Weld engine: aligns the phase of the main layer to a sidechain layer so the two sum -//! instead of cancel. Allocation-free after `new`; zero latency. +//! instead of cancelling. The audio thread runs polarity plus an allpass chain and hands +//! mono blocks to a worker thread, which measures the cross-spectrum and fits the chain. +//! Allocation-free on the audio thread after `new`; zero latency. +mod analyzer; +mod chain; +mod fit; +mod report; mod settings; -pub use settings::{Listen, Settings}; +mod worker; +pub use analyzer::{Analyzer, Update}; +pub use chain::{Chain, Fit, Section, MAX_SECTIONS}; +pub use report::{ + band_edge, curve_frequency, Band, CurvePoint, Report, Status, BANDS, CURVE, DISPLAY_HIGH, + DISPLAY_LOW, +}; +pub use settings::{Listen, Mode, Settings}; +pub use worker::Worker; -/// Levels for the editor, published a few times per second. +use crossbeam_queue::ArrayQueue; +use std::sync::{ + atomic::{AtomicUsize, Ordering}, + Arc, +}; + +/// Samples per block handed to the worker. +pub const BLOCK: usize = 256; +/// Blocks the queue holds: about 340 ms at 48 kHz. +const QUEUE_BLOCKS: usize = 64; +/// A new fit runs silently this long before it is faded in, so its filters settle. +const WARM_S: f32 = 0.04; +/// Crossfade time from the old fit to the new one. +const FADE_S: f32 = 0.04; + +/// Mono blocks of both layers, before Weld's processing. +#[derive(Clone, Copy)] +pub struct Block { + pub a: [f32; BLOCK], + pub b: [f32; BLOCK], +} + +/// Lock-free queues between one engine and its worker. +pub struct Link { + pub(crate) blocks: ArrayQueue, + pub(crate) fits: ArrayQueue, + pub(crate) settings: ArrayQueue, + dropped: AtomicUsize, +} +impl Link { + fn new() -> Self { + Self { + blocks: ArrayQueue::new(QUEUE_BLOCKS), + fits: ArrayQueue::new(4), + settings: ArrayQueue::new(1), + dropped: AtomicUsize::new(0), + } + } + /// Blocks dropped because the worker fell behind. + pub fn dropped(&self) -> usize { + self.dropped.load(Ordering::Relaxed) + } + /// Takes the next block, as the worker does. For tests and offline tools. + pub fn pop_block(&self) -> Option { + self.blocks.pop() + } + /// Takes the latest settings, as the worker does. For tests and offline tools. + pub fn pop_settings(&self) -> Option { + self.settings.pop() + } + /// Hands a fit to the engine, as the worker does. For tests and offline tools. + pub fn push_fit(&self, fit: Fit) { + self.fits.force_push(fit); + } +} + +/// Levels for the editor, published a few times per second by the audio thread. #[derive(Debug, Clone, Copy, PartialEq)] pub struct AnalyzerFrame { /// RMS of the main input in dBFS since the last frame. @@ -14,6 +84,8 @@ pub struct AnalyzerFrame { pub side_peak_db: f32, /// True once any sidechain sample above −90 dBFS arrived since activation. pub side_seen: bool, + /// Blocks the worker could not take in time, since activation. + pub dropped: usize, pub sample_rate: f32, } impl Default for AnalyzerFrame { @@ -23,6 +95,7 @@ impl Default for AnalyzerFrame { side_db: -120., side_peak_db: -120., side_seen: false, + dropped: 0, sample_rate: 48000., } } @@ -46,16 +119,34 @@ struct Levels { count: usize, } +#[derive(Debug, Clone, Copy, PartialEq)] +enum Phase { + Steady, + /// The next chain runs unheard for this many more samples. + Warm(usize), + /// The next chain fades in over this many more samples. + Fade(usize), +} + pub struct Engine { rate: f32, channels: usize, settings: Settings, + sent: Option, gain: f32, target_gain: f32, gain_coef: f32, fresh: bool, levels: Levels, side_seen: bool, + current: Chain, + next: Chain, + phase: Phase, + warm: usize, + fade: usize, + link: Arc, + block: Block, + fill: usize, } impl Engine { pub fn new(rate: f32, channels: usize) -> Result { @@ -69,12 +160,24 @@ impl Engine { rate, channels, settings: Settings::default(), + sent: None, gain: 1., target_gain: 1., gain_coef: 1. - (-1. / (0.02 * rate)).exp(), fresh: true, levels: Levels::default(), side_seen: false, + current: Chain::new(Fit::NONE, rate), + next: Chain::new(Fit::NONE, rate), + phase: Phase::Steady, + warm: ((WARM_S * rate) as usize).max(1), + fade: ((FADE_S * rate) as usize).max(1), + link: Arc::new(Link::new()), + block: Block { + a: [0.; BLOCK], + b: [0.; BLOCK], + }, + fill: 0, }) } pub const fn latency_samples(&self) -> u32 { @@ -83,9 +186,28 @@ impl Engine { pub fn channels(&self) -> usize { self.channels } + pub fn sample_rate(&self) -> f32 { + self.rate + } pub fn settings(&self) -> &Settings { &self.settings } + /// The queues for this engine's worker. + pub fn link(&self) -> Arc { + self.link.clone() + } + /// The alignment heard now (the target one during a crossfade). + pub fn fit(&self) -> Fit { + match self.phase { + Phase::Steady => *self.current.fit(), + _ => *self.next.fit(), + } + } + /// Applies `fit` at once, without a crossfade. For restoring saved state. + pub fn set_fit(&mut self, fit: Fit) { + self.current.set(fit, self.rate); + self.phase = Phase::Steady; + } pub fn set_settings(&mut self, settings: Settings) -> Result<(), &'static str> { settings.validate()?; self.settings = settings; @@ -94,11 +216,22 @@ impl Engine { self.gain = self.target_gain; self.fresh = false; } + if self.sent != Some(settings) { + self.link.settings.force_push(settings); + self.sent = Some(settings); + } Ok(()) } pub fn reset(&mut self) { self.gain = self.target_gain; self.levels = Levels::default(); + self.current.reset(); + if self.phase != Phase::Steady { + std::mem::swap(&mut self.current, &mut self.next); + self.current.reset(); + self.phase = Phase::Steady; + } + self.fill = 0; } /// One frame of main input `x` and sidechain input `side`. Mono uses index 0 only. #[inline] @@ -118,22 +251,66 @@ impl Engine { if s.abs() > 3e-5 { self.side_seen = true; } + self.feed(m, s); + if self.phase == Phase::Steady { + if let Some(fit) = self.link.fits.pop() { + self.next.set(fit, self.rate); + self.phase = Phase::Warm(self.warm); + } + } + let mut aligned = [0f32; 2]; + let mix = match self.phase { + Phase::Steady => None, + Phase::Warm(_) => Some(0.), + Phase::Fade(left) => Some(1. - left as f32 / self.fade as f32), + }; + for c in 0..n { + let a = self.current.process(c, x[c]); + aligned[c] = match mix { + None => a, + Some(t) => { + let b = self.next.process(c, x[c]); + a + t * (b - a) + } + }; + } + self.phase = match self.phase { + Phase::Steady => Phase::Steady, + Phase::Warm(1) => Phase::Fade(self.fade), + Phase::Warm(left) => Phase::Warm(left - 1), + Phase::Fade(1) => { + std::mem::swap(&mut self.current, &mut self.next); + Phase::Steady + } + Phase::Fade(left) => Phase::Fade(left - 1), + }; self.gain += (self.target_gain - self.gain) * self.gain_coef; if self.settings.bypass { return x; } let mut out = [0.; 2]; for c in 0..n { - let layer = x[c]; out[c] = self.gain * match self.settings.listen { - Listen::Layer => layer, - Listen::Sum => layer + side[c], + Listen::Layer => aligned[c], + Listen::Sum => aligned[c] + side[c], Listen::SumDry => x[c] + side[c], }; } out } + #[inline] + fn feed(&mut self, a: f32, b: f32) { + self.block.a[self.fill] = a; + self.block.b[self.fill] = b; + self.fill += 1; + if self.fill == BLOCK { + self.fill = 0; + if self.link.blocks.push(self.block).is_err() { + self.link.dropped.fetch_add(1, Ordering::Relaxed); + } + } + } /// Levels since the previous call. pub fn take_analyzer(&mut self) -> AnalyzerFrame { let n = self.levels.count.max(1) as f64; @@ -142,6 +319,7 @@ impl Engine { side_db: db((self.levels.side / n) as f32, true), side_peak_db: db(self.levels.peak, false), side_seen: self.side_seen, + dropped: self.link.dropped(), sample_rate: self.rate, }; self.levels = Levels::default(); diff --git a/plugins/weld/core/src/report.rs b/plugins/weld/core/src/report.rs new file mode 100644 index 0000000..b9a57cf --- /dev/null +++ b/plugins/weld/core/src/report.rs @@ -0,0 +1,108 @@ +//! What the analysis tells the editor. Plain `Copy` data, so it crosses threads through +//! a lock-free queue without allocating. +use crate::chain::Fit; + +/// Display bands, equally wide on a log axis from `DISPLAY_LOW` to `DISPLAY_HIGH`. +pub const BANDS: usize = 10; +/// Points of the phase curve. +pub const CURVE: usize = 96; +pub const DISPLAY_LOW: f32 = 30.; +pub const DISPLAY_HIGH: f32 = 12000.; + +/// Index of the display band that contains `f` (clamped to the outer bands). +pub fn band_of(f: f32) -> usize { + let t = (f / DISPLAY_LOW).ln() / (DISPLAY_HIGH / DISPLAY_LOW).ln(); + ((t * BANDS as f32).max(0.) as usize).min(BANDS - 1) +} +/// Lower edge of display band `i`; `band_edge(BANDS)` is the top edge. +pub fn band_edge(i: usize) -> f32 { + DISPLAY_LOW * (DISPLAY_HIGH / DISPLAY_LOW).powf(i as f32 / BANDS as f32) +} +/// Frequency of curve point `i`. +pub fn curve_frequency(i: usize) -> f32 { + DISPLAY_LOW * (DISPLAY_HIGH / DISPLAY_LOW).powf(i as f32 / (CURVE - 1) as f32) +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum Status { + /// Not enough measured audio yet. + #[default] + Waiting, + /// The sidechain carries no signal. + NoSidechain, + /// Both layers play, but one of them is under the gate. + Quiet, + /// The layers are not related closely enough to align (free-running oscillators, + /// different notes, noise). + Loose, + /// This layer is later than the sidechain layer. An allpass cannot move it earlier. + Late, + /// Measuring and aligning. + Aligned, +} + +#[derive(Debug, Clone, Copy, PartialEq, Default)] +pub struct Band { + /// Level of the unprocessed sum relative to a perfectly aligned sum, in dB (≤ 0). + pub before: f32, + /// Level of the sum with Weld's alignment, relative to a perfect one, in dB (≤ 0). + pub after: f32, + /// Power-weighted coherence of the two layers, 0..1. + pub coherence: f32, + /// False when the band carries no measured signal. + pub valid: bool, +} + +#[derive(Debug, Clone, Copy, PartialEq, Default)] +pub struct CurvePoint { + /// Phase of the main layer relative to the sidechain layer, degrees, ±180. + pub before: f32, + /// The same with Weld's alignment applied. + pub after: f32, + /// Phase of Weld's chain, degrees, ±180. + pub chain: f32, + pub coherence: f32, + pub valid: bool, +} + +#[derive(Debug, Clone, Copy, PartialEq)] +pub struct Report { + pub status: Status, + /// The alignment currently applied (or about to be). + pub fit: Fit, + /// True when this report comes with a newly accepted fit. + pub changed: bool, + pub bands: [Band; BANDS], + pub curve: [CurvePoint; CURVE], + /// What the cancellation costs between Low and High: aligned sum over unprocessed sum. + pub cost_db: f32, + /// What Weld wins back between Low and High: aligned-by-Weld sum over unprocessed sum. + pub recovered_db: f32, + /// Time offset of this layer against the sidechain layer, ms. Positive: this layer + /// is late. + pub lag_ms: f32, + /// How clear the time offset is, 0..1. + pub lag_strength: f32, + /// Measured frames in the current average. + pub frames: u32, + pub low: f32, + pub high: f32, +} +impl Default for Report { + fn default() -> Self { + Self { + status: Status::Waiting, + fit: Fit::NONE, + changed: false, + bands: [Band::default(); BANDS], + curve: [CurvePoint::default(); CURVE], + cost_db: 0., + recovered_db: 0., + lag_ms: 0., + lag_strength: 0., + frames: 0, + low: 60., + high: 6000., + } + } +} diff --git a/plugins/weld/core/src/settings.rs b/plugins/weld/core/src/settings.rs index c229c31..ac747a3 100644 --- a/plugins/weld/core/src/settings.rs +++ b/plugins/weld/core/src/settings.rs @@ -1,4 +1,4 @@ -//! Host-independent controls. Gains in dB. +//! Host-independent controls. Gains in dB, frequencies in Hz, times in seconds. use serde::{Deserialize, Serialize}; /// What the plugin outputs. @@ -13,9 +13,33 @@ pub enum Listen { SumDry, } +/// How the alignment is found. +#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, Default)] +pub enum Mode { + /// Keep measuring and refit when a clearly better alignment shows up. + #[default] + Follow, + /// Measure from the moment Learn starts, with equal weight; the result is applied + /// when Learn ends. + Learn, + /// Keep the current alignment. The display keeps measuring. + Hold, +} + #[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq)] #[serde(deny_unknown_fields)] pub struct Settings { + pub mode: Mode, + /// Averaging time of the measurement in Follow mode. + pub response: f32, + /// Lowest frequency the fit listens to. + pub low: f32, + /// Highest frequency the fit listens to. + pub high: f32, + /// Largest number of allpass sections the fit may use. + pub sections: usize, + /// Frames quieter than this (dBFS RMS) on either layer are not measured. + pub gate: f32, pub listen: Listen, pub output: f32, pub bypass: bool, @@ -23,6 +47,12 @@ pub struct Settings { impl Default for Settings { fn default() -> Self { Self { + mode: Mode::Follow, + response: 1.5, + low: 60., + high: 6000., + sections: 4, + gate: -50., listen: Listen::Layer, output: 0., bypass: false, @@ -31,10 +61,17 @@ impl Default for Settings { } impl Settings { pub fn validate(&self) -> Result<(), &'static str> { - if !self.output.is_finite() { + let values = [self.response, self.low, self.high, self.gate, self.output]; + if values.iter().any(|v| !v.is_finite()) { return Err("non-finite setting"); } - if (-12.0..=12.).contains(&self.output) { + if (0.3..=5.).contains(&self.response) + && (30.0..=400.).contains(&self.low) + && (1000.0..=12000.).contains(&self.high) + && (1..=crate::chain::MAX_SECTIONS).contains(&self.sections) + && (-70.0..=-30.).contains(&self.gate) + && (-12.0..=12.).contains(&self.output) + { Ok(()) } else { Err("setting out of range") diff --git a/plugins/weld/core/src/worker.rs b/plugins/weld/core/src/worker.rs new file mode 100644 index 0000000..c907c2b --- /dev/null +++ b/plugins/weld/core/src/worker.rs @@ -0,0 +1,77 @@ +//! The worker thread: takes blocks from the audio thread, runs the analyzer and sends +//! accepted fits back. Started and stopped outside the audio thread. +use crate::analyzer::Analyzer; +use crate::chain::Fit; +use crate::report::Report; +use crate::Link; +use crossbeam_queue::ArrayQueue; +use std::{ + sync::{ + atomic::{AtomicBool, Ordering}, + Arc, + }, + thread::JoinHandle, + time::Duration, +}; + +pub struct Worker { + stop: Arc, + thread: Option>, +} +impl Worker { + /// Starts the worker for one engine. `on_fit` runs on the worker thread for every + /// accepted fit, for example to store it in the plugin state. + pub fn spawn( + link: Arc, + rate: f32, + fit: Fit, + reports: Arc>, + mut on_fit: impl FnMut(Fit) + Send + 'static, + ) -> std::io::Result { + let stop = Arc::new(AtomicBool::new(false)); + let flag = stop.clone(); + let thread = std::thread::Builder::new() + .name("weld-analysis".into()) + .spawn(move || { + let mut analyzer = Analyzer::new(rate, fit); + while !flag.load(Ordering::Relaxed) { + let mut idle = true; + while let Some(settings) = link.settings.pop() { + analyzer.set_settings(settings); + } + while let Some(block) = link.blocks.pop() { + idle = false; + analyzer.push(&block.a, &block.b); + if let Some(update) = analyzer.poll() { + if let Some(fit) = update.fit { + link.fits.force_push(fit); + on_fit(fit); + } + reports.force_push(update.report); + } + if flag.load(Ordering::Relaxed) { + break; + } + } + if idle { + std::thread::sleep(Duration::from_millis(5)); + } + } + })?; + Ok(Self { + stop, + thread: Some(thread), + }) + } + pub fn stop(&mut self) { + self.stop.store(true, Ordering::Relaxed); + if let Some(thread) = self.thread.take() { + let _ = thread.join(); + } + } +} +impl Drop for Worker { + fn drop(&mut self) { + self.stop(); + } +} diff --git a/plugins/weld/core/tests/dsp.rs b/plugins/weld/core/tests/dsp.rs index e71d936..78e379e 100644 --- a/plugins/weld/core/tests/dsp.rs +++ b/plugins/weld/core/tests/dsp.rs @@ -95,3 +95,244 @@ fn invalid_settings_and_rates_are_rejected() { Settings::default() ); } + +use weld_core::{Analyzer, Chain, Fit, Mode, Section, Status}; + +fn random_fit(seed: u64) -> Fit { + let mut s = seed; + let mut rand = move || { + s = s + .wrapping_mul(6364136223846793005) + .wrapping_add(1442695040888963407); + (s >> 11) as f32 / (1u64 << 53) as f32 + }; + let mut fit = Fit { + flip: rand() > 0.5, + count: 1 + (rand() * 3.99) as usize, + ..Fit::NONE + }; + for section in &mut fit.sections { + *section = Section { + fc: 30. * 600f32.powf(rand()), + q: 0.2 * 20f32.powf(rand()), + }; + } + fit +} + +#[test] +fn every_chain_is_flat_in_magnitude() { + for seed in 1..40 { + let fit = random_fit(seed); + for i in 0..200 { + let f = 20. * 1000f32.powf(i as f32 / 199.); + let (re, im) = fit.response(f, 48000.); + let db = 10. * (re * re + im * im).log10(); + assert!(db.abs() < 0.01, "{fit:?} at {f} Hz: {db}"); + } + } +} + +#[test] +fn the_running_chain_matches_its_response() { + let fit = random_fit(3); + for f in [50f32, 333., 2100., 9000.] { + let mut chain = Chain::new(fit, 48000.); + let (mut i_sum, mut q_sum, mut energy) = (0f64, 0f64, 0f64); + let n = 96000; + for k in 0..n { + let t = k as f64 / 48000.; + let w = std::f64::consts::TAU * f as f64 * t; + let y = chain.process(0, w.cos() as f32) as f64; + if k > n / 2 { + i_sum += y * w.cos(); + q_sum += -y * w.sin(); + energy += y * y; + } + } + let m = (n / 2 - 1) as f64; + let (re, im) = fit.response(f, 48000.); + let measured = (2. * i_sum / m, 2. * q_sum / m); + assert!( + (measured.0 - re as f64).abs() < 0.01, + "{f}: {measured:?} vs {re},{im}" + ); + assert!( + (measured.1 - im as f64).abs() < 0.01, + "{f}: {measured:?} vs {re},{im}" + ); + assert!((2. * energy / m - 1.).abs() < 0.01); + } +} + +#[test] +fn the_impulse_response_starts_at_sample_zero() { + let mut chain = Chain::new(random_fit(5), 48000.); + let first = chain.process(0, 1.); + assert!(first.abs() > 0.01, "{first}"); +} + +#[test] +fn a_new_fit_crossfades_without_clicks() { + let mut engine = Engine::new(48000., 2).unwrap(); + engine.set_settings(Settings::default()).unwrap(); + let link = engine.link(); + let x = |n: usize| signal(n, 97.) + 0.3 * signal(n, 1210.); + let max_input_step = (1..48000) + .map(|n| (x(n) - x(n - 1)).abs()) + .fold(0f32, f32::max); + let mut previous = [0f32; 2]; + let mut largest = 0f32; + for n in 0..48000 { + if n == 4800 { + link.push_fit(Fit { + flip: true, + count: 2, + sections: [ + Section { fc: 300., q: 0.7 }, + Section { fc: 2000., q: 2. }, + Section::default(), + Section::default(), + ], + }); + } + let out = engine.process_sample([x(n), x(n)], [0.; 2]); + if n > 0 { + largest = largest.max((out[0] - previous[0]).abs()); + } + previous = out; + } + assert!( + largest <= 1.5 * max_input_step, + "{largest} vs {max_input_step}" + ); + assert!(engine.fit().flip); + assert_eq!(engine.fit().count, 2); +} + +#[test] +fn sum_uses_the_aligned_layer_and_sum_dry_the_unprocessed_one() { + for listen in [Listen::Sum, Listen::SumDry] { + let mut engine = Engine::new(48000., 1).unwrap(); + engine + .set_settings(Settings { + listen, + ..Settings::default() + }) + .unwrap(); + engine.set_fit(Fit { + flip: true, + ..Fit::NONE + }); + let out = engine.process_sample([0.5, 0.], [0.2, 0.]); + let expected = if listen == Listen::Sum { -0.3 } else { 0.7 }; + assert!((out[0] - expected).abs() < 1e-6, "{listen:?} {out:?}"); + } +} + +#[test] +fn blocks_reach_the_worker_and_drops_are_counted() { + let mut engine = Engine::new(48000., 2).unwrap(); + engine.set_settings(Settings::default()).unwrap(); + let link = engine.link(); + assert!(link.pop_settings().is_some()); + for n in 0..256 * 70 { + engine.process_sample([signal(n, 100.); 2], [signal(n, 100.); 2]); + } + assert_eq!(link.dropped(), 6); + let block = link.pop_block().unwrap(); + assert!((block.a[10] - signal(10, 100.)).abs() < 1e-6); + assert_eq!(block.a, block.b); +} + +#[test] +fn silence_holds_the_fit_and_produces_no_nan() { + let flipped = Fit { + flip: true, + ..Fit::NONE + }; + let mut analyzer = Analyzer::new(48000., flipped); + analyzer.set_settings(Settings::default()); + let zeros = [0f32; 256]; + let mut last = None; + for _ in 0..400 { + analyzer.push(&zeros, &zeros); + if let Some(update) = analyzer.poll() { + assert!(update.fit.is_none()); + last = Some(update.report); + } + } + let report = last.unwrap(); + assert_eq!(report.status, Status::NoSidechain); + assert_eq!(report.fit, flipped); + assert!(report.cost_db.is_finite() && report.recovered_db.is_finite()); + assert!(report + .bands + .iter() + .all(|b| b.before.is_finite() && b.after.is_finite())); +} + +#[test] +fn follow_mode_finds_the_flip_and_then_holds_still() { + let mut analyzer = Analyzer::new(48000., Fit::NONE); + analyzer.set_settings(Settings { + mode: Mode::Follow, + ..Settings::default() + }); + let a: Vec = (0..48000 * 6) + .map(|n| signal(n, 87.3) + 0.3 * signal(n, 261.9) + 0.1 * signal(n, 611.)) + .collect(); + let b: Vec = a.iter().map(|v| -0.7 * v).collect(); + let mut accepted = Vec::new(); + let mut last = None; + for (x, y) in a.chunks(256).zip(b.chunks(256)) { + analyzer.push(x, y); + if let Some(update) = analyzer.poll() { + if let Some(fit) = update.fit { + accepted.push(fit); + } + last = Some(update.report); + } + } + assert_eq!(accepted.len(), 1, "{accepted:?}"); + assert!(accepted[0].flip); + let report = last.unwrap(); + assert_eq!(report.status, Status::Aligned); + assert!(report.recovered_db > 10., "{}", report.recovered_db); + assert!((report.recovered_db - report.cost_db).abs() < 0.2); +} + +#[test] +fn learn_measures_and_applies_on_exit() { + let mut analyzer = Analyzer::new(48000., Fit::NONE); + let learn = Settings { + mode: Mode::Learn, + ..Settings::default() + }; + analyzer.set_settings(learn); + let a: Vec = (0..48000 * 2) + .map(|n| signal(n, 87.3) + 0.3 * signal(n, 261.9)) + .collect(); + let b: Vec = a.iter().map(|v| -v).collect(); + for (x, y) in a.chunks(256).zip(b.chunks(256)) { + analyzer.push(x, y); + let _ = analyzer.poll(); + } + analyzer.set_settings(Settings { + mode: Mode::Hold, + ..learn + }); + let update = analyzer.poll().expect("leaving Learn reports at once"); + assert!(analyzer.current().flip); + // Learn may already have applied the flip while measuring. + assert!(update.fit.is_none_or(|f| f.flip)); + // Hold never refits, even when the layers change. + let c: Vec = a.clone(); + for (x, y) in a.chunks(256).zip(c.chunks(256)) { + analyzer.push(x, y); + if let Some(update) = analyzer.poll() { + assert!(update.fit.is_none()); + } + } + assert!(analyzer.current().flip); +} diff --git a/plugins/weld/core/tests/fit.rs b/plugins/weld/core/tests/fit.rs new file mode 100644 index 0000000..be9bb4a --- /dev/null +++ b/plugins/weld/core/tests/fit.rs @@ -0,0 +1,294 @@ +//! Fit quality on a synthetic saw bass, with targets taken from the prototype. +use rustfft::{num_complex::Complex64, FftPlanner}; +use weld_core::{Analyzer, Chain, Fit, Mode, Settings, Status}; + +const RATE: f32 = 48000.; + +/// Two seconds of an F1 saw (60 harmonics, fixed random phases) plus a little noise. +fn bass() -> Vec { + let n = 2 * RATE as usize; + let mut seed = 1u64; + let mut rand = move || { + seed = seed + .wrapping_mul(6364136223846793005) + .wrapping_add(1442695040888963407); + (seed >> 11) as f64 / (1u64 << 53) as f64 + }; + let phases: Vec = (0..60).map(|_| rand() * std::f64::consts::TAU).collect(); + (0..n) + .map(|i| { + let t = i as f64 / RATE as f64; + let saw: f64 = (1..60) + .map(|k| (std::f64::consts::TAU * k as f64 * 87.3 * t + phases[k]).sin() / k as f64) + .sum(); + (0.3 * saw + 0.015 * (rand() - 0.5)) as f32 + }) + .collect() +} +fn noise(n: usize, seed: u64) -> Vec { + let mut s = seed; + (0..n) + .map(|_| { + s = s + .wrapping_mul(6364136223846793005) + .wrapping_add(1442695040888963407); + ((s >> 11) as f64 / (1u64 << 53) as f64 - 0.5) as f32 * 0.5 + }) + .collect() +} +fn delay(x: &[f32], samples: isize) -> Vec { + (0..x.len() as isize) + .map(|i| { + let j = i - samples; + if j >= 0 && (j as usize) < x.len() { + x[j as usize] + } else { + 0. + } + }) + .collect() +} +/// Direct-form biquad, b and a normalised by a0. +fn biquad(x: &[f32], b: [f64; 3], a: [f64; 3]) -> Vec { + let (mut x1, mut x2, mut y1, mut y2) = (0f64, 0f64, 0f64, 0f64); + x.iter() + .map(|&v| { + let v = v as f64; + let y = (b[0] * v + b[1] * x1 + b[2] * x2 - a[1] * y1 - a[2] * y2) / a[0]; + (x2, x1, y2, y1) = (x1, v, y1, y); + y as f32 + }) + .collect() +} +fn allpass(x: &[f32], fc: f64, q: f64) -> Vec { + let w = std::f64::consts::TAU * fc / RATE as f64; + let alpha = w.sin() / (2. * q); + let a = [1. + alpha, -2. * w.cos(), 1. - alpha]; + biquad(x, [a[2], a[1], a[0]], a) +} +fn highpass(x: &[f32], fc: f64) -> Vec { + let w = std::f64::consts::TAU * fc / RATE as f64; + let alpha = w.sin() / (2. * std::f64::consts::FRAC_1_SQRT_2); + let c = w.cos(); + biquad( + x, + [(1. + c) / 2., -(1. + c), (1. + c) / 2.], + [1. + alpha, -2. * c, 1. - alpha], + ) +} +fn power_db(x: &[f32]) -> f64 { + let skip = x.len() / 4; + let p: f64 = + x[skip..].iter().map(|&v| (v as f64).powi(2)).sum::() / (x.len() - skip) as f64; + 10. * p.max(1e-30).log10() +} +fn sum(a: &[f32], b: &[f32]) -> Vec { + a.iter().zip(b).map(|(a, b)| a + b).collect() +} +fn apply(fit: Fit, x: &[f32]) -> Vec { + let mut chain = Chain::new(fit, RATE); + x.iter().map(|&v| chain.process(0, v)).collect() +} +/// Level of A + B when every bin of A is rotated onto B: the best any phase-only +/// processing could reach. +fn ideal_db(a: &[f32], b: &[f32]) -> f64 { + let n = a.len(); + let mut planner = FftPlanner::::new(); + let fft = planner.plan_fft_forward(n); + let ifft = planner.plan_fft_inverse(n); + let mut fa: Vec = a.iter().map(|&v| Complex64::new(v as f64, 0.)).collect(); + let mut fb: Vec = b.iter().map(|&v| Complex64::new(v as f64, 0.)).collect(); + fft.process(&mut fa); + fft.process(&mut fb); + let mut y: Vec = fa + .iter() + .zip(&fb) + .map(|(a, b)| { + let unit = if b.norm() > 1e-12 { + b / b.norm() + } else { + Complex64::new(1., 0.) + }; + unit * a.norm() + b + }) + .collect(); + ifft.process(&mut y); + let out: Vec = y.iter().map(|v| (v.re / n as f64) as f32).collect(); + power_db(&out) +} +/// Power of A+B in octave bands from 60 Hz to 7.7 kHz, after minus before, in dB. +fn octave_changes(a: &[f32], aligned: &[f32], b: &[f32]) -> Vec { + let n = a.len(); + let mut planner = FftPlanner::::new(); + let fft = planner.plan_fft_forward(n); + let spectrum = |x: Vec| { + let mut v: Vec = x.iter().map(|&s| Complex64::new(s as f64, 0.)).collect(); + fft.process(&mut v); + v + }; + let before = spectrum(sum(a, b)); + let after = spectrum(sum(aligned, b)); + (0..7) + .map(|o| { + let lo = 60. * 2f64.powi(o); + let range = |s: &Vec| -> f64 { + (0..n / 2) + .filter(|&k| { + let f = k as f64 * RATE as f64 / n as f64; + f >= lo && f < 2. * lo + }) + .map(|k| s[k].norm_sqr()) + .sum() + }; + 10. * (range(&after) / range(&before)).log10() + }) + .collect() +} + +struct Outcome { + accepted: Option, + status: Status, + lag_ms: f32, + before: f64, + after: f64, + ideal: f64, + octaves: Vec, +} +fn run(a: &[f32], b: &[f32]) -> Outcome { + let mut analyzer = Analyzer::new(RATE, Fit::NONE); + analyzer.set_settings(Settings { + mode: Mode::Learn, + low: 40., + high: 8000., + ..Settings::default() + }); + for (x, y) in a.chunks(256).zip(b.chunks(256)) { + analyzer.push(x, y); + } + let update = analyzer.analyse(Some(0.1)); + let fit = *analyzer.current(); + let aligned = apply(fit, a); + Outcome { + accepted: update.fit, + status: update.report.status, + lag_ms: update.report.lag_ms, + before: power_db(&sum(a, b)), + after: power_db(&sum(&aligned, b)), + ideal: ideal_db(a, b), + octaves: octave_changes(a, &aligned, b), + } +} +fn show(name: &str, o: &Outcome) { + eprintln!( + "{name}: before {:.2} after {:.2} ideal {:.2} fit {:?} status {:?} lag {:.3} octaves {:?}", + o.before, + o.after, + o.ideal, + o.accepted.map(|f| (f.flip, f.active().to_vec())), + o.status, + o.lag_ms, + o.octaves + .iter() + .map(|v| (v * 10.).round() / 10.) + .collect::>() + ); +} + +#[test] +fn polarity_flip_needs_no_sections() { + let a = bass(); + let b: Vec = a.iter().map(|v| -v).collect(); + let o = run(&a, &b); + show("flip", &o); + let fit = o.accepted.expect("a fit"); + assert!(fit.flip); + assert_eq!(fit.count, 0); + assert!((o.after - o.ideal).abs() < 0.05); +} + +#[test] +fn an_allpass_is_found_again() { + let a = bass(); + let b = allpass(&a, 400., 0.7); + let o = run(&a, &b); + show("allpass400", &o); + let fit = o.accepted.expect("a fit"); + assert!(!fit.flip); + assert_eq!(fit.count, 1, "{fit:?}"); + let s = fit.sections[0]; + assert!((s.fc / 400. - 1.).abs() < 0.05, "{s:?}"); + assert!((s.q / 0.7 - 1.).abs() < 0.1, "{s:?}"); + assert!(o.ideal - o.after < 0.1); +} + +#[test] +fn a_late_sidechain_layer_is_caught_up() { + let a = bass(); + for ms in [0.3, 1.0] { + let b = delay(&a, (ms * 48.) as isize); + let o = run(&a, &b); + show(&format!("delay {ms}"), &o); + assert!(o.ideal - o.after < 0.3, "{ms}"); + assert!(o.after > o.before); + assert_ne!(o.status, Status::Late); + // A delay leaves a comb in the sum: notches at odd multiples of 1/(2·delay). + // A few allpass sections cannot follow a delay's phase all the way up, so they + // move the notches rather than remove them; one octave may lose some level + // while its neighbours gain more. + let mean = o.octaves.iter().sum::() / o.octaves.len() as f64; + assert!(mean > 0.4, "{ms} ms: mean octave change {mean}"); + for (i, v) in o.octaves.iter().enumerate() { + assert!(*v > -2., "{ms} ms, octave {i}: {v}"); + } + } +} + +#[test] +fn an_inverted_highpassed_layer_is_aligned() { + let a = bass(); + let b: Vec = highpass(&a, 300.).iter().map(|v| -v).collect(); + let o = run(&a, &b); + show("inverted hp300", &o); + assert!(o.ideal - o.after < 0.3); +} + +#[test] +fn unrelated_layers_are_left_alone() { + let a = bass(); + let b = noise(a.len(), 7); + let o = run(&a, &b); + show("unrelated", &o); + assert!(o.accepted.is_none()); + assert_eq!(o.status, Status::Loose); +} + +#[test] +fn a_late_main_layer_is_reported_and_not_made_worse() { + let a = bass(); + let b = delay(&a, -14); + let o = run(&a, &b); + show("ahead", &o); + assert_eq!(o.status, Status::Late); + assert!((o.lag_ms - 14. / 48.).abs() < 0.03, "{}", o.lag_ms); + assert!(o.after > o.before - 0.05); +} + +#[test] +fn one_fit_is_fast() { + let a = bass(); + let b = delay(&a, 48); + let mut analyzer = Analyzer::new(RATE, Fit::NONE); + analyzer.set_settings(Settings { + mode: Mode::Learn, + ..Settings::default() + }); + for (x, y) in a.chunks(256).zip(b.chunks(256)) { + analyzer.push(x, y); + } + let start = std::time::Instant::now(); + analyzer.analyse(Some(0.)); + let elapsed = start.elapsed(); + eprintln!("fit took {elapsed:?}"); + let limit = if cfg!(debug_assertions) { 1000 } else { 50 }; + assert!(elapsed.as_millis() < limit, "{elapsed:?}"); +} diff --git a/plugins/weld/core/tests/realtime.rs b/plugins/weld/core/tests/realtime.rs new file mode 100644 index 0000000..759422f --- /dev/null +++ b/plugins/weld/core/tests/realtime.rs @@ -0,0 +1,70 @@ +use std::{ + alloc::{GlobalAlloc, Layout, System}, + cell::Cell, +}; +use weld_core::{Engine, Fit, Listen, Section, Settings}; + +struct CheckedAllocator; +thread_local! { + static CHECK:Cell=const {Cell::new(false)}; + static OPERATIONS:Cell=const {Cell::new(0)}; +} +fn record() { + if CHECK.try_with(Cell::get).unwrap_or(false) { + let _ = OPERATIONS.try_with(|n| n.set(n.get() + 1)); + } +} +unsafe impl GlobalAlloc for CheckedAllocator { + unsafe fn alloc(&self, layout: Layout) -> *mut u8 { + record(); + unsafe { System.alloc(layout) } + } + unsafe fn dealloc(&self, pointer: *mut u8, layout: Layout) { + record(); + unsafe { System.dealloc(pointer, layout) } + } + unsafe fn realloc(&self, pointer: *mut u8, layout: Layout, size: usize) -> *mut u8 { + record(); + unsafe { System.realloc(pointer, layout, size) } + } +} +#[global_allocator] +static ALLOCATOR: CheckedAllocator = CheckedAllocator; + +#[test] +fn processing_fits_crossfades_and_a_full_queue_do_not_allocate_or_free() { + let mut engine = Engine::new(48000., 2).unwrap(); + let mut settings = Settings::default(); + engine.set_settings(settings).unwrap(); + let link = engine.link(); + let fits: Vec = (0..4) + .map(|k| Fit { + flip: k % 2 == 1, + count: k, + sections: [Section { + fc: 200. * (k + 1) as f32, + q: 0.7, + }; 4], + }) + .collect(); + CHECK.with(|v| v.set(true)); + // Nothing takes blocks, so the queue fills up and later blocks are dropped. + for i in 0..96000 { + if i % 4800 == 0 { + settings.output = ((i / 4800) % 5) as f32 - 2.; + settings.listen = [Listen::Layer, Listen::Sum, Listen::SumDry][(i / 4800) % 3]; + engine.set_settings(settings).unwrap(); + link.push_fit(fits[(i / 4800) % 4]); + } + let t = i as f32 / 48000.; + let x = (std::f32::consts::TAU * 55. * t).sin() * 0.5; + std::hint::black_box(engine.process_sample([x, -x], [x * 0.5, x])); + if i % 1600 == 0 { + std::hint::black_box(engine.take_analyzer()); + } + } + engine.reset(); + CHECK.with(|v| v.set(false)); + assert_eq!(OPERATIONS.with(Cell::get), 0); + assert!(link.dropped() > 0); +} diff --git a/plugins/weld/plugin/Cargo.lock b/plugins/weld/plugin/Cargo.lock index 6f45f9c..d9d67e3 100644 --- a/plugins/weld/plugin/Cargo.lock +++ b/plugins/weld/plugin/Cargo.lock @@ -1074,12 +1074,39 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2bf50223579dc7cdcfb3bfcacf7069ff68243f8c363f62ffa99cf000a6b9c451" +[[package]] +name = "num-complex" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" +dependencies = [ + "num-traits", +] + [[package]] name = "num-conv" version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441" +[[package]] +name = "num-integer" +version = "0.1.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b" +dependencies = [ + "num-traits", +] + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + [[package]] name = "num_enum" version = "0.7.6" @@ -1454,6 +1481,15 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4a6394b9e965e73d0a289ee54f589087e2c676aedf60885baf52c76b771e4958" +[[package]] +name = "primal-check" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc0d895b311e3af9902528fbb8f928688abbd95872819320517cc24ca6b2bd08" +dependencies = [ + "num-integer", +] + [[package]] name = "proc-macro-crate" version = "3.5.0" @@ -1547,6 +1583,20 @@ dependencies = [ "semver", ] +[[package]] +name = "rustfft" +version = "6.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "21db5f9893e91f41798c88680037dba611ca6674703c1a18601b01a72c8adb89" +dependencies = [ + "num-complex", + "num-integer", + "num-traits", + "primal-check", + "strength_reduce", + "transpose", +] + [[package]] name = "rustix" version = "0.38.44" @@ -1695,6 +1745,12 @@ dependencies = [ "serde", ] +[[package]] +name = "strength_reduce" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fe895eb47f22e2ddd4dabc02bce419d2e643c8e3b585c78158b349195bc24d82" + [[package]] name = "syn" version = "1.0.109" @@ -1864,6 +1920,16 @@ version = "0.1.36" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +[[package]] +name = "transpose" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ad61aed86bc3faea4300c7aee358b4c6d0c8d6ccc36524c96e4c92ccf26e77e" +dependencies = [ + "num-integer", + "strength_reduce", +] + [[package]] name = "ttf-parser" version = "0.25.1" @@ -2005,6 +2071,8 @@ dependencies = [ name = "weld-core" version = "0.1.0" dependencies = [ + "crossbeam-queue", + "rustfft", "serde", ] diff --git a/plugins/weld/plugin/src/lib.rs b/plugins/weld/plugin/src/lib.rs index 073194d..ea8e886 100644 --- a/plugins/weld/plugin/src/lib.rs +++ b/plugins/weld/plugin/src/lib.rs @@ -1,20 +1,23 @@ -//! CLAP audio effect with a sidechain input. All DSP state is owned by the processing thread. +//! CLAP audio effect with a sidechain input. The audio thread owns the engine; a worker +//! thread, started in `initialize`, measures and fits. mod params; use crossbeam_queue::ArrayQueue; use nih_plug::prelude::*; use nih_plug_egui::{create_egui_editor, egui}; -use params::WeldParams; +use params::{WeldParams, GESTURES}; use std::sync::Arc; -use weld_core::{AnalyzerFrame, Engine}; +use weld_core::{AnalyzerFrame, Engine, Report, Worker}; struct EditorData { ui: weld_ui::State, - gestures: [bool; 1], + gestures: [bool; GESTURES], } pub struct Weld { params: Arc, analyzer: Arc>, + reports: Arc>, engine: Option, + worker: Option, publish_count: usize, publish_interval: usize, } @@ -23,7 +26,9 @@ impl Default for Weld { Self { params: Arc::new(WeldParams::default()), analyzer: Arc::new(ArrayQueue::new(2)), + reports: Arc::new(ArrayQueue::new(2)), engine: None, + worker: None, publish_count: 0, publish_interval: 1600, } @@ -58,17 +63,21 @@ impl Plugin for Weld { fn editor(&mut self, _: AsyncExecutor) -> Option> { let params = self.params.clone(); let analyzer = self.analyzer.clone(); + let reports = self.reports.clone(); create_egui_editor( params.editor.clone(), EditorData { ui: weld_ui::State::default(), - gestures: [false; 1], + gestures: [false; GESTURES], }, |ctx, _| weld_ui::style(ctx), move |ctx, setter, data| { while let Some(frame) = analyzer.pop() { data.ui.accept_frame(frame); } + while let Some(report) = reports.pop() { + data.ui.accept_report(report); + } let before = params.settings(); let mut edited = before; nih_plug_egui::resizable_window::ResizableWindow::new("weld-editor") @@ -106,11 +115,37 @@ impl Plugin for Weld { if engine.set_settings(self.params.settings()).is_err() { return false; } + // A restored project starts with its saved alignment, without a crossfade. + let fit = self.params.stored_fit().sanitized(config.sample_rate); + engine.set_fit(fit); + if let Some(mut old) = self.worker.take() { + old.stop(); + } + let store = self.params.fit.clone(); + let Ok(worker) = Worker::spawn( + engine.link(), + config.sample_rate, + fit, + self.reports.clone(), + move |fit| { + if let Ok(mut stored) = store.write() { + *stored = fit; + } + }, + ) else { + return false; + }; + self.worker = Some(worker); context.set_latency_samples(engine.latency_samples()); self.publish_interval = (config.sample_rate / 30.) as usize; self.engine = Some(engine); true } + fn deactivate(&mut self) { + if let Some(mut worker) = self.worker.take() { + worker.stop(); + } + } fn reset(&mut self) { if let Some(e) = self.engine.as_mut() { e.reset(); diff --git a/plugins/weld/plugin/src/params.rs b/plugins/weld/plugin/src/params.rs index 899c3d1..ed83a02 100644 --- a/plugins/weld/plugin/src/params.rs +++ b/plugins/weld/plugin/src/params.rs @@ -1,8 +1,17 @@ use nih_plug::prelude::*; use nih_plug_egui::EguiState; -use std::sync::Arc; -use weld_core::{Listen, Settings}; +use std::sync::{Arc, RwLock}; +use weld_core::{Fit, Listen, Mode, Settings, MAX_SECTIONS}; +#[derive(Enum, Debug, PartialEq, Eq, Clone, Copy)] +pub enum ModeParam { + #[id = "follow"] + Follow, + #[id = "learn"] + Learn, + #[id = "hold"] + Hold, +} #[derive(Enum, Debug, PartialEq, Eq, Clone, Copy)] pub enum ListenParam { #[id = "layer"] @@ -14,12 +23,27 @@ pub enum ListenParam { SumDry, } /// Continuous controls, in the order of the gesture slots. -const FLOATS: usize = 1; +const FLOATS: usize = 5; #[derive(Params)] pub struct WeldParams { #[persist = "editor"] pub editor: Arc, + /// The applied alignment, so a project reopens with it. + #[persist = "weld-fit"] + pub fit: Arc>, + #[id = "mode"] + pub mode: EnumParam, + #[id = "response"] + pub response: FloatParam, + #[id = "low"] + pub low: FloatParam, + #[id = "high"] + pub high: FloatParam, + #[id = "sections"] + pub sections: IntParam, + #[id = "gate"] + pub gate: FloatParam, #[id = "listen"] pub listen: EnumParam, #[id = "output"] @@ -27,11 +51,68 @@ pub struct WeldParams { #[id = "bypass"] pub bypass: BoolParam, } +fn hz(value: f32) -> String { + if value >= 1000. { + format!("{:.2} kHz", value / 1000.) + } else { + format!("{value:.0} Hz") + } +} impl Default for WeldParams { fn default() -> Self { let d = Settings::default(); Self { editor: EguiState::from_size(weld_ui::WIDTH, weld_ui::HEIGHT), + fit: Arc::new(RwLock::new(Fit::NONE)), + mode: EnumParam::new("Mode", ModeParam::Follow), + response: FloatParam::new( + "Response", + d.response, + FloatRange::Skewed { + min: 0.3, + max: 5., + factor: FloatRange::skew_factor(-1.), + }, + ) + .with_unit(" s") + .with_value_to_string(Arc::new(|v| format!("{v:.2}"))), + low: FloatParam::new( + "Low", + d.low, + FloatRange::Skewed { + min: 30., + max: 400., + factor: FloatRange::skew_factor(-1.5), + }, + ) + .with_value_to_string(Arc::new(hz)), + high: FloatParam::new( + "High", + d.high, + FloatRange::Skewed { + min: 1000., + max: 12000., + factor: FloatRange::skew_factor(-1.), + }, + ) + .with_value_to_string(Arc::new(hz)), + sections: IntParam::new( + "Sections", + d.sections as i32, + IntRange::Linear { + min: 1, + max: MAX_SECTIONS as i32, + }, + ), + gate: FloatParam::new( + "Gate", + d.gate, + FloatRange::Linear { + min: -70., + max: -30., + }, + ) + .with_unit(" dB"), listen: EnumParam::new("Listen", ListenParam::Layer), output: FloatParam::new( "Output", @@ -46,9 +127,33 @@ impl Default for WeldParams { } } } +fn mode_param(mode: Mode) -> ModeParam { + match mode { + Mode::Follow => ModeParam::Follow, + Mode::Learn => ModeParam::Learn, + Mode::Hold => ModeParam::Hold, + } +} +fn listen_param(listen: Listen) -> ListenParam { + match listen { + Listen::Layer => ListenParam::Layer, + Listen::Sum => ListenParam::Sum, + Listen::SumDry => ListenParam::SumDry, + } +} impl WeldParams { pub fn settings(&self) -> Settings { Settings { + mode: match self.mode.value() { + ModeParam::Follow => Mode::Follow, + ModeParam::Learn => Mode::Learn, + ModeParam::Hold => Mode::Hold, + }, + response: self.response.value(), + low: self.low.value(), + high: self.high.value(), + sections: self.sections.value().clamp(1, MAX_SECTIONS as i32) as usize, + gate: self.gate.value(), listen: match self.listen.value() { ListenParam::Layer => Listen::Layer, ListenParam::Sum => Listen::Sum, @@ -58,8 +163,17 @@ impl WeldParams { bypass: self.bypass.value(), } } + pub fn stored_fit(&self) -> Fit { + self.fit.read().map(|f| *f).unwrap_or(Fit::NONE) + } fn floats(&self) -> [&FloatParam; FLOATS] { - [&self.output] + [ + &self.response, + &self.low, + &self.high, + &self.gate, + &self.output, + ] } /// Pushes every changed editor value to the host. Continuous controls hold one gesture /// while the pointer is down; switches are set in a single gesture. @@ -71,7 +185,13 @@ impl WeldParams { dragging: bool, open: &mut [bool; FLOATS], ) { - let values = [(old.output, new.output)]; + let values = [ + (old.response, new.response), + (old.low, new.low), + (old.high, new.high), + (old.gate, new.gate), + (old.output, new.output), + ]; for (i, (p, (a, b))) in self.floats().into_iter().zip(values).enumerate() { if a != b { if !open[i] { @@ -85,20 +205,23 @@ impl WeldParams { open[i] = false; } } + fn switch(setter: &ParamSetter, p: &P, v: P::Plain) { + setter.begin_set_parameter(p); + setter.set_parameter(p, v); + setter.end_set_parameter(p); + } if new.bypass != old.bypass { - setter.begin_set_parameter(&self.bypass); - setter.set_parameter(&self.bypass, new.bypass); - setter.end_set_parameter(&self.bypass); + switch(setter, &self.bypass, new.bypass); + } + if new.mode != old.mode { + switch(setter, &self.mode, mode_param(new.mode)); } if new.listen != old.listen { - let v = match new.listen { - Listen::Layer => ListenParam::Layer, - Listen::Sum => ListenParam::Sum, - Listen::SumDry => ListenParam::SumDry, - }; - setter.begin_set_parameter(&self.listen); - setter.set_parameter(&self.listen, v); - setter.end_set_parameter(&self.listen); + switch(setter, &self.listen, listen_param(new.listen)); + } + if new.sections != old.sections { + switch(setter, &self.sections, new.sections as i32); } } } +pub const GESTURES: usize = FLOATS; diff --git a/plugins/weld/plugin/tests/host.rs b/plugins/weld/plugin/tests/host.rs index 214d2ec..27f7dfe 100644 --- a/plugins/weld/plugin/tests/host.rs +++ b/plugins/weld/plugin/tests/host.rs @@ -111,6 +111,20 @@ fn sidechain_port_state_latency_and_audio_lifecycle() { envelope .params .insert("listen".into(), ParamValue::String("sum".into())); + envelope + .params + .insert("mode".into(), ParamValue::String("hold".into())); + envelope + .params + .insert("sections".into(), ParamValue::I32(2)); + // A saved alignment that flips polarity. + let flipped = weld_core::Fit { + flip: true, + ..weld_core::Fit::NONE + }; + envelope + .fields + .insert("weld-fit".into(), serde_json::to_string(&flipped).unwrap()); let encoded = serde_json::to_vec(&envelope).unwrap(); let mut framed = (encoded.len() as u64).to_le_bytes().to_vec(); framed.extend_from_slice(&encoded); @@ -126,13 +140,16 @@ fn sidechain_port_state_latency_and_audio_lifecycle() { serde_json::to_value(&restored.params).unwrap(), serde_json::to_value(&envelope.params).unwrap() ); - // Listen = Sum: a sidechain equal to the main input doubles the output (+6 dB). - // A silent sidechain leaves the main input alone. CLAP hosts pass every port, so - // the test always passes both (nih_plug reads one port past a short count). + let saved: weld_core::Fit = serde_json::from_str(&restored.fields["weld-fit"]).unwrap(); + assert_eq!(saved, flipped); + // Listen = Sum with the restored flip: a sidechain equal to the main input cancels + // it from the first sample on. A silent sidechain leaves the level alone. CLAP hosts + // pass every port, so the test always passes both (nih_plug reads one port past a + // short count). for (rate, with_side, expected_db) in [ - (44100., true, 6.02), - (48000., false, 0.), - (96000., true, 6.02), + (44100., true, None), + (48000., false, Some(0.)), + (96000., true, None), ] { assert!(p.activate.unwrap()(plugin, rate, 1, 512)); assert_eq!((*latency).get.unwrap()(plugin), 0); @@ -213,10 +230,10 @@ fn sidechain_port_state_latency_and_audio_lifecycle() { position += frames; } let ratio_db = 10. * (energy_out / energy_in).log10(); - assert!( - (ratio_db - expected_db).abs() < 0.2, - "{rate} {with_side} {ratio_db}" - ); + match expected_db { + Some(db) => assert!((ratio_db - db).abs() < 0.2, "{rate} {with_side} {ratio_db}"), + None => assert!(ratio_db < -60., "{rate} {with_side} {ratio_db}"), + } p.stop_processing.unwrap()(plugin); p.deactivate.unwrap()(plugin); } diff --git a/plugins/weld/ui/src/lib.rs b/plugins/weld/ui/src/lib.rs index 271c527..e943c5a 100644 --- a/plugins/weld/ui/src/lib.rs +++ b/plugins/weld/ui/src/lib.rs @@ -1,14 +1,23 @@ //! Weld editor, shared by the CLAP wrapper and the headless renderer. use common_ui::*; -use egui::{self, FontId, RichText}; -use weld_core::{AnalyzerFrame, Listen, Settings}; +use egui::{self, Color32, FontId, Pos2, Rect, RichText, Sense, Stroke, Vec2}; +use weld_core::{ + band_edge, curve_frequency, AnalyzerFrame, Listen, Mode, Report, Settings, Status, CURVE, + DISPLAY_HIGH, DISPLAY_LOW, MAX_SECTIONS, +}; pub use common_ui::style; pub const WIDTH: u32 = 1040; pub const HEIGHT: u32 = 650; +const GRID: Color32 = Color32::from_rgb(24, 26, 30); +const LOSS: Color32 = Color32::from_rgb(170, 40, 52); +const LOSS_FLOOR: f32 = -24.; +const TOP: f32 = 318.; + pub struct State { pub analyzer: AnalyzerFrame, + pub report: Report, /// True while a pointer button is held, so knob drags become one host gesture. pub gesture_active: bool, pub help: String, @@ -17,6 +26,7 @@ impl Default for State { fn default() -> Self { Self { analyzer: AnalyzerFrame::default(), + report: Report::default(), gesture_active: false, help: DEFAULT_HELP.into(), } @@ -33,12 +43,43 @@ impl State { self.analyzer.side_db = frame.side_db.max(old.side_db - 3.); self.analyzer.side_peak_db = frame.side_peak_db.max(old.side_peak_db - 3.); } + pub fn accept_report(&mut self, report: Report) { + self.report = report; + } } fn hint(r: &egui::Response, state: &mut State, text: &str) { if r.hovered() || r.has_focus() { state.help = text.into(); } } +#[allow(clippy::too_many_arguments)] +fn knob( + ui: &mut egui::Ui, + state: &mut State, + name: &str, + value: &mut f32, + range: std::ops::RangeInclusive, + default: f32, + unit: &str, + help: &str, +) { + ui.vertical_centered(|ui| { + let r = rotary(ui, name, value, range, default, unit, 58., ACCENT); + hint(&r, state, help); + }); +} +fn big(text: impl Into, size: f32, color: Color32) -> RichText { + RichText::new(text) + .font(FontId::new(size, egui::FontFamily::Name("medium".into()))) + .color(color) +} +fn hz(f: f32) -> String { + if f >= 1000. { + format!("{:.1} kHz", f / 1000.) + } else { + format!("{f:.0} Hz") + } +} pub fn show(ui: &mut egui::Ui, settings: &mut Settings, state: &mut State) { backdrop(ui); @@ -52,64 +93,446 @@ pub fn show(ui: &mut egui::Ui, settings: &mut Settings, state: &mut State) { if r.clicked() { settings.bypass = !settings.bypass; } - }); - ui.add_space(6.); - let frame = state.analyzer; - panel(ui, |ui| { - ui.set_width(ui.available_width()); - ui.set_min_height(380.); - ui.vertical(|ui| { - caption(ui, "SIDECHAIN"); - let (text, color) = if frame.side_db > -90. { - (format!("sidechain: {:.0} dBFS", frame.side_db), TEXT) - } else if frame.side_seen { - ("sidechain: silent".to_string(), MUTED) - } else { - ("no sidechain signal".to_string(), DIM) - }; - ui.label( - RichText::new(text) - .font(FontId::new(40., egui::FontFamily::Name("medium".into()))) - .color(color), - ); - ui.add_space(12.); - meter(ui, "MAIN", frame.main_db, STEEL, 520.); - meter(ui, "SIDECHAIN", frame.side_db, ACCENT, 520.); - meter(ui, "SIDE PEAK", frame.side_peak_db, FOCUS, 520.); - ui.add_space(18.); - ui.label( - RichText::new( - "FL Studio: on the other layer's mixer track, right-click the route arrow \ - to this track and choose \"Sidechain to this track\". Then open Weld's \ - wrapper settings and set the sidechain input to that track.", - ) - .size(13.) - .color(MUTED), - ); - }); - }); - ui.add_space(6.); - panel(ui, |ui| { - ui.set_width(ui.available_width()); - ui.horizontal(|ui| { - caption(ui, "LISTEN"); - for (listen, label, help) in [ - (Listen::Layer, "LAYER", "Output the main layer only. Use this in the mix."), - (Listen::Sum, "SUM", "Output main + sidechain layer. Mute the other layer's own send while listening."), - (Listen::SumDry, "SUM DRY", "Output the unprocessed main + sidechain layer, to compare with SUM."), - ] { - let r = ui.selectable_label(settings.listen == listen, RichText::new(label).size(13.)); - if r.clicked() { - settings.listen = listen; - } - hint(&r, state, help); + ui.separator(); + for (listen, label, help) in [ + (Listen::SumDry, "SUM DRY", "Output this layer unprocessed plus the sidechain layer, to compare with SUM."), + (Listen::Sum, "SUM", "Output this layer aligned plus the sidechain layer. Mute the other layer's own send while you listen."), + (Listen::Layer, "LAYER", "Output this layer only, aligned. Use this in the mix."), + ] { + let r = ui.selectable_label(settings.listen == listen, RichText::new(label).size(13.)); + if r.clicked() { + settings.listen = listen; } - ui.add_space(40.); - let d = Settings::default(); - let r = rotary(ui, "Output", &mut settings.output, -12. ..=12., d.output, " dB", 54., ACCENT); - hint(&r, state, "Output trim."); + hint(&r, state, help); + } + caption(ui, "LISTEN"); + }); + ui.add_space(6.); + ui.horizontal_top(|ui| { + panel(ui, |ui| { + ui.set_width(250.); + ui.set_min_height(TOP); + ui.vertical(|ui| summary(ui, settings, state)); }); + ui.add_space(4.); + panel(ui, |ui| { + ui.set_width(ui.available_width()); + ui.vertical(|ui| { + let report = state.report; + band_plot(ui, &report, 190.); + ui.add_space(4.); + phase_plot(ui, &report, TOP - 194.); + }); + }); + }); + ui.add_space(6.); + panel(ui, |ui| { + ui.set_width(ui.available_width()); + ui.vertical(|ui| controls(ui, settings, state)); }); ui.separator(); ui.label(RichText::new(&state.help).color(MUTED)); } + +fn status_text(report: &Report, frame: &AnalyzerFrame, mode: Mode) -> (String, Color32) { + if !frame.side_seen && frame.side_db < -90. { + return ( + "No sidechain signal. Route the other layer to Weld's sidechain input.".into(), + DIM, + ); + } + match report.status { + Status::NoSidechain => ("The sidechain is silent.".into(), MUTED), + Status::Waiting => ("Listening…".into(), MUTED), + Status::Quiet => ("One layer is under the gate.".into(), MUTED), + Status::Loose => ( + "Layers unrelated: free-running oscillators or different notes. Enable retrigger.".into(), + FOCUS, + ), + Status::Late => ( + format!( + "This layer is {:.2} ms late. Weld cannot move it earlier: put Weld on the other layer.", + report.lag_ms + ), + ACCENT, + ), + Status::Aligned => match mode { + Mode::Follow => ("Following.".into(), STEEL), + Mode::Learn => (format!("Learning… {} frames", report.frames), ACCENT), + Mode::Hold => ("Holding the alignment.".into(), STEEL), + }, + } +} + +fn summary(ui: &mut egui::Ui, settings: &Settings, state: &mut State) { + let report = state.report; + let frame = state.analyzer; + caption(ui, "RECOVERED"); + let measured = matches!( + report.status, + Status::Aligned | Status::Late | Status::Loose + ); + ui.label(big( + if measured { + format!("{:+.1} dB", report.recovered_db) + } else { + "—".into() + }, + 44., + if measured { TEXT } else { DIM }, + )); + ui.label( + RichText::new(if measured { + format!( + "Cancellation cost {:.1} dB, {}–{}", + report.cost_db, + hz(report.low), + hz(report.high) + ) + } else { + "Average per band, between Low and High".into() + }) + .size(12.) + .color(MUTED), + ); + ui.add_space(10.); + caption(ui, "ALIGNMENT"); + let fit = report.fit; + ui.label( + RichText::new(if fit.flip { + "Polarity flipped" + } else { + "Polarity normal" + }) + .size(14.) + .color(if fit.flip { ACCENT } else { STEEL }), + ); + if fit.count == 0 { + ui.label(RichText::new("No allpass sections").size(13.).color(MUTED)); + } + for s in fit.active() { + ui.label( + RichText::new(format!("{:>9} Q {:.2}", hz(s.fc), s.q)) + .monospace() + .size(13.) + .color(STEEL), + ); + } + ui.add_space(10.); + let (text, color) = status_text(&report, &frame, settings.mode); + ui.label(RichText::new(text).size(13.).color(color)); + ui.add_space(10.); + meter(ui, "MAIN", frame.main_db, STEEL, 150.); + meter(ui, "SIDECHAIN", frame.side_db, ACCENT, 150.); + if frame.dropped > 0 { + ui.label( + RichText::new(format!("{} blocks skipped (CPU busy)", frame.dropped)) + .size(11.) + .color(DIM), + ); + } +} + +fn freq_x(rect: Rect, f: f32) -> f32 { + let t = (f / DISPLAY_LOW).ln() / (DISPLAY_HIGH / DISPLAY_LOW).ln(); + rect.left() + rect.width() * t.clamp(0., 1.) +} +fn frequency_grid(p: &egui::Painter, plot: Rect, labels: bool) { + for f in [50f32, 100., 200., 500., 1000., 2000., 5000., 10000.] { + let x = freq_x(plot, f); + p.line_segment( + [Pos2::new(x, plot.top()), Pos2::new(x, plot.bottom())], + Stroke::new(1.0_f32, GRID), + ); + if labels { + p.text( + Pos2::new(x, plot.bottom() + 3.), + egui::Align2::CENTER_TOP, + if f >= 1000. { + format!("{}k", f / 1000.) + } else { + format!("{f}") + }, + FontId::proportional(10.), + DIM, + ); + } + } +} +fn fit_range(p: &egui::Painter, plot: Rect, report: &Report) { + let (a, b) = (freq_x(plot, report.low), freq_x(plot, report.high)); + p.rect_filled( + Rect::from_min_max(Pos2::new(a, plot.top()), Pos2::new(b, plot.bottom())), + 0., + Color32::from_rgba_unmultiplied(255, 255, 255, 5), + ); +} + +/// Per band: what the unprocessed sum loses against a perfect alignment (red), and the +/// part of it Weld wins back (accent). +fn band_plot(ui: &mut egui::Ui, report: &Report, height: f32) { + let (outer, _) = + ui.allocate_exact_size(Vec2::new(ui.available_width(), height), Sense::hover()); + let p = ui.painter_at(outer); + p.rect_filled(outer, 2., SCREEN); + let plot = Rect::from_min_max( + outer.left_top() + Vec2::new(34., 22.), + outer.right_bottom() - Vec2::new(10., 16.), + ); + p.text( + outer.left_top() + Vec2::new(8., 5.), + egui::Align2::LEFT_TOP, + "SUM LEVEL AGAINST A PERFECT ALIGNMENT", + FontId::proportional(10.), + MUTED, + ); + let legend = [(LOSS, "lost to cancellation"), (ACCENT, "won back by Weld")]; + let mut x = outer.right() - 10.; + for (color, label) in legend.iter().rev() { + let galley = p.layout_no_wrap(label.to_string(), FontId::proportional(10.), MUTED); + x -= galley.size().x; + p.galley(Pos2::new(x, outer.top() + 5.), galley, MUTED); + x -= 14.; + p.rect_filled( + Rect::from_min_size(Pos2::new(x, outer.top() + 7.), Vec2::splat(8.)), + 1., + *color, + ); + x -= 14.; + } + fit_range(&p, plot, report); + let y = |db: f32| plot.top() + plot.height() * (db / LOSS_FLOOR).clamp(0., 1.); + for db in [0., -6., -12., -18., -24.] { + let yy = y(db); + p.line_segment( + [Pos2::new(plot.left(), yy), Pos2::new(plot.right(), yy)], + Stroke::new(1.0_f32, if db == 0. { EDGE } else { GRID }), + ); + p.text( + Pos2::new(outer.left() + 6., yy), + egui::Align2::LEFT_CENTER, + if db == 0. { + "0".into() + } else { + format!("{db:.0}") + }, + FontId::proportional(10.), + if db == 0. { MUTED } else { DIM }, + ); + } + frequency_grid(&p, plot, true); + for (i, band) in report.bands.iter().enumerate() { + if !band.valid { + continue; + } + let (a, b) = ( + freq_x(plot, band_edge(i)) + 2., + freq_x(plot, band_edge(i + 1)) - 2., + ); + let top = y(0.); + let before = y(band.before); + let after = y(band.after.max(band.before)); + let loose = band.coherence < 0.3; + // The whole loss in red, then the recovered part over it in accent. + p.rect_filled( + Rect::from_min_max(Pos2::new(a, top), Pos2::new(b, before.max(top + 1.))), + 1., + if loose { + LOSS.gamma_multiply(0.35) + } else { + LOSS + }, + ); + if after > top + 0.5 { + p.rect_filled( + Rect::from_min_max(Pos2::new(a, after), Pos2::new(b, before)), + 0., + if loose { + ACCENT.gamma_multiply(0.35) + } else { + ACCENT + }, + ); + } else if before > top + 0.5 { + p.rect_filled( + Rect::from_min_max(Pos2::new(a, top), Pos2::new(b, before)), + 1., + if loose { + ACCENT.gamma_multiply(0.35) + } else { + ACCENT + }, + ); + } + p.line_segment( + [Pos2::new(a, after), Pos2::new(b, after)], + Stroke::new(1.5_f32, TEXT), + ); + if loose { + // Hatching marks bands where the layers do not hold a stable phase. + let mut x = a; + while x < b { + p.line_segment( + [ + Pos2::new(x, plot.bottom()), + Pos2::new((x + 8.).min(b), plot.bottom() - 8.), + ], + Stroke::new(1.0_f32, DIM), + ); + x += 5.; + } + p.text( + Pos2::new((a + b) / 2., plot.bottom() - 12.), + egui::Align2::CENTER_BOTTOM, + "loose", + FontId::proportional(10.), + DIM, + ); + } + } +} + +/// Phase of this layer against the sidechain layer, before and after, plus the chain's own +/// phase. +fn phase_plot(ui: &mut egui::Ui, report: &Report, height: f32) { + let (outer, _) = + ui.allocate_exact_size(Vec2::new(ui.available_width(), height), Sense::hover()); + let p = ui.painter_at(outer); + p.rect_filled(outer, 2., SCREEN); + let plot = Rect::from_min_max( + outer.left_top() + Vec2::new(34., 20.), + outer.right_bottom() - Vec2::new(10., 6.), + ); + p.text( + outer.left_top() + Vec2::new(8., 4.), + egui::Align2::LEFT_TOP, + "PHASE AGAINST THE SIDECHAIN LAYER", + FontId::proportional(10.), + MUTED, + ); + let legend = [(DIM, "before"), (ACCENT, "after"), (FOCUS, "Weld's chain")]; + let mut x = outer.right() - 10.; + for (color, label) in legend.iter().rev() { + let galley = p.layout_no_wrap(label.to_string(), FontId::proportional(10.), MUTED); + x -= galley.size().x; + p.galley(Pos2::new(x, outer.top() + 4.), galley, MUTED); + x -= 16.; + p.line_segment( + [ + Pos2::new(x, outer.top() + 10.), + Pos2::new(x + 10., outer.top() + 10.), + ], + Stroke::new(2.0_f32, *color), + ); + x -= 12.; + } + fit_range(&p, plot, report); + let y = |deg: f32| plot.center().y - plot.height() / 2. * (deg / 180.).clamp(-1., 1.); + for deg in [-180f32, -90., 0., 90., 180.] { + let yy = y(deg); + p.line_segment( + [Pos2::new(plot.left(), yy), Pos2::new(plot.right(), yy)], + Stroke::new(1.0_f32, if deg == 0. { EDGE } else { GRID }), + ); + if deg.abs() < 180. { + p.text( + Pos2::new(outer.left() + 6., yy), + egui::Align2::LEFT_CENTER, + format!("{deg:.0}°"), + FontId::proportional(10.), + if deg == 0. { MUTED } else { DIM }, + ); + } + } + frequency_grid(&p, plot, false); + let curve = |pick: fn(&weld_core::CurvePoint) -> f32, + color: Color32, + width: f32, + fade: bool| { + let mut previous: Option<(Pos2, f32)> = None; + for i in 0..CURVE { + let c = report.curve[i]; + if !c.valid { + previous = None; + continue; + } + let v = pick(&c); + let pos = Pos2::new(freq_x(plot, curve_frequency(i)), y(v)); + if let Some((last, last_v)) = previous { + // Do not draw the jump where the phase wraps. + if (v - last_v).abs() < 180. { + let alpha = if fade { 0.25 + 0.75 * c.coherence } else { 1. }; + p.line_segment([last, pos], Stroke::new(width, color.gamma_multiply(alpha))); + } + } + previous = Some((pos, v)); + } + }; + curve(|c| c.chain, FOCUS, 1.5, false); + curve(|c| c.before, MUTED, 1.5, true); + curve(|c| c.after, ACCENT, 2., true); +} + +fn controls(ui: &mut egui::Ui, settings: &mut Settings, state: &mut State) { + let d = Settings::default(); + ui.horizontal(|ui| { + caption(ui, "MODE"); + for (mode, label, help) in [ + (Mode::Follow, "FOLLOW", "Keep measuring and refit when a clearly better alignment shows up."), + (Mode::Learn, "LEARN", "Measure from now with equal weight, for example over one drop. Switch to HOLD to keep the result."), + (Mode::Hold, "HOLD", "Keep the current alignment. The display keeps measuring. The alignment is saved with the project."), + ] { + let r = ui.selectable_label(settings.mode == mode, RichText::new(label).size(14.)); + if r.clicked() { + settings.mode = if mode == Mode::Learn && settings.mode == Mode::Learn { + Mode::Hold + } else { + mode + }; + } + hint(&r, state, help); + } + ui.add_space(30.); + caption(ui, "SECTIONS"); + let minus = ui.button(RichText::new("−").size(14.)); + if minus.clicked() && settings.sections > 1 { + settings.sections -= 1; + } + ui.label(RichText::new(format!("{}", settings.sections)).monospace().size(14.).color(TEXT)); + let plus = ui.button(RichText::new("+").size(14.)); + if plus.clicked() && settings.sections < MAX_SECTIONS { + settings.sections += 1; + } + let help = "Most allpass sections the fit may use. Fewer is gentler; Weld uses fewer when they are enough."; + hint(&minus, state, help); + hint(&plus, state, help); + }); + ui.add_space(4.); + ui.columns(6, |c| { + knob(&mut c[0], state, "Response", &mut settings.response, 0.3..=5., d.response, " s", + "Averaging time in FOLLOW. Longer is steadier; shorter follows switch-ups faster."); + knob(&mut c[1], state, "Low", &mut settings.low, 30. ..=400., d.low, " Hz", + "Lowest frequency the fit listens to. Leave the sub to Anvil."); + knob(&mut c[2], state, "High", &mut settings.high, 1000. ..=12000., d.high, " Hz", + "Highest frequency the fit listens to."); + knob(&mut c[3], state, "Gate", &mut settings.gate, -70. ..=-30., d.gate, " dB", + "Moments where either layer is quieter than this are not measured."); + knob(&mut c[4], state, "Output", &mut settings.output, -12. ..=12., d.output, " dB", + "Output trim."); + let report = state.report; + c[5].vertical(|ui| { + caption(ui, "TIME OFFSET"); + let clear = report.lag_strength > 0.3; + ui.label(big( + if clear { format!("{:+.2} ms", report.lag_ms) } else { "—".into() }, + 22., + if clear { TEXT } else { DIM }, + )); + ui.label( + RichText::new("Positive: this layer is late. Weld can only delay it, so a late layer needs Weld on the other one.") + .size(11.) + .color(DIM), + ); + }); + }); +} diff --git a/tools/editor-check/Cargo.lock b/tools/editor-check/Cargo.lock index 35838ae..d719d79 100644 --- a/tools/editor-check/Cargo.lock +++ b/tools/editor-check/Cargo.lock @@ -648,6 +648,8 @@ checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" name = "weld-core" version = "0.1.0" dependencies = [ + "crossbeam-queue", + "rustfft", "serde", ] diff --git a/tools/editor-check/src/main.rs b/tools/editor-check/src/main.rs index cbc672d..f735f97 100644 --- a/tools/editor-check/src/main.rs +++ b/tools/editor-check/src/main.rs @@ -10,7 +10,11 @@ fn main() { .find(|a| a[0] == "--plugin") .map(|a| a[1].as_str()) .unwrap_or("rivet"); - if !["rivet", "patina", "anvil", "crucible", "lathe", "temper", "weld"].contains(&plugin) { + if ![ + "rivet", "patina", "anvil", "crucible", "lathe", "temper", "weld", + ] + .contains(&plugin) + { eprintln!("Use --plugin rivet, patina, anvil, crucible, lathe, temper or weld"); std::process::exit(2); } @@ -145,16 +149,41 @@ fn main() { let mut weld_settings = weld_core::Settings::default(); let mut weld_state = weld_ui::State::default(); if weld { - // Two layers of an F1 saw bass; the sidechain layer is polarity-flipped. + // Two layers of an F1 saw bass. The sidechain layer is polarity-flipped and runs + // through an allpass at 400 Hz, as an EQ in its chain would leave it. The real + // analyzer measures them and its fits go back to the engine, as on the worker. let mut engine = weld_core::Engine::new(48000., 2).unwrap(); engine.set_settings(weld_settings).unwrap(); + let link = engine.link(); + let mut analyzer = weld_core::Analyzer::new(48000., weld_core::Fit::NONE); + let mut other = weld_core::Chain::new( + weld_core::Fit { + flip: true, + count: 1, + sections: [weld_core::Section { fc: 400., q: 0.7 }; weld_core::MAX_SECTIONS], + }, + 48000., + ); let mut saw = common_dsp::Saw::new(48000., 43.65); - for i in 0..36000 { + for i in 0..48000 * 4 { let x = (saw.sample() * 1.6).tanh() * 0.4; - engine.process_sample([x, x], [-0.7 * x, -0.7 * x]); + let b = 0.8 * other.process(0, x); + engine.process_sample([x, x], [b, b]); if i % 1600 == 0 { weld_state.accept_frame(engine.take_analyzer()); } + while let Some(settings) = link.pop_settings() { + analyzer.set_settings(settings); + } + while let Some(block) = link.pop_block() { + analyzer.push(&block.a, &block.b); + if let Some(update) = analyzer.poll() { + if let Some(fit) = update.fit { + link.push_fit(fit); + } + weld_state.accept_report(update.report); + } + } } weld_state.accept_frame(engine.take_analyzer()); }