Ultra Low Latency FFT

2026-02-22 17:02 UTC gpt-5-2-thinking Open in ChatGPT ↗

Below is a “minimum-theoretical-latency” audio analyzer built around DSPGraph / DOTS Audio and the DSPClock (sample-accurate timebase). It runs the FFT on the audio mixer thread (inside the IAudioOutput hook), so your analysis latency is bounded by the hardware/Unity DSP buffer (i.e., essentially dspBufferSize / sampleRate), not by Update() timing. DSPGraph exposes DSPBufferSize, SampleRate, and a sample-count DSPClock directly. citeturn1view1

Notes before you paste:

DSPGraph is an experimental package (Unity has said it remains experimental in the past). citeturn0search9turn1view0

In Unity’s built-in audio system, the DSP timebase is AudioSettings.dspTime (seconds). Here we use DSPGraph.DSPClock (samples) for even more direct sample indexing. citeturn0search13turn1view1

To actually get minimum latency you must set Unity’s DSP Buffer Size to the smallest possible (Project Settings → Audio → DSP Buffer Size, typically “Best Latency”), and keep your FFT size aligned to the DSP buffer.

UltraLowLatencyFftAnalyzer.cs (single file)

//
// Package requirement:
//   - com.unity.audio.dspgraph (DSPGraph Audio Framework)
// Also recommended:
//   - Burst, Collections, Mathematics
//
// What this does:
//   - Hooks Unity's default audio output using DSPGraph's IAudioOutput.
//   - On the audio mixer thread, it reads the final mixed output buffer,
//     accumulates into an FFT window, runs a radix-2 FFT (Burst-friendly),
//     and publishes a magnitude spectrum + a sample-accurate DSPClock timestamp.
//   - On the main thread, you can pull the latest spectrum without allocations.
//
// Why this is "minimum theoretical latency":
//   - Analysis runs in EndMix() on the mixer thread, per audio buffer.
//   - The earliest you can know about audio is one DSP buffer "late":
//     latency ≈ dspBufferSize / sampleRate seconds (plus ADC/driver I/O if mic).

using System;
using System.Threading;
using Unity.Burst;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Mathematics;
using UnityEngine;

// DSPGraph namespace
using Unity.Audio;

public sealed class UltraLowLatencyFftAnalyzer : MonoBehaviour
{
    [Header("FFT")]
    [Tooltip("Must be a power of two. If set to 0, uses the current DSP buffer size (recommended for minimum latency).")]
    public int fftSize = 0;

    [Tooltip("If true, mixes all output channels down to mono before FFT.")]
    public bool mixToMono = true;

    [Tooltip("Applies a Hann window before FFT (recommended).")]
    public bool useHannWindow = true;

    [Header("Readback")]
    [Tooltip("If true, magnitude is converted to dBFS (log). Otherwise linear magnitude.")]
    public bool outputDb = false;

    // Public readback API
    public int CurrentFftSize => _fftSize;
    public int SampleRate => _sampleRate;
    public int DspBufferSize => _dspBufferSize;

    // Latest published timestamp in samples (DSPClock at publish time)
    public long LatestDspClockSamples => Interlocked.Read(ref _latestClockSamples);

    // Estimated lower bound on analysis latency (seconds)
    public double TheoreticalMinLatencySeconds => (_sampleRate > 0 && _dspBufferSize > 0)
        ? (double)_dspBufferSize / _sampleRate
        : 0.0;

    // --- Internal state ---
    private AudioOutputHandle _outputHandle;
    private FftOutputJob _job; // struct copied into output system
    private GCHandle _thisHandle;

    private int _fftSize;
    private int _sampleRate;
    private int _dspBufferSize;
    private long _latestClockSamples;

    // Double-buffered spectrum storage (main thread owns these arrays)
    private NativeArray<float> _spectrumA;
    private NativeArray<float> _spectrumB;
    private volatile int _publishedIndex = -1; // 0 => A, 1 => B

    // We expose a "pull" that copies into your buffer (no allocations).
    // Returns false if no spectrum has been published yet.
    public bool TryGetLatestSpectrum(NativeArray<float> destination, out long dspClockSamples)
    {
        dspClockSamples = LatestDspClockSamples;

        int idx = _publishedIndex;
        if (idx < 0) return false;

        var src = (idx == 0) ? _spectrumA : _spectrumB;
        if (!src.IsCreated || destination.Length != src.Length)
            throw new ArgumentException("Destination length must equal FFT bin count (fftSize/2).");

        NativeArray<float>.Copy(src, destination);
        return true;
    }

    // Convenience overload for float[] (alloc-free if you reuse the array).
    public bool TryGetLatestSpectrum(float[] destination, out long dspClockSamples)
    {
        dspClockSamples = LatestDspClockSamples;

        int idx = _publishedIndex;
        if (idx < 0) return false;

        var src = (idx == 0) ? _spectrumA : _spectrumB;
        if (!src.IsCreated || destination.Length != src.Length)
            throw new ArgumentException("Destination length must equal FFT bin count (fftSize/2).");

        src.CopyTo(destination);
        return true;
    }

    private void OnEnable()
    {
        // Pin 'this' so the job can call back into a stable pointer for publishing.
        _thisHandle = GCHandle.Alloc(this, GCHandleType.Pinned);

        // Create the output job and attach to default output.
        // IAudioOutput requires Initialize/BeginMix/EndMix/Dispose. citeturn3view0
        _job = new FftOutputJob
        {
            OwnerPtr = (IntPtr)_thisHandle.AddrOfPinnedObject(),
            RequestedFftSize = fftSize,
            MixToMono = mixToMono ? 1 : 0,
            UseHannWindow = useHannWindow ? 1 : 0,
            OutputDb = outputDb ? 1 : 0,
        };

        // Attach to Unity's default output. citeturn2search12
        _outputHandle = _job.AttachToDefaultOutput();
    }

    private void OnDisable()
    {
        // Dispose the output hook (this calls job.Dispose on the mixer thread).
        if (_outputHandle.IsValid)
        {
            _outputHandle.Dispose();
            _outputHandle = default;
        }

        if (_thisHandle.IsAllocated)
            _thisHandle.Free();

        if (_spectrumA.IsCreated) _spectrumA.Dispose();
        if (_spectrumB.IsCreated) _spectrumB.Dispose();
    }

    // Called by the audio job (mixer thread) to publish results into the double buffer.
    // IMPORTANT: Keep this tiny and allocation-free. No Unity API calls.
    private unsafe void PublishSpectrum(float* mags, int binCount, long dspClockSamples)
    {
        // Lazily allocate spectrum buffers on the main object (still safe; uses unmanaged alloc).
        // First publish will happen on mixer thread; allocations are NativeArray unmanaged, but
        // you should still aim to have them created early. If you want zero risk, pre-create
        // in Initialize via a custom bootstrap — this keeps the script "single file".
        if (!_spectrumA.IsCreated)
        {
            _spectrumA = new NativeArray<float>(binCount, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
            _spectrumB = new NativeArray<float>(binCount, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
        }

        // Write into the non-published buffer, then atomically flip.
        int next = (_publishedIndex == 0) ? 1 : 0;
        var dst = (next == 0) ? _spectrumA : _spectrumB;

        // Copy from job memory into published buffer
        UnsafeUtility.MemCpy(
            dst.GetUnsafePtr(),
            mags,
            binCount * sizeof(float)
        );

        Interlocked.Exchange(ref _latestClockSamples, dspClockSamples);
        Volatile.Write(ref _publishedIndex, next);
    }

    // Mixer-thread job: hooks output, runs FFT per buffer.
    [BurstCompile]
    private struct FftOutputJob : IAudioOutput
    {
        public IntPtr OwnerPtr;

        public int RequestedFftSize;
        public int MixToMono;     // 1/0
        public int UseHannWindow; // 1/0
        public int OutputDb;      // 1/0

        private DSPGraph _graph;

        // Output format info
        private int _channels;
        private int _sampleRate;
        private int _dspBufferSize;

        // FFT config
        private int _fftSize;
        private int _binCount;

        // Accumulator for time-domain window
        private NativeArray<float> _time;         // mono window (fftSize)
        private int _timeWrite;                   // ring write head

        // Complex FFT buffer (in-place)
        private NativeArray<float2> _freq;        // length fftSize

        // Window coefficients
        private NativeArray<float> _hann;         // length fftSize

        // Magnitude output (binCount)
        private NativeArray<float> _mags;         // job-local mags for publish

        // Twiddle factors and bit-reversal indices
        private NativeArray<float2> _twiddles;    // length fftSize/2
        private NativeArray<int> _bitRev;         // length fftSize

        public void Initialize(int channelCount, SoundFormat format, int sampleRate, long dspBufferSize)
        {
            _channels = channelCount;
            _sampleRate = sampleRate;
            _dspBufferSize = (int)dspBufferSize;

            // Create a DSPGraph that matches the output device.
            // DSPGraph.Create(outputFormat, outputChannels, dspBufferSize, sampleRate). citeturn1view1
            _graph = DSPGraph.Create(format, channelCount, _dspBufferSize, _sampleRate);

            // Choose FFT size:
            // - If RequestedFftSize == 0: use DSP buffer size (best latency).
            // - Else: clamp to power-of-two >= dspBufferSize (to avoid repeated partial windows).
            _fftSize = RequestedFftSize > 0 ? NextPow2(math.max(RequestedFftSize, _dspBufferSize)) : NextPow2(_dspBufferSize);

            _binCount = _fftSize / 2;

            _time = new NativeArray<float>(_fftSize, Allocator.Persistent, NativeArrayOptions.ClearMemory);
            _freq = new NativeArray<float2>(_fftSize, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
            _mags = new NativeArray<float>(_binCount, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);

            if (UseHannWindow != 0)
            {
                _hann = new NativeArray<float>(_fftSize, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
                for (int i = 0; i < _fftSize; i++)
                {
                    // Hann: 0.5 - 0.5*cos(2*pi*n/(N-1))
                    _hann[i] = 0.5f - 0.5f * math.cos((2f * math.PI * i) / (_fftSize - 1));
                }
            }

            // Precompute FFT helpers
            _twiddles = new NativeArray<float2>(_fftSize / 2, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
            for (int k = 0; k < _twiddles.Length; k++)
            {
                float phase = -2f * math.PI * k / _fftSize;
                _twiddles[k] = new float2(math.cos(phase), math.sin(phase));
            }

            _bitRev = new NativeArray<int>(_fftSize, Allocator.Persistent, NativeArrayOptions.UninitializedMemory);
            int bits = (int)math.log2(_fftSize);
            for (int i = 0; i < _fftSize; i++)
                _bitRev[i] = BitReverse(i, bits);

            _timeWrite = 0;
        }

        public void BeginMix(int frameCount)
        {
            // Mix frameCount frames in the graph (jobified by default).
            _graph.BeginMix(frameCount);
        }

        public void EndMix(NativeArray<float> output, int frames)
        {
            // Read mixed audio into 'output' (Unity passes this buffer in).
            // We fill it from the graph mix.
            _graph.ReadMix(output, frames, _channels);

            // Push samples into the mono window accumulator.
            // frames == dsp buffer frames (typically).
            int totalSamples = frames * _channels;

            if (MixToMono != 0 && _channels > 1)
            {
                for (int f = 0; f < frames; f++)
                {
                    float sum = 0f;
                    int baseIdx = f * _channels;
                    for (int c = 0; c < _channels; c++)
                        sum += output[baseIdx + c];

                    float mono = sum / _channels;
                    _time[_timeWrite] = mono;
                    _timeWrite = (_timeWrite + 1) & (_fftSize - 1);
                }
            }
            else
            {
                // Use channel 0 only
                for (int f = 0; f < frames; f++)
                {
                    float s = output[f * _channels];
                    _time[_timeWrite] = s;
                    _timeWrite = (_timeWrite + 1) & (_fftSize - 1);
                }
            }

            // For minimum latency, run FFT once per output buffer.
            // We build a contiguous window ending at the newest sample.
            int start = _timeWrite; // oldest sample index in ring
            for (int i = 0; i < _fftSize; i++)
            {
                int idx = (start + i) & (_fftSize - 1);
                float w = (UseHannWindow != 0) ? _hann[i] : 1f;
                _freq[i] = new float2(_time[idx] * w, 0f);
            }

            // FFT in-place (radix-2, iterative, Burst-friendly)
            Radix2Fft.InPlace(_freq, _bitRev, _twiddles);

            // Magnitude spectrum (bins 0..N/2-1), normalized by N
            float invN = 1f / _fftSize;
            for (int k = 0; k < _binCount; k++)
            {
                float2 z = _freq[k];
                float mag = math.sqrt(z.x * z.x + z.y * z.y) * invN;

                if (OutputDb != 0)
                {
                    // dBFS-ish: 20*log10(mag), protect against log(0)
                    mag = 20f * math.log10(math.max(mag, 1e-12f));
                }

                _mags[k] = mag;
            }

            // Publish to the MonoBehaviour (main object) with sample-accurate timestamp.
            // DSPClock is "number of samples processed since the DSPGraph was created". citeturn1view1
            long dspClockSamples = _graph.DSPClock;

            unsafe
            {
                var owner = (UltraLowLatencyFftAnalyzer*)OwnerPtr;
                owner->PublishSpectrum((float*)_mags.GetUnsafeReadOnlyPtr(), _binCount, dspClockSamples);
            }
        }

        public void Dispose()
        {
            if (_graph.Valid)
                _graph.Dispose();

            if (_time.IsCreated) _time.Dispose();
            if (_freq.IsCreated) _freq.Dispose();
            if (_mags.IsCreated) _mags.Dispose();
            if (_hann.IsCreated) _hann.Dispose();
            if (_twiddles.IsCreated) _twiddles.Dispose();
            if (_bitRev.IsCreated) _bitRev.Dispose();
        }

        private static int NextPow2(int v)
        {
            v = math.max(2, v);
            v--;
            v |= v >> 1;
            v |= v >> 2;
            v |= v >> 4;
            v |= v >> 8;
            v |= v >> 16;
            v++;
            return v;
        }

        private static int BitReverse(int x, int bits)
        {
            int y = 0;
            for (int i = 0; i < bits; i++)
            {
                y = (y << 1) | (x & 1);
                x >>= 1;
            }
            return y;
        }
    }

    [BurstCompile]
    private static class Radix2Fft
    {
        // In-place iterative radix-2 FFT with:
        // - explicit bit-reversal permutation
        // - precomputed twiddles e^{-j 2pi k / N}
        public static void InPlace(NativeArray<float2> data, NativeArray<int> bitRev, NativeArray<float2> twiddles)
        {
            int n = data.Length;

            // Bit-reversal permute into a scratchless in-place swap
            for (int i = 0; i < n; i++)
            {
                int j = bitRev[i];
                if (j > i)
                {
                    float2 tmp = data[i];
                    data[i] = data[j];
                    data[j] = tmp;
                }
            }

            // Cooley-Tukey iterative
            for (int len = 2; len <= n; len <<= 1)
            {
                int half = len >> 1;
                int step = n / len; // twiddle step

                for (int i = 0; i < n; i += len)
                {
                    int tw = 0;
                    for (int j = 0; j < half; j++)
                    {
                        float2 u = data[i + j];
                        float2 v = data[i + j + half];

                        float2 w = twiddles[tw];

                        // v * w (complex multiply)
                        float2 t = new float2(
                            v.x * w.x - v.y * w.y,
                            v.x * w.y + v.y * w.x
                        );

                        data[i + j] = u + t;
                        data[i + j + half] = u - t;

                        tw += step;
                    }
                }
            }
        }
    }
}

How to use

Install DSPGraph: Package Manager → add DSPGraph Audio Framework (com.unity.audio.dspgraph). DSPGraph is Unity’s low-level C# jobified audio engine layer. citeturn1view0turn1view1
Enable Burst (Jobs → Burst → Enable Compilation). DSPGraph output jobs should be Burst-compiled to avoid GC stalls on the mixer thread. citeturn3view0
Set DSP Buffer Size to the smallest possible in Project Settings → Audio (commonly “Best Latency”).
Add UltraLowLatencyFftAnalyzer to any GameObject.
Read spectrum from your visualizer code:

public class SpectrumConsumer : MonoBehaviour
{
    public UltraLowLatencyFftAnalyzer analyzer;
    private float[] _bins;

    void Start()
    {
        _bins = new float[analyzer.CurrentFftSize / 2];
    }

    void Update()
    {
        if (analyzer.TryGetLatestSpectrum(_bins, out long clock))
        {
            // _bins now holds magnitude (linear or dB), clock is sample-accurate.
            // Example: use _bins[bin] to drive visuals.
        }
    }
}

What “minimum theoretical latency” means here

Your analysis cadence is once per DSP buffer (EndMix is called per output buffer).
So a hard lower bound is roughly:
latency ≈ DSPBufferSize / SampleRate seconds (plus device/driver I/O if you’re analyzing microphone input).
This is the same underlying idea as Unity’s AudioSettings.dspTime clock being based on processed sample count, but here we also get the DSPClock in samples directly from DSPGraph. citeturn0search13turn1view1

If you tell me whether you’re analyzing (A) the final master output (what this script does), (B) a specific bus/group, or (C) live microphone input, I can tailor the graph hookup so you’re analyzing exactly the signal you want (and keep the same sample-accurate timestamping).

DOTS Audio (DSPGraph) — Everything You Actually Need to Know

DOTS Audio is Unity’s low-level, jobified audio engine built around a graph architecture. It’s exposed primarily through the DSPGraph package.

Package: com.unity.audio.dspgraph
Namespace: Unity.Audio

It is fundamentally different from the classic AudioSource / AudioMixer system.

1️⃣ What DOTS Audio Actually Is

At its core:

A C# DSP graph engine
Fully jobified
Sample-accurate
Designed for:
- procedural synthesis
- generative audio
- modular DSP systems
- ultra-low-latency analysis
- custom audio engines
- large-scale runtime patching

You don’t “play clips.”
You build a signal graph of DSP nodes.

2️⃣ Architecture Overview

The Core Objects

DSPGraph
DSPNode
DSPCommandBlock
IAudioOutput

Signal Flow

[Your Nodes] → [Graph] → [Mix] → [Audio Output Device]

Everything is:

Pull-based
Buffer-driven
Sample clock–based

3️⃣ The DSP Clock (Critical Concept)

DSPGraph exposes:

long DSPClock;     // number of samples processed since graph creation
int SampleRate;
int DSPBufferSize;

This gives you:

Absolute sample position
Deterministic timing
Sub-frame synchronization
Perfect sync with visuals if you convert samples → time

Time in seconds:

seconds = DSPClock / SampleRate

This is superior to Time.time for A/V sync.

4️⃣ How It Differs From Classic Unity Audio

Feature	Classic Audio	DOTS Audio
Architecture	Component-based	DSP graph
Threading	Mixed	Fully jobified
Timing	dspTime (double)	Sample clock
Custom DSP	Limited	Native
Latency	Higher	Lower possible
Determinism	No	Yes
Runtime patching	Limited	Yes

Classic system:

Easier
Production-ready
Good for games

DOTS:

Lower level
Experimental
Power-user oriented

5️⃣ What Makes It Powerful

1️⃣ Jobified DSP

Your DSP runs inside Unity’s Job System.

2️⃣ Burst Compatible

You can Burst-compile DSP code.

3️⃣ No GC

Fully NativeArray-based.

4️⃣ Dynamic Graph Patching

You can:

Add/remove nodes
Reconnect graph
Rewire routing
All at runtime

This is modular-synth-level flexibility.

6️⃣ What You Actually Build

You define:

struct MyNode : IAudioKernel<MyParams, MyProviders>

Inside:

public void Execute(ref ExecuteContext<MyParams, MyProviders> ctx)
{
    var input = ctx.Inputs.GetSampleBuffer(0);
    var output = ctx.Outputs.GetSampleBuffer(0);

    for (int i = 0; i < ctx.DSPBufferSize; i++)
    {
        output[i] = input[i] * gain;
    }
}

You are literally writing the DSP loop.

7️⃣ Minimal Mental Model

Think:

VCV Rack
Max/MSP
TouchDesigner audio
Unreal Audio Mixer low-level

But:

Written in C#
Running inside Unity
Jobified

8️⃣ How You Connect Things

Graph operations happen through:

DSPCommandBlock cmd = graph.CreateCommandBlock();
cmd.CreateDSPNode(...);
cmd.Connect(nodeA, 0, nodeB, 0);
cmd.Release();

Graph changes are scheduled safely on the audio thread.

You NEVER modify the graph directly.

9️⃣ Audio Output

To actually hear sound:

You attach to an output:

outputHandle = myOutputJob.AttachToDefaultOutput();

IAudioOutput allows you to intercept:

BeginMix
EndMix
Output buffer

This is how you:

Build analyzers
Capture final mix
Inject DSP

🔟 Latency Model

Latency is determined by:

DSPBufferSize / SampleRate

Typical:

48000 Hz
256 buffer

Latency ≈ 5.3ms

If you set 128 buffer:

≈ 2.6ms

That is near the theoretical limit.

11️⃣ Where DOTS Audio Shines

✔ Modular synth engines

✔ Procedural music

✔ Generative systems

✔ AV-reactive installations

✔ Custom mixing engines

✔ Sample-accurate event scheduling

✔ Large-scale runtime patching

12️⃣ Where It Is Weak

❌ No editor tooling

❌ No mixer UI

❌ No AudioClip convenience

❌ Limited documentation

❌ Experimental package

❌ Not widely battle-tested

It’s not a drop-in replacement for AudioSource.

13️⃣ Relationship to Entities (ECS)

Despite the name:

DOTS Audio does NOT require ECS.

It uses:

C# Jobs
NativeCollections
Burst

You can use it in classic MonoBehaviour projects.

14️⃣ Scheduling Sample-Accurate Events

You can schedule events based on DSPClock:

Example:

if (graph.DSPClock >= triggerSample)
    playEnvelope();

This gives you:

Sample-accurate triggering
Perfect beat sync
Deterministic timing

This is huge for AV sync.

15️⃣ Writing a Custom DSP Node (Conceptual Example)

Example: Oscillator

phase += frequency / sampleRate;
if (phase > 1f) phase -= 1f;

output[i] = math.sin(phase * 2f * math.PI);

Because you control:

Phase
Sample rate
Clock

You can make:

Poly synths
Sequencers
Granular engines
Convolution
Spectral effects

16️⃣ Memory Model

Everything uses:

NativeArray
Allocator.Persistent
No managed allocations

You must:

Dispose everything
Avoid GC
Avoid UnityEngine API inside DSP code

17️⃣ FFT & Analysis

DOTS Audio is ideal for:

Per-buffer FFT
Spectral processing
Real-time analyzers
Beat detection
Onset detection
Reactive systems

Because:

You run on mixer thread
No frame delay
Sample clock accurate

18️⃣ Graph Lifecycle

Create graph
Create nodes
Connect nodes
BeginMix
EndMix
Dispose

Graph lives independently of scene objects if needed.

19️⃣ Performance Characteristics

Extremely efficient if:

Burst enabled
Small buffer sizes
Power-of-two FFTs
No allocations
Avoid main-thread sync

It can outperform classic Unity audio for custom DSP.

20️⃣ Big Picture Use Cases

For someone like you (creative coding / live AV):

DOTS Audio is powerful if you want:

TouchDesigner-like modular DSP inside Unity
Sample-accurate A/V sync
Visual systems locked to DSP clock
Fully procedural systems
Custom runtime patch graphs

It becomes the audio equivalent of a node graph system.

21️⃣ Should You Use It?

Use DOTS Audio if:

You are building an audio engine
You need sample accuracy
You want ultra-low-latency analysis
You are comfortable writing DSP code
You want runtime patchable graphs

Do NOT use it if:

You just need music + SFX
You want stable production audio tools
You rely heavily on AudioMixer UI workflows

22️⃣ Strategic View for AV Systems

If you’re building:

A reactive visual engine
A VVVV/TouchDesigner hybrid in Unity
A graph-based creative coding environment

DOTS Audio gives you:

Deterministic time domain
Full signal access
Modular DSP backend
Perfect sync anchor

It can become your master timing source.

23️⃣ Current State (Reality Check)

Still experimental
Not deeply documented
API may change
Not officially replacing classic audio

But:

Extremely powerful
Architecturally clean
Future-oriented

24️⃣ The Core Mental Model (If You Remember Nothing Else)

DOTS Audio is:

A jobified, sample-accurate, modular DSP graph engine inside Unity.

It is closer to:

Max/MSP
VCV Rack
Unreal low-level audio mixer

Than to:

AudioSource

If you’d like, next I can:

🔬 Break down how to architect a modular synth in DSPGraph
🎛 Explain how to build a live AV clock system around DSPClock
⚙ Compare DOTS Audio vs Unreal’s Audio Mixer
🧠 Design a TouchDesigner-like audio graph architecture in Unity

Tell me which direction you want.

Here’s a complete DOTS Audio / DSPGraph “generate a tune” example you can paste into Unity. It builds a DSPGraph with a Burst-compiled synth node (2 detuned sines + simple ADSR-ish envelope), then runs a 16-step sequencer on the main thread that drives the node via DSPCommandBlock.SetFloat.

This is based on the same minimal “driver + OnAudioFilterRead pump” pattern as a known-working DSPGraph oscillator gist (so you can actually hear it without extra plumbing). citeturn3view0
DSPGraph concepts: nodes via IAudioKernel, graph editing via DSPCommandBlock, and DSPGraph.Create(...). citeturn1search2turn2search4

1) Install prerequisites

Package Manager → add DSPGraph Audio Framework (com.unity.audio.dspgraph) citeturn2search4
Enable Burst (recommended; DSPGraph output jobs can otherwise run in Mono and be GC-stall prone). citeturn1search10

2) Paste this script: `DotsTuneSynth.cs`

using System;
using Unity.Burst;
using Unity.Collections;
using Unity.Mathematics;
using UnityEngine;
using Unity.Audio;

/// <summary>
/// DOTS Audio / DSPGraph tune generator:
/// - Creates a DSPGraph
/// - Creates a Burst-compiled synth node (2 detuned sines + simple ADSR)
/// - Runs a 16-step sequencer driving Frequency + Gate parameters
/// - Pumps audio into Unity via OnAudioFilterRead (simple driver pattern)
///
/// Tested API style follows Unity DSPGraph examples in the wild. citeturn3view0
/// DSPGraph manual / core types: citeturn2search4turn1search2
/// </summary>
public sealed class DotsTuneSynth : MonoBehaviour
{
    [Header("Sequencer")]
    [Range(40, 220)] public float bpm = 124f;
    [Tooltip("16-step pattern. Use -1 for rest. Values are MIDI notes (e.g., 60 = C4).")]
    public int[] midiPattern = new int[16]
    {
        60, -1, 60, -1,
        67, -1, 65, -1,
        60, -1, 60, -1,
        67, 65, 62, -1
    };

    [Header("Synth")]
    [Range(0f, 0.8f)] public float volume = 0.18f;
    [Range(0f, 0.03f)] public float attack = 0.005f;
    [Range(0f, 0.5f)] public float release = 0.08f;
    [Range(-20f, 20f)] public float detuneCents = 6.0f;

    // DSPGraph + node
    private DSPGraph _graph;
    private DefaultDspGraphDriver _driver;
    private DSPNode _synthNode;

    // Pump buffer
    private NativeArray<float> _output;

    // Audio config
    private int _sampleRate;
    private int _frames;
    private int _channels;

    // Sequencer state
    private int _lastStep = -1;
    private bool _running;

    // Using AudioSettings dspTime (seconds) for stable scheduling vs frame time.
    // (We still generate the sound in DSPGraph; this is only sequencer timing.)
    void Awake()
    {
        var cfg = AudioSettings.GetConfiguration();
        _sampleRate = cfg.sampleRate;
        _frames = cfg.dspBufferSize;

        // This example assumes stereo output.
        _channels = 2;

        // Create graph. DSPGraph.Create(format, channels, dspBufferSize, sampleRate). citeturn3view0turn2search4
        _graph = DSPGraph.Create(SoundFormat.Stereo, _channels, _frames, _sampleRate);

        // Driver: BeginMix/EndMix pair around graph mixing (see gist pattern). citeturn3view0
        _driver = new DefaultDspGraphDriver { Graph = _graph };
        _driver.Initialize(_channels, SoundFormat.Stereo, _sampleRate, _frames);

        // Create synth node + connect to graph root.
        var cmd = _graph.CreateCommandBlock();
        _synthNode = cmd.CreateDSPNode<SynthParams, SynthProviders, TwoSineSynthKernel>();

        // Add an outlet for stereo
        cmd.AddOutletPort(_synthNode, _channels, SoundFormat.Stereo);

        // Connect node outlet 0 -> RootDSP inlet 0
        cmd.Connect(_synthNode, 0, _graph.RootDSP, 0);

        // Initialize parameters
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Volume, volume);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Attack, attack);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Release, release);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.DetuneCents, detuneCents);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Gate, 0f);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.FrequencyHz, 440f);
        cmd.Complete();

        _output = new NativeArray<float>(_channels * _frames, Allocator.Persistent, NativeArrayOptions.ClearMemory);

        // Start audio right away; some setups like a tiny delay, but often fine.
        _driver.BeginMix(_frames);
        _running = true;
    }

    void OnDestroy()
    {
        if (_running)
        {
            var cmd = _graph.CreateCommandBlock();
            cmd.ReleaseDSPNode(_synthNode);
            cmd.Complete();
        }

        _driver.Dispose();
        if (_output.IsCreated) _output.Dispose();
    }

    void Update()
    {
        if (!_running) return;

        // 16th-note step duration
        double secondsPerBeat = 60.0 / math.max(1e-6f, bpm);
        double secondsPerStep = secondsPerBeat / 4.0; // 16th notes (4 steps per beat)

        double t = AudioSettings.dspTime; // stable audio clock
        int step = (int)math.floor(t / secondsPerStep) % math.max(1, midiPattern.Length);
        if (step == _lastStep) return;
        _lastStep = step;

        int midi = midiPattern[step];
        float gate = (midi < 0) ? 0f : 1f;
        float freq = (midi < 0) ? 440f : MidiToHz(midi);

        // Update node params atomically via command block
        var cmd = _graph.CreateCommandBlock();
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Volume, volume);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Attack, attack);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Release, release);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.DetuneCents, detuneCents);

        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.FrequencyHz, freq);
        cmd.SetFloat<SynthParams, SynthProviders, TwoSineSynthKernel>(_synthNode, SynthParams.Gate, gate);
        cmd.Complete();
    }

    // Pump DSPGraph into Unity audio pipeline (classic pattern shown in the DSPGraph gist). citeturn3view0
    void OnAudioFilterRead(float[] data, int channels)
    {
        if (!_running) return;

        _driver.EndMix(_output, _frames);

        // Copy NativeArray -> float[]
        int n = math.min(data.Length, _output.Length);
        for (int i = 0; i < n; i++)
            data[i] = _output[i];

        _driver.BeginMix(_frames);
    }

    static float MidiToHz(int midi)
    {
        // A4=69 => 440 Hz
        return 440f * math.pow(2f, (midi - 69) / 12f);
    }

    // ==========================
    // DSPGraph types
    // ==========================

    public enum SynthParams
    {
        FrequencyHz,
        Gate,
        Volume,
        Attack,
        Release,
        DetuneCents
    }

    public enum SynthProviders
    {
        // Not used in this example; kept for the required generic signature.
        Provider
    }

    [BurstCompile]
    public struct TwoSineSynthKernel : IAudioKernel<SynthParams, SynthProviders>
    {
        private float _phaseA;
        private float _phaseB;

        // Envelope state
        private float _env;
        private float _gateSmoothed;

        public void Initialize() { }
        public void Dispose() { }

        public void Execute(ref ExecuteContext<SynthParams, SynthProviders> ctx)
        {
            // NOTE: ctx.DSPBufferSize exists, but many examples use a known frame count.
            // We'll use ctx.DSPBufferSize for correctness.
            int frames = ctx.DSPBufferSize;

            // Single stereo output buffer (interleaved)
            var outBuf = ctx.Outputs.GetSampleBuffer(0).Buffer;

            float sr = ctx.SampleRate;
            float twoPi = math.PI * 2f;

            for (int i = 0; i < frames; i++)
            {
                float freq = ctx.Parameters.GetFloat(SynthParams.FrequencyHz, i);
                float gate = ctx.Parameters.GetFloat(SynthParams.Gate, i);
                float vol  = ctx.Parameters.GetFloat(SynthParams.Volume, i);

                float atk  = math.max(1e-5f, ctx.Parameters.GetFloat(SynthParams.Attack, i));
                float rel  = math.max(1e-5f, ctx.Parameters.GetFloat(SynthParams.Release, i));
                float detC = ctx.Parameters.GetFloat(SynthParams.DetuneCents, i);

                // Smooth gate a bit to avoid clicks on hard transitions
                _gateSmoothed = math.lerp(_gateSmoothed, gate, 0.02f);

                // One-pole ADSR-ish: attack toward 1 when gate on, release toward 0 when off
                float atkCoeff = 1f - math.exp(-1f / (atk * sr));
                float relCoeff = 1f - math.exp(-1f / (rel * sr));
                float target = (_gateSmoothed > 0.5f) ? 1f : 0f;
                float coeff = (target > _env) ? atkCoeff : relCoeff;
                _env = math.lerp(_env, target, coeff);

                // Detune: cents -> ratio
                float detuneRatio = math.pow(2f, detC / 1200f);

                float stepA = (twoPi * freq) / sr;
                float stepB = (twoPi * (freq * detuneRatio)) / sr;

                float s = 0.55f * math.sin(_phaseA) + 0.45f * math.sin(_phaseB);
                s *= _env * vol;

                _phaseA += stepA;
                _phaseB += stepB;
                if (_phaseA > twoPi) _phaseA -= twoPi;
                if (_phaseB > twoPi) _phaseB -= twoPi;

                // Interleaved stereo (L,R)
                int baseIdx = i * 2;
                outBuf[baseIdx]     = s;
                outBuf[baseIdx + 1] = s;
            }
        }
    }

    [BurstCompile]
    public struct DefaultDspGraphDriver : IAudioOutput
    {
        public DSPGraph Graph;
        private int _channelCount;

        public void Initialize(int channelCount, SoundFormat format, int sampleRate, long dspBufferSize)
        {
            _channelCount = channelCount;
        }

        public void BeginMix(int frameCount) => Graph.BeginMix(frameCount);

        public void EndMix(NativeArray<float> output, int frames)
        {
            Graph.ReadMix(output, frames, _channelCount);
        }

        public void Dispose()
        {
            if (Graph.Valid)
                Graph.Dispose();
        }
    }
}

3) How to run it

Create an empty GameObject → add DotsTuneSynth
Press Play
Edit the pattern in the inspector (use MIDI notes, -1 = rest)

Try a simple scale:

60, 62, 64, 65,
67, 69, 71, 72,
-1, -1, -1, -1,
60, 67, 72, -1

4) If you want “more tune” (still simple)

Add a second node for bass (same kernel, one octave down)
Or switch waveform (saw/pulse) by changing s = ... in the kernel
Add a very cheap lowpass in the kernel (one-pole filter) for a “synth” vibe

5) Notes you’ll care about

This uses the DSPGraph graph model (IAudioKernel, DSPCommandBlock, RootDSP connection). citeturn3view0turn1search2
Sequencing uses AudioSettings.dspTime (audio clock) for stability; the actual sound generation is inside the DSP kernel.
For lower latency: set Unity’s DSP Buffer Size smaller (Project Settings → Audio). Smaller buffers = tighter timing but more CPU.

If you want, I can give you a version that is 100% sample-accurate sequencing inside the kernel (no Update() timing at all) using the graph’s sample clock / counters, so the tune timing is locked to the DSP buffer boundaries.