[fluid-dev] Multi-channel output configuration

[Marcus Weseloh]

Hi all,

JJC has recently proposed a new feature that would allow more control
over the external and internal audio routing. You can find the
proposal and subsequent discussion in the following issue and
pull-request:
https://github.com/FluidSynth/fluidsynth/issues/669
https://github.com/FluidSynth/fluidsynth/pull/672

During the discussion I suggested that we should maybe re-work the
current way of configuring the audio routing beforehand. The current
system with synth.audio-channels, synth.audio-groups,
synth.effects-channels and synth.effects-groups is quite complicated
to understand and also quite limited. So I would like to propose a new
way to configure and think about audio routing in FluidSynth,
especially with regard to multi-channel output configurations. Please
bear with me, this is going to be quite a long email...

Here comes my proposal:

 Routing Configuration interface
========================
The main idea is that we cleanly separate the creation and the
configuration of audio channels. Creating more ouput channels or
effects units does not mean that they will automatically be used.
Instead you need to configure the routing to make use of them.

So with the following three settings, we control how many channels and
units we instantiate on startup of the synth. So these settings need
to be set before creating the synth:

- synth.output-channels : the number of stereo output channels to open
(default 1)
- synth.bus-channels : the number of internal stereo busses to create
(default 0)
- synth.fx-units : the number of effect units (chorus+reverb) to
create (default 1)

And with the following two shell commands we control the audio routing
within FluidSynth:

reset_audio_routes
    - resets all routing to the default setup (more on that below)

audio_route <source> [<target>  ... ]
   - configures the audio routing from <source> to any number of
<targets>. Leave out <target> if you want to completely ignore audio
from <source>

<source> and <target> can be something like
- ch:1 for MIDI channel 1
- chfx:15 for MIDI channel 15 effect sends (reverb left, chorus right)
- out:1 for the first output channel
- bus:2 for the second internal bus
- fx:3 for the third fx unit

Of course there would also be corresponding API calls for these shell
commands. But I'll leave them out for now because I think they should
be fairly similar to the shell commands.


Default Configuration
================
The default setup of FluidSynth is what 99% of all users need and what
is also currently the default: all 16 MIDI channels render their audio
into the first output channel, send audio to the single fx unit, the
fx unit mixes it's output to the output channel.
So if we were to explicitly configure it, it would look like this
(channels 3 - 15 left out to keep this mail shorter):

# synth.output-channels=1, synth.bus-channels=0, synth.fx-units=1
audio_route ch:1 out:1
audio_route chfx:1 fx:1
audio_route ch:2 out:1
audio_route chfx:2 fx:1
...
audio_route ch:16 out:1
audio_route chfx:16 fx:1
audio_route fx:1 out:1


Now let's look at a few other use-cases that we could support using
this interface.

Separate Dry and Wet Audio
======================
The user wants the output from the effects unit separate instead of
mixed with the output channel.

# synth.output-channels=2, synth.bus-channels=0, synth.fx-units=1
audio_route fx:1 out:2

MIDI channels 5 and 6 on a separate output, including effects
==============================================
All MIDI channels output as normal to output 1, but channels 5 and 6
should render to output 2 instead, including their effects signals.

# synth.output-channels=2, synth.bus-channels=0, synth.fx-units=2
audio_route ch:5 out:2
audio_route chfx:5 fx:2
audio_route ch:6 out:2
audio_route chfx:6 fx:2
audio_route fx:2 out:2

Duplicate output for MIDI channel 1
===========================
All MIDI channels output as normal, only MIDI channel 1 should be
heard on the first and second output (including effects).

# synth.output-channels=2, synth.bus-channels=0, synth.fx-units=2
audio_route ch:1 out:1 out:2
audio_route chfx:1 fx:1 fx:2
audio_route fx:2 out:2

Use LADSPA reverb effect for MIDI channel 7
==============================
All MIDI channels output as normal, but MIDI channel 7 should skip the
internal effects unit and be processed by a LADSPA reverb effect
instead (and no chorus), then mixed into the output.

# synth.output-channels=1, synth.bus-channels=1, synth.fx-units=1
audio_route chfx:7 bus:1
audio_route bus:1 out:1
ladspa_effect e1 /usr/lib/ladspa/tap_reverb.so
ladspa_link e1 "Input Left" bus:1:left  (reverb send is on left
channel of the bus)
ladspa_link e1 "Input Right" bus:1:left
ladspa_link e1 "Output Left" out:1:left
ladspa_link e1 "Output Right" out:1:right
ladspa_start


Final Notes
=========
All calls to audio_route (and the corresponding API calls) would be
real-time capable and can be issued white the synth is playing. So you
could move MIDI channels between outputs on the fly. Using the API
instead of the shell commands would even allow to create a small
handler that reacts to external MIDI commands and changes the routing
based on certain rules (which would satisfy JJC use-case).

Now I understand that not everything that this configuration interface
offers is currently possible. And I don't think that is very important
at the moment, we could start with a simpler version and gradually
extend it to have more functionality. But I personally would think
that a configuration interface like this is much clearer than the
current system and more predictable, even though it might look more
complicated at first. And it would be a good base to implement JJCs
feature request.

I'm looking forward to your thoughts!

Cheers
Marcus