How cross-thread signalling works in Ardour
Why we need this
Ardour's architecture is fundamentally rooted in a system of
anonymous callbacks that allows changes in state to be propagated to
an arbitrary collection of objects without those objects being known
to the object undergoing the state change. For example, when the
mute state of a track changes, we notify interested objects of this
via an object called a
signal. Interested objects
register their interest by
connecting to the signal,
providing a callback that will be invoked at appropriate times. We
call the notification (and invocation of all attached callbacks)
"emitting" the signal (the analogy is to a broadcast system that
signals some event by just broadcasting it to whoever is listening).
For example, the GUI wants to know about changes to the mute state
of a track - it can connect to a
signal by providing a
callback (functor/closure) that will be called when the state changes.
The object that owns the signal (in this case the track)
has no idea who has connected to the signal, no idea about their
types or behavior. It simply emits the signal when the mute state
changes, and leaves the callbacks that have been connected to the
signal to do whatever they do (e.g. change the color of the mute
button for that track).
In a single-threaded program this is a very simple architecture to implement. However, in a program where the signal may be emitted by one thread but must be handled in a different thread, things get significantly more complex. This is the case when one of the threads that attaches callbacks to signals is the GUI: we have an absolute rule that all GUI operations must be carried out in the single, main GUI thread. Consequently, if a signal is emitted by another thread, we cannot simply invoke the callback in the emitting thread: we require a mechanism to ensure that the callback is invoked in the receiving thread instead.
This document describes that mechanism, as it exists in April 2023.
- We do not create threads randomly. Most threads are created during program startup. Later-created threads are rare and created one at a time for specific purposes.
- Realtime threads must be able to use signals without violating real-time safety (as far as possible given underlying OS limitations)
- Realtime threads do not emit signals very often
- A thread/event loop that is interested in the state change
announced by a signal calls
Signal<>::connectand passes a callback as a functor/closure
- The provied functor is rewrapped as a call
EventLoop::call_slot()and then added to list stored inside the
- A different thread emits
Signal<>, which involves walking the list of functors/callbacks, and invokes each one.
- This invokes
EventLoop::call_slot()for each, which checks if the calling thread is the thread running the event loop that registered the functor/callback
- If the calling thread is the event loop, then the inner
functor/callback (the one provided when connecting to
Signal<>) is invoked directly.
- If the calling thread is not the event loop, then a new "CallSlot" request is created for the target event loop, written to a buffer of such requests, and the target event loop is notified that there is a request pending.
- Receiving thread wakes up to the notification, scans for new requests and if it finds any, invokes the functor/callback stored in the request.
This is an OS-dependent mechanism for waking up another thread. On Linux we use fifos (pipes). It is hard to avoid the possibility of blocking when writing to an OS primitive, so we just have to pick the best option an OS offers us.
The object wraps the underlying OS primitive into something that be waited on by a glib event loop.
An abstract class that contains the implementation for roughly half of the architecture described here. The rest is left to AbstractUI<RequestObject>.
EventLoop, this is convenient wrapper
around a glib event loop, and provides the basic idea of a user
interface thread that handles notifications from our cross-thread
BaseUI. When inherited from by, for
example Gtkmm2ext::UI or MidiSurface, it represents a thread running a glib
event loop, where various events will drive a body of code that
provides a user interface for the program (not necessarily
graphical). Each derived class specifies a RequestObject type, which
defines the set of requests that can be delivered to the event loop
that is running the UI.
Both the GUI and all control surfaces have an IS-A relationship
RequestBuffer is a single-reader, single-writer FIFO
(also known as a lock-free FIFO, or ringbuffer) that holds a fixed
number of the
RequestObject type for that
The signal-emitting thread writes the correct values
RequestObject in the
then signals the target thread using
CrossThreadChannel. This wakes the target thread if
it is sleeping.
The signal-handling thread is woken by
CrossThreadChannel, and proceeds to check the
number of readable
RequestObjects in each RequestBuffer
Every event loop object has a
std::map that associates
pthread_t with a RequestBuffer. This means that at some
point, the event loop object must discover the
that identifies every possible sending thread, and then create a
RequestBuffer for that thread, and create a new map entry that
associates the two.
Consequently, before any thread may emit
Signal<> it must first
This call does two things:
- It pre-registers the thread, storing the thread name and pthread_t and a likely number of requests required for its request buffer in a static data structure.
- A signal is emitted
ThreadCreatedWithRequestSize) which is handled by all currently existing
Existing event loops will have
EventLoop::register_thread() method invoked. This
will create a new per-thread
RequestBuffer for the
Event loops that are created in the future (after the creation of the (potentially) signal emitting thread) will scan the static data structure during their construction, and will create request buffers for everything they find there.
Request Buffer memory management
As of April 2023, the request buffers and map entries all leak, because there is no easy to enforce cleanup routine when a given thread exits. We do not regard this as a problem, because the resources involved are very small, and there are not many threads that exit before program end anyway.
We could investigate
pthread_cleanup_push() as a way to
notify event loops that a thread has exited, and thus cleanup the
request buffers and map entries associated with it.
Each event loop's
protected by a RWLock, which allows signal emitting threads (which
only need to read the map) to hold concurrent read locks, and not
block other reader's progress.
A write-lock is taken on the map when creating and
RequestBuffers for threads. However, as
stated in our Assumptions section above, almost all thread creation happens at
a time where realtime behavior is not required. It is possible that
later thread creation and RequestBuffer registration could interfere
with a realtime thread, but this would require this to take place in
parallel with a signal emission by a realtime thread, and these are
rare (and typically connected with other user operations that would
not happen in parallel with the signal. The main examples would be
the PortRegisteredOrUnregistered or PortConnectedOrDisconnected
The static (global) data structure that holds pre-registered thread information is protected by a static Mutex. This data structure is only written or read in non-RT sections, and so possibly blocking on the mutex is entirely acceptable.