How cross-thread signalling works in Ardour

Why we need this

Ardour's architecture is fundamentally rooted in a system of anonymous callbacks that allows changes in state to be propagated to an arbitrary collection of objects without those objects being known to the object undergoing the state change. For example, when the mute state of a track changes, we notify interested objects of this via an object called a signal. Interested objects register their interest by connecting to the signal, providing a callback that will be invoked at appropriate times. We call the notification (and invocation of all attached callbacks) "emitting" the signal (the analogy is to a broadcast system that signals some event by just broadcasting it to whoever is listening).

For example, the GUI wants to know about changes to the mute state of a track - it can connect to a signal by providing a callback (functor/closure) that will be called when the state changes. The object that owns the signal (in this case the track) has no idea who has connected to the signal, no idea about their types or behavior. It simply emits the signal when the mute state changes, and leaves the callbacks that have been connected to the signal to do whatever they do (e.g. change the color of the mute button for that track).

In a single-threaded program this is a very simple architecture to implement. However, in a program where the signal may be emitted by one thread but must be handled in a different thread, things get significantly more complex. This is the case when one of the threads that attaches callbacks to signals is the GUI: we have an absolute rule that all GUI operations must be carried out in the single, main GUI thread. Consequently, if a signal is emitted by another thread, we cannot simply invoke the callback in the emitting thread: we require a mechanism to ensure that the callback is invoked in the receiving thread instead.

This document describes that mechanism, as it exists in April 2023.

Fundamental Assumptions

• We do not create threads randomly. Most threads are created during program startup. Later-created threads are rare and created one at a time for specific purposes.
• Realtime threads must be able to use signals without violating real-time safety (as far as possible given underlying OS limitations)
• Realtime threads do not emit signals very often

Conceptual Design

• A thread/event loop that is interested in the state change announced by a signal calls Signal<>::connect and passes a callback as a functor/closure
• The provied functor is rewrapped as a call to EventLoop::call_slot() and then added to list stored inside the Signal<>
• A different thread emits Signal<>, which involves walking the list of functors/callbacks, and invokes each one.
• This invokes EventLoop::call_slot() for each, which checks if the calling thread is the thread running the event loop that registered the functor/callback
• If the calling thread is the event loop, then the inner functor/callback (the one provided when connecting to the Signal<>) is invoked directly.
• If the calling thread is not the event loop, then a new "CallSlot" request is created for the target event loop, written to a buffer of such requests, and the target event loop is notified that there is a request pending.
• Receiving thread wakes up to the notification, scans for new requests and if it finds any, invokes the functor/callback stored in the request.

Involved Classes

CrossThreadChannel

This is an OS-dependent mechanism for waking up another thread. On Linux we use fifos (pipes). It is hard to avoid the possibility of blocking when writing to an OS primitive, so we just have to pick the best option an OS offers us.

The object wraps the underlying OS primitive into something that be waited on by a glib event loop.

EventLoop

An abstract class that contains the implementation for roughly half of the architecture described here. The rest is left to AbstractUI<RequestObject>.

BaseUI

Derives from EventLoop, this is convenient wrapper around a glib event loop, and provides the basic idea of a user interface thread that handles notifications from our cross-thread signalling system.

AbstractUI<RequestObject>

Derives from BaseUI. When inherited from by, for example Gtkmm2ext::UI or MidiSurface, it represents a thread running a glib event loop, where various events will drive a body of code that provides a user interface for the program (not necessarily graphical). Each derived class specifies a RequestObject type, which defines the set of requests that can be delivered to the event loop that is running the UI.

Both the GUI and all control surfaces have an IS-A relationship with AbstractUI<RequestObject>.

Request Buffers

A RequestBuffer is a single-reader, single-writer FIFO (also known as a lock-free FIFO, or ringbuffer) that holds a fixed number of the RequestObject type for that particular EventLoop.

The signal-emitting thread writes the correct values to a RequestObject in the RequestBuffer, and then signals the target thread using a CrossThreadChannel. This wakes the target thread if it is sleeping.

The signal-handling thread is woken by the CrossThreadChannel, and proceeds to check the number of readable RequestObjects in each RequestBuffer it has.

Thread/Buffer Management

Thread Registration

Every event loop object has a std::map that associates a pthread_t with a RequestBuffer. This means that at some point, the event loop object must discover the pthread_t that identifies every possible sending thread, and then create a new RequestBuffer for that thread, and create a new map entry that associates the two.

Consequently, before any thread may emit a Signal<> it must first call PBD::notify_event_loops_about_thread_creation()

This call does two things:

1. It pre-registers the thread, storing the thread name and pthread_t and a likely number of requests required for its request buffer in a static data structure.
2. A signal is emitted (ThreadCreatedWithRequestSize) which is handled by all currently existing EventLoop objects.

Existing event loops will have their EventLoop::register_thread() method invoked. This will create a new per-thread RequestBuffer for the emitting thread.

Event loops that are created in the future (after the creation of the (potentially) signal emitting thread) will scan the static data structure during their construction, and will create request buffers for everything they find there.

Request Buffer memory management

As of April 2023, the request buffers and map entries all leak, because there is no easy to enforce cleanup routine when a given thread exits. We do not regard this as a problem, because the resources involved are very small, and there are not many threads that exit before program end anyway.

We could investigate pthread_cleanup_push() as a way to notify event loops that a thread has exited, and thus cleanup the request buffers and map entries associated with it.

Thread Safety

Each event loop's map<pthread_t,RequestBuffer> is protected by a RWLock, which allows signal emitting threads (which only need to read the map) to hold concurrent read locks, and not block other reader's progress.

A write-lock is taken on the map when creating and registering RequestBuffers for threads. However, as stated in our Assumptions section above, almost all thread creation happens at a time where realtime behavior is not required. It is possible that later thread creation and RequestBuffer registration could interfere with a realtime thread, but this would require this to take place in parallel with a signal emission by a realtime thread, and these are rare (and typically connected with other user operations that would not happen in parallel with the signal. The main examples would be the PortRegisteredOrUnregistered or PortConnectedOrDisconnected signals).

The static (global) data structure that holds pre-registered thread information is protected by a static Mutex. This data structure is only written or read in non-RT sections, and so possibly blocking on the mutex is entirely acceptable.