The Ardour Canvas
The main area of the editor in Ardour is normally referred (by developers) to as "the canvas". It displays tracks and busses as arrangements of data of various kinds along a horizontal timeline. The term "canvas" comes from several GUI toolkits, which use this word to describe a GUI element on which arbitrary objects can be placed, stacked on top of each other and moved around. Another phrase for the same thing is a "scene graph".
Historical Background
Ardour2 and versions of Ardour3 up to and including 3.5.x used a canvas called "GnomeCanvas" which actually had nothing really to do with the GNOME project. It had all the features one would really want from a canvas/scene graph but all drawing was done using the CPU and the appearance of the canvas was first drawn into an RGBA buffer in memory before being rendered to some surface/element known to the native window system. The "libart_lgpl" library was used for drawing, along with the option to use direct pixel manipulations.
The performance of the canvas was not bad on Linux, but worse on OS X where the primitive used to render the RGBA buffer to the window was not very efficient. The main problem for Ardour with the GnomeCanvas was that it could not utilize graphics hardware to do drawing under any circumstances, which meant that drawing many common "modern" object renderings (notably, gradients) were not available as basic operations and relied on the CPU, even if a GPU was available that could the job faster and better.
This made it hard to really move forward with GnomeCanvas as the basis for any future changes to Ardour's appearance. We did not want to write our own primitives for gradient rendering, or for rendering rectangles with rounded corners, amongst other things. If a GPU was available, we wanted the canvas to use it when rendering. In addition, nobody was maintaining GnomeCanvas anymore. In fact that final few releases of GnomeCanvas were all based on patches that came from the Ardour project. The decision was clear: we should switch to a new canvas implementation.
The GTK project has steadfastly refused to identify any other canvas implementation as "blessed", meaning that there have been several efforts to improve on GnomeCanvas. In fact, the GNOME project used to have a useful page on this (https://wiki.gnome.org/ProjectRidley/CanvasOverview) that has now been deleted. Of the candidate canvases listed there, the only clearly appropriate for Ardour was "goocanvas". It was written in C++, retained most of the GnomeCanvas semantics, added a few useful items, and used Cairo for drawing/rendering. It seemed close to ideal.
However, a more detailed inspection of GooCanvas didn't leave us overwhelmed by the feeling that we had found the right solution for our purposes. So in about the middle of 2012, Carl Hetherington set about implementing a brand new canvas. It was almost entirely disconnected from GTK itself, used Cairo for all rendering, and was carefully designed to tackle some of the scaling issues that we were aware of (it is entirely possible to have thousands or even tens of thousands of items in the editor canvas). Carl's work is the basis of the canvas that was introduced in Ardour 3.6 although several basic aspects of its design and implementation have been changed since his first version.
Requirements
For a canvas-type GUI widget/object to be useful for Ardour, it has to have the following properties:
- No limit to the number of items present on the canvas
- A coordinate space that extends along both X and Y axes at least as far as Ardour's 64 bit timeline
- Full use of a powerful drawing API (Cairo is the reference drawing API, and given its likely adoption as part of a future C language specification, a very desirable API to have available)
- Ability to utilize GPU rendering operations when appropriate.
- Common operations must scale to conditions involving on the order of 10k items. This means that they must avoid recursion wherever possible
- Ability to independently scroll different sections of the canvas. For example, Ardour's rulers scroll left and right but not move in response to vertical scroll requests. In contrast, the main track display scrolls along both axes. At some time in the future, when track headers are a part of the canvas, the headers will scroll only up and down, not left and right.
- Ability to place items on the canvas that span independently scrolling regions (e.g. the playhead, which covers both the rulers/timebars and the tracks, even though those two areas scroll independently)
There are also several features that we do not require or use:
- Scaling and rotational transforms
- 3D
The Implementation
Concepts
Canvas
A Canvas is an object on which zero or
more Items can be drawn and displayed to the
user. The Canvas can also receive events notifying it of
interactions with the user (such as a button press, a motion
event, a key press, etc). There is no limit to the number or
type of Items that can appear on a Canvas, and the full extent
of a Canvas spans the maximal size of a double precision
floating point value. All coordinates are double precision
floating point.
The base class for the Canvas has no relationship with GTK at all. But it also is an abstract class that requires a few methods to be implemented in order to be instantiable. The GtkCanvas is a concrete class that causes the Canvas to be rendered in a GtkEventBox, and to receive events via the GtkEventBox.
The Canvas implementation also provides a utility class, GtkCanvasViewport, which packages a GtkCanvas and adds two GtkAdjustments which can be used to scroll the contents of the canvas. If scrolling is not required, then it is possible to use GtkCanvas directly.
Item
An Item is a discrete entity that can be placed on
the canvas. Items can be drawn, hidden, moved in either
direction, and raised or lowered in the stacking order on the
Canvas (sometimes called Z order or Z-axis stacking). Item
implementations are completely responsible for drawing the
Item's appearance under all conditions. Items can receive events
notifying it of interactions with the user. All Items inherit
from a simple base class (ArdourCanvas::Item). The
key method of an item implementation (the one that
differentiates it from any other type of item) is
Item::render(), a virtual method called to draw all or part of
the Item as necessary. Each Item has a pointer to a parent,
and to the top level canvas. Each Item has a position specified
in the coordinates of its parent. If this position is (0,0),
then the Item appears at the origin of its parent. If the
position is (10,10) then the Item appears 10 pixels right and 10
pixels below the origin of its parent. The Item's origin is
always (0,0) in its own coordinate space.
Note that the actual drawn area of an Item may have little to do with the position of the Item. The position may just act as the origin for the coordinate system of the Item, with rendered pixels being arbitrary distances away from this origin.
The rectangular area that the Item will draw to is given by
its bounding box, returned as
a boost::optional<Rect>. It is an optional
type because the bounding box may be undefined if (for example)
a Group has no children or a Rectangle has zero area. The
bounding box of a Group is the union of the bounding boxes of
all of its children.
Item children
All Items may have zero or more child Items added to them,
created a recursive nesting that defines the structure of a
particular Canvas. In general, Items that draw particular forms
(e.g. rectangles or waveviews) will not generally ever have
children added to them, because their drawing code ignores the
existence of children entirely. However, a Container is a
derived type of Item which does pay attention to the presence of
children when it draws itself, specifically by
calling Item::render_children().
Container
A Container is a special kind of item that draws
nothing by itself, but does draw its children. It allows the
easy relative placement of a set of Items as well as a way to
move a set of Items as a single entity. When the Container is
moved, all of its children will draw to a new position on the
Canvas.
Note that the position of a Container's children is defined by
each child's position() method. A Container does not
arrange its children in any automatic way, so it is the
responsibility of other code to place Items correctly within the
Container.
In the future there may be derived classes based on Container which do set their children's position (e.g a "Box" type which arranges them along an axis, or a "Table" type which packs its children into a tabular arrangement).
Root Group
A Canvas has a single top level Container, called "root", as its only Item. It has one special function: to track its own bounding box size and notify the canvas of changes that may affect the display of the canvas using the underlying windowing system.ScrollGroup
A ScrollGroup is a special kind of Container which
responds to instructions from the Canvas to "scroll" - that is,
to alter what is visible within the window in which the Canvas
is viewed. A ScrollGroup can respond differently to horizontal
and vertical scroll commands (it can even not respond to them at
all, if necessary). A ScrollGroup that responds to both scrolls
will move its children within the window whenever any scroll
command is received.
If you want items on the Canvas to move in response to scrolling, you need to add them to a ScrollGroup with the appropriate sensitivity to scroll commands. Items without a ScrollGroup in their ancestors will not move when the Canvas is scrolled.
ScrollGroups should occur immediately below the "root" group on a Canvas, and should not overlap. Their placement is achieved by setting their position. Nothing will prevent overlaps, but the result is ambiguity in which ScrollGroup covers a given area. They may be moved on demand just like any other Item.
Coordinate Systems
In a given Canvas, there are 3 coordinate systems in use, all
with coordinates specified as double precision floating point. A
single axis value is stored in a type called Coord,
and a pair of axes values used to define a point in a coordinate
space is stored in a type called Duple. These names
were chosen partly to avoid with the primitives found in various
native Window system APIs.
All 3 coordinate systems use the same convention as Cairo and X Window, with the origin at the upper left, increasing x-axis values indicate rightward movement along the axis, and increasing y-axis values indicate downward movement along the axis.
- Window Coordinate Space
-
(0,0) refers to the pixel at the upper left of the window in which the Canvas is displayed. Coordinates can be negative but never for drawing purposes. Coordinates can be of any positive size, but for drawing purposes should never exceed the height or width of the window in which the Canvas displayed. Thus if the window is W pixels wide and H pixels high, then (W-1, H-1) refers to the pixel at the lower right corner of the window. All coordinates should be non-fractional because they should refer to actual pixel positions in the displayed window.
Events are delivered to the Canvas in Window coordinate space.
- Canvas Coordinate Space
-
A vast coordinate space. (0,0) is the center, and it extends by approximately 1e307 in both directions along each axis.
The relationship to Window coordinate space depends on the scrolling commands that have been issued to the canvas and the arrangement of ScrollGroups within the canvas.
Canvas::window_to_canvas()andCanvas::canvas_to_window()can be used to convert between the two spaces. If the canvas has not been scrolled, then (0,0) in Window Coordinate space corresponds to (0,0) in Canvas Coordinate space.Note that if the Canvas contains more than 1 ScrollGroup, there is no single mapping between a Window Coordinate space axis value and a Canvas Coordinate Space axis value. The transform depends on precisely which ScrollGroup covers an (x,y) coordinate, which in turns defines the offsets between Window and Canvas Coordinate spaces.
Events are delivered to canvas items in Canvas Coordinate Space.
- Item Coordinate Space
-
A translated version of canvas coordinate space. (0,0)
refers to the origin of the item (even though it may never
draw anything there). (10,10) is 10 pixels right of and
below the origin.
Item::item_to_window(),Item::window_to_item(),Item::item_to_canvas()andItem::canvas_to_item()exist to convert between Item Coordinate space and the other two.
Available Item Types
Many of the items listed below are derived from one or both of
ArdourCanvas::Fill and ArdourCanvas::Outline. These base classes
provide a few methods to control how the item is drawn,
specifically how/whether it is filled and how/whether an outline
is drawn. Common methods
include Fill::set_fill_color, Fill::set_gradient, Outline::set_outline_width
and Outline::set_outline_color.
Because the canvas uses Cairo for drawing, and because single-pixel width lines are a common requirement, it may be important to read and carefully understand this Cairo FAQ answer. The short version is that you can't optimize the design of a drawing API for both trivially obvious semantics when filling and when drawing lines. Cairo's API was optimized to make fill semantics trivially obvious and line drawing slightly more complex. Perhaps you would have done it differently.
In general, the Canvas items that draw lines or outlines default to rendering single-pixel wide lines, and do so using the 0.5 coordinate offset "trick" mentioned in the Cairo FAQ cited above.
- Rectangle
- Draws a rectangle. Each edge can be specified as outlined or not. Note: the edges of a rectangle are on its boundaries, not outside them. The fill and outline colors can be specified independently.
- Arc
- Draws some part of a circle, with a specified angle. The arc can be outlined and filled, with two separately specified colors.
- Circle
- Derived from Arc, but always draws the full 360
or 2*PI radians. - Line
- Draws a straight line between two points. Width of the line can be controlled using ::set_outline_width().
- PolyLine
- Draws a series of straight line segments between any number of points. Width of the line can be controlled using ::set_outline_width().
- Curve
- Draws a smooth curve through an arbitrary series of data points. Internal implementation uses Catmull-Rom splines, with the option to use either Centripetal, Chordal or Uniform curves (Centripetal is generally the best for drawing smooth curves). This curve differs from those offered by Cairo, which are Bezier splines and are not guaranteed to avoids "knots" when interpolating between points, which is critical when representing various kinds of data in Ardour.
- Polygon
- Draws a series of straight line segments between any number of points, but always closes the path to link the first and last points. Separate outline and fill colors may be specified.
- Arrow
- Draws a straight line between two points, and offers the option to draw an arrow of various sizes and shapes at each end.
- LineSet
- Maintains information about a set of all-horizontal or all-vertical lines. Each line has its own startpoint and width, but they all have the same origin along one axis and the same extent. We use this in Ardour for the grid/measure lines, and also for the ruled background of MIDI tracks. It is significantly more efficient than using the equivalent number of Line items. Note that the drawn lines never receive Event notifications of any kind, neither does the LineSet as a whole.
- Text
- Draws a short piece of text. The text will be drawn using Pango, and can have its color, font and alignment characteristics fully specified (no outline option is available, however). The text to be drawn can also be limited by a pixel width, so that the item never displays "too much" text regardless of what text it is asked to display.
- Image
- Draws an image. The image is given to the Item as a raw data buffer with a specified width, height, stride and data format.
- Pixbuf
- Draws an image. The image is given to the Item a Gdk::Pixbuf.
- WaveView
- A specialized item for Ardour which draws a representation of an audio signal.
- XFadeCurve
- A specialized item for Ardour which draws a representation of the pair of curves that fade material in and out near the near the ends of a region.
- Ruler
- A specialized item which draws a ruler with a series of 3 levels of tick marks each with optional annotation. Methods to determine where the ticks should be placed and what annotations (if any) should be drawn near them are handed to the Ruler constructor, and called whenever the visible part of the ruler changes. A Ruler does not know about or care about the nature of the time units it is displaying.
The Coordinate Space problem
In Carl's original implementation, the Canvas Coordinate Space
and the Window Coordinate Space were unified. We took advantage
of Cairo's use of double for its own coordinate
space, assuming that to accomplish scrolling we could simply use
cairo_translate() to move objects as needed when
rendering.
Unfortunately, Cairo has a deep and longstanding bug (feature?)
which means that although it uses doubles for coordinates, the
actual size of the coordinate space it can manage correctly is
limited to roughly 32767 pixels x 32767 pixels. Any attempt to
use cairo_translate() to move outside this area, or
any use of coordinates outside of this area causes drawing
glitches and outright errors. Although the Cairo developers are
aware of this bug, no fix for it appears in sight, and so we
were forced to work around it. The Ardour canvas is required to
span a much larger coordinate space than Cairo can handle.
The adopted solution is not as clean as Carl's original equivalence relationship, but is still not too hard to comprehend. Items maintain their own coordinates using the full range of a double precision float. However, rendering is done by converting all coordinates used for drawing into window coordinate space. If the item's bounding box doesn't intersect with the area visible in the Window, then the item will not be asked to Render itself. If it is asked then the transformed coordinates passed to various Cairo calls. Since the limits on window dimensions more or less match Cairo's internal limitations, these coordinates are guaranteed to be in range for Cairo's API.
Rendering Model
The concrete base class (e.g. GtkCanvas) receives "expose" or "draw" notifications from the underlying window system, telling it that part or all of the window in which the canvas is displayed needs to be redrawn. These notifications arise from two sources:
- Window-system level changes involving the visibility of the canvas' window (e.g. inital display, becoming visible after another window is moved away from the canvas' window, etc)
- Redraw requests queued by the canvas itself in response to changes in its Items. For example, when an Item is added, moved or resized, some or all of the part of the Canvas visible in the window may need to be redrawn. Of course, the Item could be completely off-screen, in which cases such changes will not cause a redraw request.
The rest of the rendering implementation has no relationship to Gtk or the native window system, and consists of zero or more calls to the Cairo API to alter the appearance of a Cairo surface (assumed to be the window in which the Canvas is displayed). The Canvas calls ::render() on its root group, and this finds all children that are fully or partially visible within the redraw area and then (recursively) calls their ::render() method. Note that every item is passed the same original area to be redrawn as was given the Canvas itself. Each item converts its bounding box into Window Coordinate space, and then decides precisely what and how it will draw. Some items (e.g. the Image item) may simply render using a pre-drawn, memory-cached Cairo surface. Others will draw directly using some programmatic logic and common Cairo drawing operations (for example cairo_rectangle(), cairo_line_to(), cairo_arc(), cairo_stroke(), cairo_fill()).
Event Propagation
The concrete canvas object (e.g. GtkCanvas) receives events from the native window system (e.g. X Window on Linux, Quartz on OS X) via the GTK toolkit. The following event types are explicitly handled by the canvas:
- draw/expose (described above)
- button press and release
- motion
- enter and leave
Current Item
By default, button press and motion events will be delivered to a designated item termed the "current item". If a drag is in process, there is an alternate item, the "grab item" to which events will be delivered. Maintaining the correct current item for event delivery is one of the central tasks of the canvas implementation.
This is implemented in Canvas::pick_current_item (Duple const&
point, int state). The design of the canvas allows for the use
of "smart" methods of determining which items cover a given
point, but at this time (June 2014) we use a naive, O(N) linear
search through each group. This is done via the LookupTable
object table associated with each Item. The code contains an
attempted "smart" but unfinished and untested implementation
(OptimizingLookupTable). For now, DumbLookupTable is used to
find items covering a point, and that just iterates over a list
of children and calls Item::covers() on each one.
Enter/Leave
For every enter/leave event and for every motion event, as well as whenever there are any changes to an existing Item or when an Item is added or removed from the canvas, the canvas takes the current pointer position and determines what item covering that position should be considered the "current item". Recall that there may be many items covering the pointer position - the canvas will use the upper-most item as "current item".
Whenever the "current item" changes, a series of events must be sent to various items in the canvas to notify them of the change so that they may, if necessary, modify their appearance. The existing current item is sent a "leave" event, with the "detail" field of the event set to indicate whether the new current item is a child, parent or unrelated item. Then the parent chain for the old current item is traversed, with each of them being sent a leave event (again with the "detail" field set appropriately).
After the leave events have been sent, the parent chain for the new current item are sent an enter event (with the "detail" field correctly set), and finally the new current item is sent a enter event also.
Once this set of events has been delivered, all Items have been appropriately notified of a change in the current item, and "current item" is finally reset. Subsequent button and motion events will be delivered to that item (unless a grab is in effect).
Note that enter/leave events do not cascade to parents as described in the next section. They are explicitly delivered by the canvas, without regard for whether or not a particular item returns true or false to indicate that it handled the event.
Parent/Child chain
Each canvas item has a member:
sigc::signal<bool,GdkEvent*> Event; to which
arbitrary functors can be attached as handlers. Each handler
returns a boolean value to indicate whether or not it handled
the signal/event or not. This signal will be "emitted" as part
of the event delivery mechanism for that event.
If an event is delivered to an Item, zero or more handlers for
the event will be invoked. If any of them return true, the event
is considered to have been handled, and no further processing
will be done relating directly to the event. Note that as in
GTK, the connection order of the handlers can have important
implications. If there are two functors/handlers connected to an
item's Event signal, and the first one always returns true, then
the second one will never be
invoked. sigc::signal::connect() has arguments to
control the ordering of handlers as they are connected.
If there are no handlers, or none of the handlers returns true, then the event is considered unhandled, and is redelivered to the item's parent. This cascade-to-parent continues until the event is delivered to the root group. If it is still unhandled at that point, we notify GTK that the canvas has not handled the event, allowing it to continue with its own propagation strategy (e.g. to the top-level window in which the canvas occurs).
We use sigc::signal rather
than PBD::Signal here because the signal is
connected to and delivered ("emitted") in a GUI context that is
always serialized and always single threaded. PBD::Signal
(unlike sigc::signal) is thread-safe, but we don't need those
semantics for GUI-related signals.
How Scrolling Works
Canvas::scroll_to (Duple const& point) is invoked to tell
the canvas to redisplay its contents so that where appropriate,
they are offset by point.x and point.y
along each axis.
However, as noted above, the Ardour canvas contains various conceptual (and actual) groups of items which need to respond differently to scroll commands. Rulers scroll left/right only; track headers scroll up/down only; main track displays scroll left/right and up/down.
ScrollGroups are the currently implemented solution for this
problem. When a ScrollGroup is created, its constructor requires
a specifier of which axes (X, Y, X&Y or none) it will
scroll. When Canvas::scroll_to is called, the
canvas iterates over all children of its "root" group. Each
ScrollGroup that it finds at this level has
its ScrollGroup::scroll_to method invoked. This
alters the _scroll_offset member of the
ScrollGroup.
As described above, when items are rendered they first convert
from their own coordinate space to the window space. Every item
has a pointer to its "scroll parent" (a Scrollgroup; the
ScrollGroup may actually be several "generations" from the Item,
but every item has only 1 scroll parent). The
implementation of Item::item_to_window uses the
ScrollGroup parent's scroll_offset to determine
what (if any) offset in effect due to scrolling, and
incorporates this into the returned value.
As a concrete example: an Item has a scroll parent which
responds to both vertical and horizontal scrolling. The canvas
has been scrolled by (-100, 100). The Item sits at (40,40) in
Canvas Coordinate space. It calls item_to_window on
a point (x,y) within its coordinate space. It first converts to
canvas coordinate space, generating (x+40,y+40). Then it applies
the scroll offset of its scroll parent (-100,100) to give
(x+40-100,y+40+100). The item coordinate (x,y)
is thus (x-60,y+140) in window coordinates.
A similar item with a scroll parent that responds only to horizontal scrolling would do the same calculation and end up with (x-60,y+40). The two items would thus appear at different positions along the y axis, as appropriate given their scroll parents' sensitivity to scrolling.
Items that are have no scroll parent will never scroll. Although it is possible to also use a scroll group with no scroll sensitivity, scroll groups may not overlap, so this no-scroll-parent => no-scroll-behaviour feature has some potential use for global canvas items that should remain in the window at all times but that span different sections of the canvas.
Note that because items have direct pointers to their scroll parent, this design adds a very lightweight O(1) operation to compute the effect of scroll on the item_to_window/window_to_item transforms. We do not have to traverse parent/child chains, which would make the computation O(N) where N represents the depth of a given item from the canvas root.