Latency compensation: how is output latency accounted for in Ardour?

Installation & Configuration Linux
#1

I’ve been spending quite a bit of time testing how Ardour and JACK behave with an analog loopback cable, to find out how to set things up for perfectly synchronised multi-track recording. There’s a bit of background stuff at the following thread too:
http://ardour.org/node/2929

Having looked at the JACK source and adjusted a few things to work as I think they should with Ardour, my main question is: how exactly does Ardour apply latency compensation? AFAICT, it bases its adjustments on the per-port latencies reported by jackd, and obviously it treats software sources received via JACK differently from outside-the-box audio (reflected in the “Align with capture time” / “Align with existing material” track menu items).

So, for input port latency, the material being recorded is offset by the port’s latency so that it’s correctly aligned in time when played back, correct? And for output latency, Ardour would start playing regions early, so that when you hear the sound it’s correctly aligned in time, and if you play in sync with it, anything you record as you play along will also be correctly aligned. However, I’ve found that overdubs come in quite late, and it gets progressively worse as the total JACK buffer size is increased. To be precise, the overdubs are p x (n - 1) frames late, where p = period size in frames and n = number of periods per buffer. This formula holds for all sampling rates, period sizes and nperiods that I’ve tried.

From what I understand of the architecture of JACK and Ardour, I don’t believe this should be happening. Also, I’ve noticed that setting additional output latency in JACK apparently has no effect on the alignment of new regions being recorded - only the extra input latency changes things. So, does Ardour actually use the output latency to adjust the alignment of playback of existing tracks?

I should explain my JACK tweaks as well. For the common case of recording a new track along with an existing one (or, equivalently, when doing an analog loopback test), I believe the combined latency of the playback-record loop would be p x (n + 1) frames (plus some extra latency due to hardware, speed of signal in wire/air, and other things outside the box). That’s p x n for the output latency, plus another p for the input latency, giving p x (n + 1). I’m finding that I can only get perfect synchronisation of the loopback signal if I modify JACK (0.116.2 SVN 3649) so that the input latency is n x p, rather than just p. (This is in drivers/alsa/alsa_driver.c, BTW, where jack_port_set_latency is called for the capture and playback ports.) In the original JACK code, the output latency is essentially calculated by p x (n - 1). I had thought this should be p x n, from reading Paul’s slides on JACK’s design. However, changing the output latency logic seems to have no effect on the Ardour loopback test, only on the output port latencies reported by jack_lsp.

I can understand that accounting for the output latency as part of the input latency isn’t appropriate, as there could be other routes from software out into the real world with different latencies (MIDI being a good example). But overdubbing new tracks onto existing ones seems like such a common scenario that it should probably work out of the box, IMO.

BTW, Looking at the code, I’m glad to see that JACK also can account for additional hardware latency reported by the ALSA driver (driver->capture_frame_latency). Is this widely supported? I don’t think the driver for my Echo Gina24 implements this, as I always see about 135 frames of extra latency, regardless of the sampling rate, period size or number of periods in JACK.

Sorry for the long and technical posting - I hope I’ve made my confusion clear, if you know what I mean. :slight_smile: I at least have a workaround now, but I don’t believe it’s the correct way to do it.

Regards,
Chris

#2

When I´m doing overdubs, I setup jack to a decent buffer size and period setting, and then measure a loopback with Jdelay. This is the value (divided by 2) I´m using for jacks -I and -O option (input/output latency). To see if this is correct, I just record the metronome with a microphone, and zoom in to check if the clicks are above eachother… Simple and effective:)

Roald

#3

Hi Roald,

I’ve been using jdelay/jack_delay to help diagnose what’s going on too - very handy utilities you’ve written, thank you, Fons, if you’re listening! :slight_smile: Interestingly, Ardour does seem to compensate for 1 period (presumably due to input latency) compared with what jack_delay reports, but the problem is it isn’t compensating for enough periods - there’s another nperiods-1 of them due to the output latency.

I’d prefer not to have to fudge delays that are essentially due to internal audio buffering into the -I and -O latency settings - IIUC, those are intended to compensate for external delays that JACK has no way of knowing about. I have used -I in that way in the past out of desperation, but the problem is that the correct value to use for -I changes depending on the period size and number of periods (though thankfully not the sampling rate). I do think this is something that should basically Just Work, given the thought that’s gone into the excellent design of JACK and Ardour.

I previously was able to do as you’re doing - measure the latency and divide it by 2 - but I think something has changed in the past couple of years and that no longer seems to work across different buffer settings…

I should also reiterate that the multitrack latencies I was seeing using JACK2 (1.9.2) were different. I’m pretty sure that shouldn’t happen, if they’re based on the same specification.

#4

Ah-ha, so I think my modifications to jackd are basically just to compensate for the fact that the output latency doesn’t seem to be compensated for by Ardour (2.8.3 anyway) on playback. The existing jackd code uses p x (n - 1) as the latency of output ports, which is the exact amount by which the loopback recordings/overdubs are lagging in my testing. I’ve noticed Ardour also seems a bit erratic in when it applies the Artificial Latency plugin (it always seems to take at least one play-and-stop before the latency adjustment takes effect).

I’ve been all over the place trying to figure this one out - I’d even wondered if it was an ALSA problem - but now I’m thinking it looks like Ardour is not adjusting for output port latency on playback. Can someone confirm this? Am I just overlooking some option to turn it on in the GUI? :wink:

Thanks,
Chris

#5

Edited for clarity, 2009-10-19 21:17 NZDT

Speaking of the Artificial Latency plugin, I’ve also observed some strange behaviour when this is enabled in my loopback test session in Ardour.

In the session, I have a track that plays clicks (on each beat). The output from this track goes to my sound interface, where I have a cable patched back into an input on the same sound interface. A second track in Ardour then records this incoming loopback signal.

Now, if I add another track, add the Artificial Latency plugin to it, and set the latency to a non-zero value, and then run the loopback test again (still recording to the second track), the newly recorded loopback track is further delayed, by the amount set in the plugin. Note that the plugin is on a completely separate track from the playback and record ones involved in the loopback test. Shouldn’t the latency plugin only affect the track that it’s on, by effectively moving it earlier in time, relative to the rest of the session, by the specified amount?

What’s more, if I put the latency plugin on the track providing the loopback test signal, adjusting the plugin latency has no effect on the delay observed on the signal returned via the loopback! Have I completely misunderstood how the Artificial Latency plugin is supposed to work? Could it be related to the lack of output latency compensation I observed above? If someone has a clue-stick on them, hit me. :slight_smile:

#6

Maybe its a good idea to take this to the mailing list or IRC, I think you´ll get more response from people with know-how. I´d love to do some tests like you with a loopback cable, but my presonus firepod let me down (AC connector broke:( )

Roald

#7

@screwtop:

  1. -I and -O are critical in order to provide JACK with information that is never ever available from any other part of the audio API stack. No device driver can tell JACK what the delay caused by its D/A converters, or its A/D converters, or any other part of the signal chain. Thus, if you want accurate results, you must (a) use jdelay or something similar to measure the delays (b) use -I and/or -O to let JACK know the values.

  2. the artificial latency plugin does NOT actually delay any signal by its claimed latency. it simply reports a fictitious latency value. its original use was anticipated being as a way to correct for other plugins that did the opposite - reported zero latency by actually did delay the signal.

the rest of your observations/questions will need to wait till i’m back in the right frame of mind.

#8

Thank you, Paul.

Responding to 1), I guess I misunderstood the driver->capture/playback_frame_latency in JACK’s ALSA driver code. I assumed that the extra latency being added there was something reported by the ALSA driver for extra hardware latency like ADC/DAC. Still, that would be a nice capability, I think, as it would mean that jackd’s -I and -O would truly only be necessary to compensate for un-knowable outside-the-box latencies. I guess that’s one for the ALSA crew… :wink: Also, I guess ideally -I and -O would be able to be set per-port rather than globally…

The problem I’m describing is not that the loopback shows a delay per se, it’s that the delays are much bigger than they should be: a number of periods, in fact. Across the board, I’m observing p x (n - 1) + 135 frames delay in the recorded loopback signal. I understand that I would need to distribute the 135 frames across the -I and -O arguments in order to get perfect sync, as that’s obviously intrinsic to the sound hardware; it’s the remaining p x (n - 1) frames that are bothering me, as that’s apparently due to buffering within JACK/ALSA that Ardour doesn’t seem to be compensating for.

Re. 2): good - sounds like my understanding is correct: it’s effectively a “negative delay” effect. For example, if a recorded waveform appears 10 ms late on the track display, you could add the latency plugin with 10 ms to bring it into sync with the rest of the session, correct? I’m not sure that that’s actually happening properly on my system, but maybe my testing method is flawed. :slight_smile: Will do more testing soon…

Thanks for looking over my ramblings!

#9

@screwtop: information about converter and other circuit-based delay is almost NEVER available to a device driver. Its not there for CoreAudio, ALSA or WDM/WaveRT drivers. The driver simply has no place to get the information from. Contemporary h/w simply doesn’t report it.

#10

@paul:

Sure, that information may not be something that the driver can probe for, but it should generally be available to the device driver developer, either from spec sheets (if you’re lucky), asking the hardware engineers, or doing the measurements on the hardware itself with an oscilloscope or whatever. The intrinsic input and output latency of the converters would just be another hard-coded property in the driver, like the device name or the maximum sampling rate.

I could imagine it getting more complicated if the hardware latency varied with sampling rate or other factors. Hardware with some analog and some digital I/O channels (e.g. ADAT) would likely have different intrinsic I/O latencies depending on the channel, and any outboard ADAT stuff would of course introduce further latency that couldn’t be known. I still think it’s an interesting idea, though. :slight_smile:

#11

The latency does vary with SR, and it can also vary with the specific connector path used.

#12

Sorry, I should be clearer in what I mean by “latency”. On my hardware, I see a constant 135 frames of I + O latency due to my hardware, which doesn’t change with SR, in the sense that it’s always 135 frames. The latency measured as a time interval of course would change with SR. Sorry about that! I assume that’s why jackd’s -I and -O are specified in frames rather than ms. It was the latency in frames that I was imagining could be stored in the driver code, since that appears to be constant (at least in my experience).

My big question remains: is Ardour supposed to compensate for the output latency reported by JACK when playing back? And why does jackd’s -O setting apparently have no effect on the loopback test? Is the extra p x (n - 1) frames of delay that I’m observing perhaps due to additional buffering going on somewhere deeper?

Thanks for your time in considering this: I’ll definitely be leaving some money in the tip jar. :slight_smile:

#13

Oh, BTW, it looks like ALSA’s snd_pcm_hw_params::fifo_size may be close to what I’m talking about (IIUC, it would at least account for the hardware buffering latency, which would probably be the bulk of it). But apparently it’s set by hardly any drivers, and AFAICT not really used by any software. Oh, well… :slight_smile:

#14

Update: I’m wrong in this posting in saying that the input latency compensation is being correctly applied. I’d noticed that Ardour appeared to be applying some input latency compensation, but it was actually not the right amount.

OK, I’ve been doing some sanity testing with very large -I and -O settings in jackd. -I and -O are being correctly honoured by Ardour and JACK - sorry for doubting! But somehow there’s still consistently another (n - 1) x p frames of latency being introduced into the analog loopback. I’m really starting to wonder if it’s an ALSA problem. I’ve been careful to use hw:0 for all my testing to try to keep any extraneous software layers out of it.

BTW, the reason I thought the output latency wasn’t being applied was because Ardour’s click track is (of course) played with latency compensation. If the click track is routed to a hardware output port, the click track’s playback will inherit the latency of that port. Thus, Ardour will start playing the click track early to compensate. If I increase -O, Ardour correctly compensates by starting the click track earlier by the same amount before the transport appears to start rolling, cancelling out the extra output latency. As they say, my bad! I have to say, having all the port latency stuff co-ordinated centrally by JACK is just excellent.

One other question: some AD/DA hardware inverts the polarity of the signal (this shows up on jack_delay test as Inv, right?). It would be nice to compensate for this as well - does JACK have an inverting/non-inverting property among its port information to cover this eventuality?

#15

I’m now pretty certain the problems with overdub/loopback synchronisation are in fact to do with the input buffering. The modification I made in JACK’s alsa_driver.c, setting the input port latency to n x p + I instead of just p + I, means that I now get a consistent, small delay in the loopback test, which isn’t affected by period size or number of periods. I can now tell jack to use (for my hardware) 135 frames of input latency, and the analog loopback shows perfect sync.

FYI, with this change, and p = 2048, n = 2 buffering, jack_lsp -l reports 4231 frames latency for the input ports, and 2183 frames for the output ports. jack_delay now reports 4231.482 frames (44.078 ms) delay for the loopback. I’m glad they now seem to agree on something. :slight_smile:

Now, how do I work out how to measure -O accurately, independently of -I…? :wink:

#16

one problem - that isn’t the input port latency :frowning: a sample that arrives at the external connector at time T becomes available to a JACK client at time T + I + ((n_periods-1) * period_frames) later. so even if this fixes the issue you’ve seen, its not actually a correct fix.

#17

oh, regarding polarity: right click on every track/bus name at the top of the mixer strip. there is a “polarity” entry there. JACK can’t/won’t do this automatically because JACK doesn’t measure polarity without user intervention and adding an API to individually specify this for every h/w port seems … baroque in the extreme.

#18

OK, I think I’ve misinterpreted some things from your LAD 2003 slides. So if sound leaves a JACK client at time T, what would be the formula for when it would reach the external output connector?

Thanks for the polarity pointer, too. “Baroque in the extreme” - thanks for putting it politely. ;D

#19

Edited for clarity, 2009-10-22 20:28 NZDT

I feel like I’m either being stupid or stubborn here (perhaps both!), but I’ll try to explain what I’m seeing in Ardour, with JACK modified to artificially report no input latency and a fixed 2 s of output latency.

jack_lsp -l reports:

...
system:capture_2
	total latency = 0 frames
system:playback_1
	total latency = 192000 frames
...

So far, so good.

Observations from analog loopback test, Fs = 96 kHz:

  1. Ardour starts playing the existing material 2 s early (as expected).
  2. The transport appears to start rolling (i.e. the playhead begins moving) 2 s after clicking Play. (correct; side-effect of 1)
  3. The capture track begins recording immediately, i.e. before the playhead starts moving, starting from the original playhead position. I think this is the problem. The recording position (i.e. the right edge of the growing region being recorded) is therefore 2 s ahead of the playhead on the timeline. Shouldn't it actually be recording at the playhead, if the input latency is 0?
  4. Even though JACK reported 2 s of output latency, a massive and bogus value, and Ardour did start playback early, the recorded loopback track is still aligned with the source track on the timeline!

Please tell me I’m not crazy! :slight_smile:

#20

I can confirm that this behaviour is not seen in Ardour 2.1 (2496), checking on the installation I have on another computer, using large values of -O and -I again to make it clear what’s going on. In 2.1, the playhead actually jumps back visibly by O when it starts playing, but otherwise the output latency works the same as in 2.8.3. The difference is that in 2.1, newly recorded material is laid down I frames behind the playhead, as it should be. In 2.8.3, it’s O frames ahead of where it should be, i.e. O frames ahead of I frames behind the playhead (for example, with zero input latency, it’s actually ahead of the playhead, by O frames).

OK, I’m pretty certain this is a bug. OK if I file a report?