Fix clap detection silently closing mid-recording audio gaps

Real bug found on the user's actual file (a ~110min recorded capture),
not reproducible with any of our synthetic fixtures: plain ffmpeg -vn
extraction just concatenates decoded audio packets and ignores their
PTS, so a genuine multi-second discontinuity mid-recording (e.g. a
brief audio-device dropout during a long capture session -- common in
OBS-style long sessions, unrelated to A/V sync) gets silently closed
up instead of preserved as a gap. Every onset detected after that
point then gets reported however much earlier the gap was, while
onsets before it stay correct -- exactly matching the reported
symptom: dead-on for some files, off by several seconds on others,
with the 'sort of sound' being irrelevant since it's not about
detection quality at all.

Diagnosed by scanning the real file's audio packet timeline for
non-uniform PTS spacing: found exactly one jump, +1.621s at ~928s in,
matching (almost to the millisecond) the 1.6s gap between the source's
true duration and what plain -vn extraction produced. Fixed by adding
aresample=async=1 to the extraction filter chain, which conforms
decoded audio to its own PTS and fills gaps with silence instead of
dropping them -- confirmed the extracted WAV's duration now matches
the source exactly.

This is a different failure mode from the leading start_time offset
already handled (aresample=async=1 alone does NOT fix that case -- it
re-zeros to its own first frame same as plain extraction, confirmed
by testing on the existing OUTOFSYNC fixture), so both fixes now run
together in the same filter chain. Verified no regression on every
existing single-track, leading-offset, and multi-track fixture.
This commit is contained in:
ThePendulum
2026-07-26 05:28:17 +02:00
parent 76b5f827be
commit 97cbf38d97
+18 -1
View File
@@ -77,9 +77,26 @@ def _extract_audio_wav(input_path: str, audio_stream_index: Optional[int] = None
os.close(fd)
cmd = ["ffmpeg", "-y", "-i", input_path, "-vn"]
cmd += ["-map", f"0:{audio_stream_index}" if audio_stream_index is not None else "a:0"]
# aresample=async=1 keeps the decoded audio conformed to its own
# internal PTS, filling any gap with silence instead of silently
# closing it up. Long recordings (stream captures, OBS-style long
# sessions) can have a genuine multi-second discontinuity mid-file --
# e.g. a brief audio-device dropout -- from a real capture glitch, not
# anything to do with sync between tracks. Plain -vn extraction just
# concatenates packets and ignores the gap, silently shortening the
# output and shifting every onset after that point earlier by however
# long the gap was. Confirmed on a real ~110min capture with exactly
# one such gap (+1.621s at ~15.5min in): without this filter the
# extracted WAV came out 1.6s short of the source's true duration;
# with it, the duration matches exactly. This is a different failure
# mode from the leading start_time offset below (which aresample alone
# does NOT fix -- it re-zeros to its own first frame, same as plain
# extraction), so both are needed together, not either/or.
af_chain = ["aresample=async=1"]
if audio_start > 0.001:
delay_ms = int(round(audio_start * 1000))
cmd += ["-af", f"adelay={delay_ms}|{delay_ms}"]
af_chain.append(f"adelay={delay_ms}|{delay_ms}")
cmd += ["-af", ",".join(af_chain)]
cmd += ["-acodec", "pcm_s16le", "-ar", "22050", "-ac", "1", wav_path]
result = subprocess.run(cmd, capture_output=True, text=True)