Root cause of the reported 'timestamps off by different amounts,
sometimes early, sometimes late' bug: _extract_audio_wav() never
passed an explicit -map to ffmpeg, so on multi-audio-track files it
relied on ffmpeg's own default-stream heuristic -- which has no
knowledge of, and isn't guaranteed to agree with, whichever audio
track mpv is actually configured to play. A wrong-track clap is
essentially unrelated to the true one (not a small timestamp-math
error), which explains the inconsistent, file-dependent symptom --
and our single-audio-track synthetic fixtures never exercised this
path at all, since they only ever had one track to pick.
Fixed by threading an explicit ffmpeg stream index through
detect_claps() -> _extract_audio_wav(), sourced in main.py from mpv's
own track-list (the ff-index of whichever audio track has
selected=True). Detection now always analyzes the exact track the
user is listening to, on both 'Detect All' and 'Detect Next'.
Also: _probe_audio_start_time() used to silently fall back to 0.0 if
ffprobe couldn't parse start_time as a plain float (e.g. N/A) -- same
failure class (silently wrong instead of loud). Now emits a Python
warning instead, so a genuinely unparseable/unusual container is at
least visible rather than silently mis-corrected.
Verified two ways: (1) ruled out an initial alternate hypothesis
(video stream's own start_time mattering) empirically against the
existing single-track fixtures before ruling it out. (2) Built a new
two-audio-track fixture (sync_test_MULTITRACK.mp4: track 1 has a beep
at 3s, track 2 at 7s) and confirmed end-to-end through the real app --
switching mpv's active track from aid=1 to aid=2 correctly flips which
clap gets detected, which is exactly the mechanism this bug required.
Ports scripts/detect_claps.py (a librosa-based onset detector from an
earlier session) into clap_detection.py, shared by both the CLI script
and a new GUI feature: 'Detect All Claps' (whole-file scan) and
'Detect Next Clap' (from cursor, first result only), with results
shown as clickable markers on a new MarkerStrip widget above the seek
slider -- clicking one seeks the player to that exact timestamp.
Caught a real correctness bug while testing against the out-of-sync
fixture (not just the plain in-sync one): ffmpeg's default -vn audio
extraction silently drops the audio stream's own start_time and
renormalizes to t=0 -- which is exactly the offset this whole tool
exists to fix, so losing it meant detected timestamps pointed at the
wrong track's timing entirely. Fixed by probing the true start_time
via ffprobe and re-inserting it as explicit silence (adelay) before
analysis. Also dropped an earlier attempt at an ffmpeg -ss seek
optimization for the from-cursor case, since combining it with the
start_time fix meant two independent timestamp-origin corrections
stacked on each other -- correct in theory, too easy to get subtly
wrong in a tool where a silent wrong-timestamp bug is the worst
possible failure mode. Full-file decode + filter in Python is slower
but unambiguous.
Verified with a smoke test that drives the real MainWindow buttons,
confirms the detected marker lands near the fixture's known beep
(t=5.350s, allowing for onset-detection lag), and simulates an actual
QMouseEvent click on the marker to confirm click-to-seek works
end-to-end.
ThePendulum