A sparse video stream can report a transient EOF during a restart. The
DVD-Audio still track is read lazily and its first frame may arrive a
moment after the video chain already drained. Treat a frame arriving
while the restart is in progress as the restart's first frame.
Sparse video with continuous audio broke playback timing in several
ways:
- The first frame after a seek/restart was always shown immediately.
With sparse video the frame at the seek target may not exist, and the
next still can lie minutes ahead. Showing it early displays the wrong
image.
- hr-seeks dropped frames with PTS below the target. For sparse video
the nearest preceding still is the correct frame to show.
- Frame scheduling used the incremental A/V delay accounting, which is
unusable across the video-EOF gaps between stills. Schedule sparse
frames directly against the audio clock instead.
The old code had several interacting problems that prevented A/V from
reaching a steady state.
When the A/V difference was large, it applied the maximum allowed
compensation in one step. That was fine for catching up, but the value
was never re-evaluated, so audio speed routinely overshot the target.
Once the difference crossed zero (or re-entered the acceptable range) it
would run a linear regression over recent av_diff samples and snap the
audio speed to the estimated drift. The fatal flaw was that those
samples already included the controller's own compensation: even when
the underlying drift was zero, the controller would "detect" the drift
it had just induced and try to cancel it. Between these rare, sparse
updates audio speed stayed frozen, so in practice the controller was
introducing drift more than it was fixing it. On high fps outputs,
where the vsync range is tight, this degenerated into a ping-pong:
overshoot, snap back, overshoot again, never settling.
The new code replaces the bang-bang direction controller and one-shot
regression with a proportional controller driven by a low-pass-filtered
av_diff. The LP filter (1 s time constant) rejects per-frame vsync
jitter so the compensation no longer chases noise. The target then
decays av_diff exponentially toward zero with a 10 s recovery time.
Adaptive slew caps per-frame motion to the desired correction magnitude,
so large excursions are absorbed quickly while near-equilibrium motion
stays below audibility. The result is a continuous, gentle control loop
that actually reaches steady state and does so without audible
audio-speed modulation.
Previously this would be called on every frame presented, this only
needs to be called if display sync was active for the last frame but
isn't active anymore.
This hid the bug mentioned in #16947 if VO was active.
0ab3184526 introduced most of the stuff
being removed here. The idea is that when doing encoding the core should
wait until everything is initialized. This is true, but the old
implementation is overly complicated and confusing. What we need is for
both the AO to be initialized and the VO to be reconfigured for encoding
to start. Because of the way initialization works, the VO reconfig
always happens first. Changing the order in playloop would of course
break this assumption, but we can trust that nobody would arbitrarily do
that. So all that needs to be done here is that write_video should wait
until the AO is up before pushing frames. And that only applies in
encoding mode and if there is a selected audio stream in the first
place. This is a simple check.
With that in place, all of this extra junk can be dropped. We don't need
special booleans or events for the simple task of making sure the AO and
VO are both up.
There's a weird mix and match of ways to set special things about a VO
so let's consolidate it all to use VO_CAP bits. More importantly this
fixes do_redraw not being properly ignored/skipped in the untimed VO
case. This was intended in 8657b8e1f3 but
later the code got reorganized/moved around to make it no longer a no-op
but we just got lucky and there were not any known problems with the
broken logic until 5338f4ba70.
Need to shuffle some stuff around but it's pretty simple. Mute the
audio stream before the frame step and then unmute it when we're done.
It's exactly the same as the default play mode otherwise.
Users basically never need --hwdec=auto in reality and sometimes it
bites them unexpectedly (e.g. #15814). Unfortunately the name is kind of
nice sounding and if you don't read the manual closely, you might
inadvertently end up using it. So just make auto the same as auto-safe.
auto-unsafe is introduced to have the old behavior of auto instead. A
similar thing is done with the -copy variants. Try to update the
documentation of this section a little bit and discourage people from
using auto-unsafe because it's kind of dumb.
The previous commit put all the pieces in place so now this can be
implemented. Scheduling frames is already written with the assumption
that display sync maybe turn on/off at any time. So all that needs to be
done is check if the VO reports that it is not visible. If so, simply
flip mpctx->display_sync_active to false for that frame and skip the
display sync frame handling. It will become true again when the mpv
window comes back into view.
Waiting for audio_started to be set to true takes too long which causes
us to miss it for the first frame, instead invert the condition so it's
set on the first frame.
Fixes#14615
Effectively reverts 7051e94e4b. There's
been some confusion with how audio pts gets used internally in mpv which
leads to weird results. The crux of the problem is essentially that
playing_audio_pts can return negative numbers and in many places this is
not expected. This is the first step in trying to hopefully iron out all
the weird corner cases.
The logic in question was added in 201bef7ee1
for the VDPAU vsync frame timing algorithm, but after the code was moved
around for several times, it is now used for all VOs with non-display-sync
mode (where the video is synced to audio or system time). It nonetheless
likely never did whatever it was intended for.
This "correction" reduced the VO frame duration by the amount that the
frame is fallen behind the ideal time. Since a frame is presented between
pts (the ideal time for which the frame is scheduled) and pts + duration
(the end time for which frame drop is determined), and pts is already
computed from the current time and the deviation from the ideal time, this
"correction" causes the end time to have the deviation added twice, which
is nonsense: if the deviation is -0.5x the frame duration, the frame is
dropped, even though it should be displayed from now to 0.5x the frame
duration from now.
It was not noticed at that time probably because the VDPAU secret rabbit
code undid some of its damage, and the subsequent development focus on
display-sync modes resulted in negligence and simplification of audio mode
(e.g. b8bcf0f466), but the generic VO frame
drop algorithm has been observed to cause inconsistent frame drops with
this "correction": playing a 59.94 Hz video at 2x speed on a 60 Hz display
results in spending over half of the time dropping adjacent frames instead
of every other frames, visually causing stuttering, while it should only
happen briefly when the pts is very close to vsync time.
Fix this by deleting this logic, making the VO frame always having the
duration of the video frame.
Fixes: 201bef7ee1
Ended up being too flawed and caused trouble in other areas. There's
other approaches to trying to solve the issue this meant to address in
the works that should be better, so let's wait on that. Fixes#13613 and
fixes#13622.
This reverts commit e3af545421.
This seems more robust than relying on the audio status to actually be
playing. For files where there is no audio or the audio start is
delayed, this guards against that but it allows the subtraction to
always occur otherwise on normal files with audio.
Playback speed changes should be treated as a discontinuity just like
seeking. Previously, this was being treated internally as just plain
normal playback, but that can't really work. The frame timings from
before the speed change and after the speed change are completely
different and shouldn't be compared to each other. This lead to frames
being adjusted to weird places and possibly even being skipped (as if
mpv was seeking) on speed changes. What we should do is clear out and
reset all av related fields like what happens when you seek, but it is
not quite as aggressive. No need to do a full video state reset or such.
We also wait an arbitrary amount of frames before adjusting for av sync
again. compute_audio_drift already used a magic number of 10 which
sounds reasonable enough so define that and use it here. Fixes#13513.
When calculating the audio pts, mpv multiplies the ao delay by the
current audio speed and subtracts it from the written audio pts. This
doesn't really make sense though. mpctx->video_pts is never affected by
the playback speed, and this leads to weird behavior like the audio-pts
property changing values while paused merely because the playback speed
changes. Remove the multiplication and simply subtract the delay by a
factor of 1 instead. When updating the av_diff in player/video, this
does actually need to take in account the audio speed so we do the
calculation there.
32-bit signed integer can hold ~2.1s stored as nanoseconds. While frame
duration doesn't make sense to be this long, the existing clamp is to
10s. Change type to double, which is consistent with other fields in
vo_frame.
This workarounds the issue with vo_gpu that would freeze in certain
condition. I haven't go deep inside to understand why, so this should be
considered as a workaround.
Fixes: #13256
This reverts commit cb2b579f61.
This breaks files where the audio starts much later after the video
since mpv reads the first audio packet even if it isn't supposed to
start yet, resulting in the audio_status being STATUS_READY for the
entire time mpv is "waiting" for the first audio packet to be played.
During move of this code from vo_gpu_next.c to video.c someone(TM) tried
to be smart and simplify the expression. The num_vsync includes error
compensation which can cause it to display +-1 vsync at the same rate.
We explicitly don't want to include this in "ideal" parameters.
Also num_vsyncs was already rounded so we produced off timings in
general.
Revert back to proper way of translating the time.
Fixes: 5e5a325
Relative to frame PTS timeline as oposed to display vblank.
Those values are relative to unadjusted video timeline. They will be
used by gpu-next where it expect virtual frame vsync, not display vblank
time.
Currently VOCTRL are completely unusable for frequent data query. Since
the HDR parameter addition to video-params, the parameters can change
each frame. In which case observe on those parameter would be triggered
constantly. The problem is that quering those parameters involves VOCTRL
which in turn involves whole render cycle of delay.
Instead update VO params on each draw_frame. This requires changes to VO
reconfiguration condition, but in practice it should only be triggered
when image size or data layout changes. In other cases it will be
handled internal by VO driver.
I'm not quite happy with this solution, but don't see better one without
changing observe/notify logic significantly. There is no good way
currently to handle VOCTRL that are constantly queried.
This adds unfortunate synchronization of player command with VO thread,
but there is not way around that and if too frequent queries of this
param becomes a problem we can thing of other solutions.
Changes the way to get data from VO driver added by a98c5328dc
Fixes: 84de84bFixes: #12825
This would cause mpv to, in some very specific scenarios, have a
negative vsync_offset after seeking which would result in mpv requesting
a pts before the first frame to libplacebo.
Fix it by setting it to 0 when we reset state, such as after seeking.
Fixes: https://github.com/mpv-player/mpv/issues/12813
Previously, the av sync change calculation was only done if the
audio_status was STATUS_PLAYING, but there is at least one or two more
states where this should be done. player/audio is capable of adding
delay if the state is anything besides STATUS_EOF. This means that while
calling adjust_sync, the delay value could have changed from the audio
side of the equation from the previous playloop, and it doesn't
necessarily mean that the current audio_status is STATUS_PLAYING either.
So the old code would technically skip this case. In practice, this is
just one frame so it hardly matters, but it should be taken into
account. For example, STATUS_READY is definitely possible here in
adjust_sync. I'm not sure if it's actually possible for STATUS_SYNCING
to happen but the audio code can change add delay with that status, so
it doesn't hurt. STATUS_DRAINING is probably not relevant, but again
include it to mirror the audio code logic. Of course, STATUS_EOF is
obviously a no-no since that means no audio at all, so we return from
there. I didn't take hard measurements or anything, but this does seem
to result in slightly smaller av sync fluctuations on discontinuities
(like seeking) which makes sense because we're now making an additional
correction that wasn't previously done.
Another change is to always try adjust_sync as long as the frame_time is
nonzero. The old logic only did this if the video_status was playing or
greater, but it is possible to get new frames with a different PTS that
do not have that status. The audio is still playing so logically it
should be adjusted as well. Again, this happens for just one frame, so
it doesn't really matter in practice but it should make more sense. A
zero frame_time is skipped since that would mean the pts did not advance
and the previous playloop should have done the adjustment for that time
already.
3038e578af recently changed the logic here
so it wouldn't trigger on still images, but after thinking about the
code here some more, I don't believe it's needed at all. Doing an ao
reset when you flip the video track is very disruptive and not really
desirable at all. Since the ao no longer adds bogus delay values while
the video is off, there should be no need to do a full reset for syncing
reasons. The delay value will be zero, so we can let the audio just play
normally and let the video code do its thing. There is one slight trick
here however. When using a display sync mode, part of the syncing code
will try to update the audio and video playback speed. This can cause an
audio underrun if we're just turning the video back one. An easy way to
avoid most of these is to not update the speed if we are in the
STATUS_SYNCING state for video. This isn't quite perfect and underruns
are still possible, but it actually seems to depend on the AO. e.g. I
couldn't trigger an underrun with alsa but I could with pipewire. In any
case, the audio artifact here is much less noticeable than doing a full
ao reset, so it's still an improvement.
The point of the mpctx->delay field is for calculating a/v sync and
adjusting the frame timings appropriately. However, the frame_time was
always subtracted from mpctx->delay regardless of the audio status. This
meant that for a video with no audio, every single frame had a
subtraction with nothing ever added to it meaning that you get massive
negative delay times. For weird videos where the audio starts way later,
the massive delay leads to the VO sleeping for basically about as long
as the video was previously playing without audio. This results in
nothing being rendered during that brief period of time and just overall
badness. When using display-sync, it happens to work since the video
doesn't adjust itself based on audio and it renders anyway.
The fix is to simply not touch mpctx->delay in player/video.c unless
there's actually audio playing. This is what the rest of the code
already does aside from setting it to 0 on resets or EOFs. Move the
calculation into adjust_sync after the audio status check. It works
exactly the same as before except that we don't constantly subtract
bogus values when there's no audio playing. The reverse situation in
player/audio.c also has the same issue. For something that is only
audio, mpctx->delay is always added to but nothing will ever subtract
from it. It's not really clear if this particular version could ever
cause a real bug, but logically it needs to be guarded in the same way.
The field here should only be updated if the video is actually playing.
Fixes#12025.
vo_still_displaying() is racey with vo_request_wakeup_on_done() and above
that it doesn't work as expected. VO can run out of work and go to sleep
for 1000s, while the play thread still returns on vo_still_displaying()
check, because of a check `now < frame_end` so it never advances and go
to sleep itself.
This fixes dead lock that we have when image parameters changes during
playback.
This reverts commit 0c9ac5835b.
Fixes: #12575
Setting `--video-crop=0x0+0+0` applies full frame crop, ignoring the
container one. Setting --video-crop=0 disables manual crop and restores
container one if it is available.
When playing a sparse video stream, the debug log gets hit with the
video EOF constantly since the audio is still playing. There's no
practical use for this so just do add some logic to only signal it once
if it is sparse.
Kacper Michajłow