Skip to content

Commit

Permalink
DASH: Be more optimistic on the buffer depth of DASH contents
Browse files Browse the repository at this point in the history
Overview
--------

We were debugging some ad insertion usage, where an ad would be
dynamically inserted at a live content's start on the initial Manifest
request.

An application reported to us that while playing such content with the
RxPlayer, we were starting a few seconds into the ad instead of right
at the beginning of it. When looking at segment requests we saw that the
first media segment was never requested: we always started with the second
one instead.

Issue
-----

After investigation, I found out that one of the issue, was that the
`MPD@timeShiftBufferDepth` attribute of the produced stream was on the
first Manifest request exactly equal to the duration of the video segments
generated until now.

Yet the RxPlayer always interpreted that it could only play segments
whose ending timestamp is inferior or equal to the "live edge" (the live
position basically, which monotically increases as time increases) and,
more importantly here, whose starting timestamp is superior or equal to
that same live edge minus the `timeShiftBufferDepth`.

All this is not shocking, and other players have a similar interpretation
here, but I noticed that this was why we were not requesting the first
segment on the problematic content: as some milliseconds passed between
the time the Manifest is parsed and the time at which we began choosing a
segment to request, our "live edge" as further in the future (by the same
amount of milliseconds) and thus the minimum timestamp we can request will
also be further in the future - and ahead of the first segment position
by the same amount of milliseconds.

How to fix this
---------------

So the fix seemed to be to either raise the `MPD@timeShiftBufferDepth`
server-side or to raise the tolerance of what can still be requested
RxPlayer-side.

I dived into the DASH specification (5th edition) - which is a little
hard to interpret here - but I found the following passage:

> Assume the parameter availabilityTimeOffset is determined as the sum of all values of
@availabilityTimeOffset on all levels that are processed in determining the URL for the
corresponding segment. If the attribute @availabilityTimeOffset is not present, the value is of
availabilityTimeOffset is 0. Then for services with MPD@type='dynamic', the Segment
availability start time Tavail[i] for a Segment i in a specific Period is determined as
MPD@availabilityStartTime + PeriodStart + MediaSegment[i].startTime +
MediaSegment[i].duration - availabilityTimeOffset and the Segment availability end time is
determined as MPD@availabilityStartTime + PeriodStart + MediaSegment[i].startTime +
@timeshiftBufferDepth + 2*MediaSegment[i].duration

At the very end, it seems that we have to re-apply a segment's duration,
on top of the `timeShiftBufferDepth` to know when it is not available
anymore (they indicate "2 times the duration").

So I guess we may have been too pessimistic with how we previously
considered whether a segment could be requested or not.

How other players treat this
----------------------------

Because the people behind the ad-insertion tool told me they succeeded
to rely on the shaka-player, I looked at what they did on this issue.

It turns out that they have roughly the same interpretation that we had
for the minimum requestable position (`live edge -
timeshiftBufferDepth`) but their "live edge" was basically offset by `-
MPD@maxSegmentDuration`, so in the end they were removing the MPD's
maximum segment duration from the minimum requestable position
relatively to us.

If I had to guess why they do that, it may be that they clonflate the
"live edge" concept (the timestamp pointing to the current last media
data basically) and what in the RxPlayer we call the "maximum safe
position" (the last timestamp for which we're sure that a segment is
currently available) where the RxPlayer make those two very different
concepts and properties.
  • Loading branch information
peaBerberian committed Jul 16, 2024
1 parent 4faffb4 commit c453d11
Show file tree
Hide file tree
Showing 8 changed files with 68 additions and 34 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
availabilityStartTime: 0,
serverTimestampOffset: undefined,
});
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(undefined);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(3)).toEqual(undefined);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(undefined);
});

it("should return 0 through `getEstimatedMinimumSegmentTime` for a static content", () => {
Expand All @@ -21,10 +21,10 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
availabilityStartTime: 0,
serverTimestampOffset: 555555,
});
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(0);
manifestBoundsCalculator.setLastPosition(5555, 2135);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(4)).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(0);
});

it("should return 0 through `getEstimatedMinimumSegmentTime` if the `serverTimestampOffset` was never set nor the last position for a dynamic content with no timeShiftBufferDepth", () => {
Expand All @@ -34,9 +34,9 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
availabilityStartTime: 0,
serverTimestampOffset: undefined,
});
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(0);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(28)).toEqual(0);
});

it("should return `false` through `lastPositionIsKnown` if `setLastPositionOffset` was never called", () => {
Expand All @@ -47,7 +47,7 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
serverTimestampOffset: undefined,
});
expect(manifestBoundsCalculator.lastPositionIsKnown()).toEqual(false);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(undefined);
expect(manifestBoundsCalculator.lastPositionIsKnown()).toEqual(false);
});

Expand Down Expand Up @@ -86,9 +86,10 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
});
manifestBoundsCalculator.setLastPosition(1000, 10);
performanceNow = 25000;
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(1010);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(1010);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(50)).toEqual(960);
performanceNow = 35000;
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(1020);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(1020);
mockPerformanceNow.mockRestore();
});

Expand All @@ -104,18 +105,27 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
serverTimestampOffset: 7000,
});
manifestBoundsCalculator.setLastPosition(3000, 10);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(
7 + 5 - 4 - 3,
);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(4)).toEqual(
7 + 5 - 4 - 3 - 4,
);
performanceNow = 25000;
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(
7 + 25 - 4 - 3,
);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(10)).toEqual(
7 + 25 - 4 - 3 - 10,
);
performanceNow = 35000;
manifestBoundsCalculator.setLastPosition(84546464, 5642);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(
7 + 35 - 4 - 3,
);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(
7 + 35 - 4 - 3 - 5,
);
mockPerformanceNow.mockRestore();
});

Expand All @@ -132,10 +142,12 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
});
manifestBoundsCalculator.setLastPosition(1000, 0);
performanceNow = 50000;
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(1045);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(1045);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(40)).toEqual(1005);
manifestBoundsCalculator.setLastPosition(0, 0);
performanceNow = 55000;
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(50);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(50);
expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(40)).toEqual(10);
mockPerformanceNow.mockRestore();
});

Expand Down
4 changes: 3 additions & 1 deletion src/parsers/manifest/dash/common/indexes/template.ts
Original file line number Diff line number Diff line change
Expand Up @@ -557,7 +557,9 @@ export default class TemplateRepresentationIndex implements IRepresentationIndex
}

const { duration, timescale } = this._index;
const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime();
const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime(
duration / timescale,
);
if (firstPosition === undefined) {
return undefined;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -796,7 +796,9 @@ export default class TimelineRepresentationIndex implements IRepresentationIndex
if (!this._isDynamic) {
return;
}
const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime();
const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime(
(this._index.timeline[0]?.duration ?? 0) / this._index.timescale,
);
if (isNullOrUndefined(firstPosition)) {
return; // we don't know yet
}
Expand Down
10 changes: 7 additions & 3 deletions src/parsers/manifest/dash/common/manifest_bounds_calculator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,14 @@ export default class ManifestBoundsCalculator {
/**
* Estimate a minimum bound for the content from the last set segment time
* and buffer depth.
* Consider that it is only an estimation, not the real value.
* Consider that it is only an estimate, not the real value.
* @param {number} segmentDuration - In DASH, the buffer depth actually also
* depend on a corresponding's segment duration (e.g. a segment become
* unavailable once the `timeShiftBufferDepth` + its duration has elapsed).
* This argument can thus be set the approximate duration of a segment.
* @return {number|undefined}
*/
getEstimatedMinimumSegmentTime(): number | undefined {
getEstimatedMinimumSegmentTime(segmentDuration: number): number | undefined {
if (!this._isDynamic || this._timeShiftBufferDepth === null) {
return 0;
}
Expand All @@ -121,7 +125,7 @@ export default class ManifestBoundsCalculator {
if (maximumBound === undefined) {
return undefined;
}
const minimumBound = maximumBound - this._timeShiftBufferDepth;
const minimumBound = maximumBound - (this._timeShiftBufferDepth + segmentDuration);
return minimumBound;
}

Expand Down
16 changes: 15 additions & 1 deletion src/parsers/manifest/dash/common/parse_mpd.ts
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,7 @@ function parseCompleteIntermediateRepresentation(
args.referenceDateTime,
);
const timeShiftBufferDepth = rootAttributes.timeShiftBufferDepth;
const maxSegmentDuration = rootAttributes.maxSegmentDuration;
const { externalClockOffset: clockOffset, unsafelyBaseOnPreviousManifest } = args;

const { externalClockOffset } = args;
Expand All @@ -279,7 +280,6 @@ function parseCompleteIntermediateRepresentation(
manifestBoundsCalculator,
manifestProfiles: mpdIR.attributes.profiles,
receivedTime: args.manifestReceivedTime,
timeShiftBufferDepth,
unsafelyBaseOnPreviousManifest,
xlinkInfos,
xmlNamespaces: mpdIR.attributes.namespaces,
Expand Down Expand Up @@ -381,6 +381,20 @@ function parseCompleteIntermediateRepresentation(
// can go even lower in terms of depth
minimumTime = minimumSafePosition;
timeshiftDepth = timeShiftBufferDepth ?? null;
if (timeshiftDepth !== null) {
// The DASH spec implies that a segment is still available after a given
// `timeShiftBufferDepth` for a time equal to its duration
// (What I interpret from "ISO/IEC 23009-1 fifth edition 2022-08
// A.3.4 Media Segment list restrictions).
//
// This `timeshiftDepth` property is global for the whole Manifest (and
// not per segment), thus we cannot do exactly that, but we can take the
// anounced `maxSegmentDuration` by default instead. This may be a little
// too optimistic, but would in reality not lead to a lot of issues as
// this `timeshiftDepth` property is not the one that should be relied on
// to know which segment can or cannot be requested anymore.
timeshiftDepth += maxSegmentDuration ?? 0;
}
if (
timeshiftDepth !== null &&
minimumTime !== undefined &&
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/scenarios/dash_live.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -250,8 +250,8 @@ describe("DASH live content (SegmentTimeline)", function () {
});

expect(manifestLoaderCalledTimes).to.equal(1);
await checkAfterSleepWithBackoff(null, () => {
expect(player.getMinimumPosition()).to.be.closeTo(1527507768, 1);
await checkAfterSleepWithBackoff({ maxTimeMs: 2000 }, () => {
expect(player.getMinimumPosition()).to.be.closeTo(1527507763, 1);
});
});
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ describe("DASH live content multi-periods (SegmentTemplate)", function () {
const maxPos = player.getMaximumPosition();
expect(maxPos).to.be.closeTo(now, 2);
const minPos = player.getMinimumPosition();
expect(minPos).to.be.closeTo(now - manifestInfos.tsbd, 2);
expect(minPos).to.be.closeTo(now - manifestInfos.tsbd - 2, 2);
expect(manifestLoaderCalledTimes).to.equal(1);
});
});
16 changes: 8 additions & 8 deletions tests/integration/scenarios/dash_live_utc_timings.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1553521448, 3);
expect(player.getMinimumPosition()).to.be.closeTo(1553521446, 3);
expect(player.getMaximumPosition()).to.be.closeTo(1553521748, 3);
},
untilSuccess() {
Expand All @@ -56,7 +56,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
},
untilSuccess() {
Expand Down Expand Up @@ -95,7 +95,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1558791848, 3);
expect(player.getMinimumPosition()).to.be.closeTo(1558791846, 3);
},
untilSuccess() {
expect(player.getMinimumPosition()).to.equal(null);
Expand All @@ -119,7 +119,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
},
untilSuccess() {
Expand Down Expand Up @@ -158,7 +158,7 @@ describe("DASH live - UTCTimings", () => {
Date.now() / 1000 - manifestInfos.availabilityStartTime;
const minimumPosition = maximumPosition - timeShiftBufferDepth;

expect(player.getMinimumPosition()).to.be.closeTo(minimumPosition, 3);
expect(player.getMinimumPosition()).to.be.closeTo(minimumPosition - 2, 3);
expect(player.getMaximumPosition()).to.be.closeTo(maximumPosition, 3);
},
untilSuccess() {
Expand All @@ -184,7 +184,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
},
untilSuccess() {
Expand Down Expand Up @@ -218,7 +218,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1553521448, 3);
expect(player.getMinimumPosition()).to.be.closeTo(1553521446, 3);
},
untilSuccess() {
expect(player.getMinimumPosition()).to.equal(null);
Expand All @@ -242,7 +242,7 @@ describe("DASH live - UTCTimings", () => {
{ maxTimeMs: 1000 },
{
resolveWhen() {
expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
},
untilSuccess() {
Expand Down

0 comments on commit c453d11

Please sign in to comment.